Prompt engineering · training for AI and dev teams
Prompt engineering training: from “it seems to work” to a score
Codemation is a Dutch company that teaches teams to work with AI. This prompt engineering training is for AI and dev teams that build an LLM into a product: you learn to measure prompts with a test set and evals, instead of judging by feel. It is not a course in ChatGPT tips for the office. Chris Blokland gives the training on site or online, in Dutch or English. We set the programme, length and price together in a free 30-minute intake.
The programme
Six parts, from gut feeling to measuring.
We work on a prompt from your own product. In the intake we choose it together, and tune the depth to what your team already knows.
- Part 1
From “it seems to work” to a measured prompt
Why trying and looking is not enough, and what a first measurement looks like.
- Part 2
Building a test set from real cases
How you collect cases from real use, what makes a good case, and how the set grows from the mistakes you meet.
- Part 3
Evals and LLM-as-judge
How you have a model judge whether an answer is good, and where that goes wrong.
- Part 4
Calibrating the judge against people
How you check that the LLM judge agrees with people, by having people label blind.
- Part 5
Tracing and prompt versions
Seeing what happened in each step, and versioning prompts so you know which version got which score.
- Part 6
Comparing models on quality and cost
The same test set across several models, then choosing on quality and cost.
Who it is for
For AI and dev teams.
The training is for teams that build or run an LLM feature in a product or an internal tool and want to know whether a change is really better.
- AI and dev teams with a prompt or agent in a product
- Tech leads who want prompts to have a test set and a score, the way code has tests
- Not for office staff who want to chat better with AI: the AI Kickstart is for that
How it works
How it runs.
On site or online
At your office, or online in consultation. For a team.
Dutch or English
The training is in the language of your team.
Length and price at the intake
We set the programme, length and price together with you in a free 30-minute conversation.
Your trainer
Training from someone who does this every day.
Chris Blokland is the founder of Codemation and has been programming for 23 years. For the platform's own agents he runs evaluation pipelines: test sets and eval lanes, an LLM judge he calibrates against people who label blind, tracing in Langfuse, and comparing models on cost and quality. What he teaches comes from that work. Chris on LinkedIn.
Questions we often get
Good to know
How is this different from an ordinary prompt engineering course?
Many courses teach you to write better prompts for ChatGPT. This training teaches you to measure prompts: a test set, evals and a score, for a prompt in your own product. That way you see whether a change is really better.
What is an eval?
An eval is an automated measurement of how well a prompt or agent does on a set of cases. You run it after every change, and see whether the score goes up or down.
What is a test set and how big should it be?
A test set is a collection of real cases with what you count as a good answer. You start small, with cases from real use, and let it grow with the mistakes you meet. We discuss the size in the intake.
Can an LLM judge its own output?
As a judge (LLM-as-judge) a model can score answers, but you have to check that it agrees with people. Calibrating that is part of the training.
Which tools do you use?
Chris himself works with test sets and eval lanes and with Langfuse for tracing. The training is about the way of working; which tools you use we discuss in the intake.
Do we work on our own product?
Yes. The aim is that one prompt in your own product has a test set and a score.
What does the training cost and how long does it take?
That is not fixed yet. We set the programme, length and price together with you in the free 30-minute intake.
Do we need to be able to programme?
The training is for AI and dev teams, so at least someone who works with code or APIs. For teams without technical knowledge there is the AI Kickstart.
Last updated:
Free intake · 30 minutes
Does this suit your team?
In half an hour we look at your prompt or agent and what you want to be able to measure. No obligation. Choose a time below.