Talk to us
Glossary · AI

What is Evaluation (Evals)?

Definition

Evaluations, or evals, are repeatable tests that measure how well an AI system performs a task, using a set of realistic inputs with known good answers or scoring criteria. Evals can be scored by exact match, code checks, human reviewers or another model acting as a grader. Teams run them before launch and before every prompt, model or workflow change.

Why it matters for your business

Without evals, nobody can say whether an AI system is getting better or worse, and every change is a guess. A modest set of real examples, including tricky edge cases and questions the system should refuse, turns subjective impressions into a pass rate you can track. Evals make it safer to switch to a newer or cheaper model, because you compare results on your own tasks rather than on public benchmarks. When buying or commissioning AI work, ask to see the eval set and the results.

Example

Before launching an invoice extraction workflow, an operations team assembles 200 past invoices whose correct values are already in the accounting system. Each prompt or model change is scored on how many fields match exactly. When a newer, cheaper model is released, the team runs the same set and switches only if it matches the current model's accuracy.

Keep reading

Next steps

01 Browse free n8n templates Fifty starter workflows you can import today. No email required. 02 Take the 5-minute AI readiness assessment Get a score and a recommended first project in your browser. 03 Get a written scope and estimate Tell us what you want to automate. We reply within two business days.