Evals Before Agents: The Regression Suite Is What Makes an AI Feature Shippable
An AI feature without a regression suite is a demo with a deploy pipeline. It works on the day you present it and drifts on a Tuesday in week 6, and you find out from a customer rather than from CI.
Every retail AI project we join arrives at the same fork. The prototype is convincing, the stakeholder wants it live before peak season, and nobody can answer the only question that matters: how do you know tomorrow's version is not worse than today's.
The answer is a golden set and a grader, wired into CI, built before the agent. That ordering is unpopular because evals feel like overhead and the demo feels like progress. It is still the ordering that gets features shipped instead of quietly retired 4 months later.
Why the regression suite is the shippability gate
Traditional software fails loudly. A null pointer throws, a schema mismatch 500s, and the alert fires. AI features fail quietly, producing well-formed output that is wrong in a way no exception handler will catch.
That is the entire argument. Without a suite that scores output against known-good answers, the only detector you have is a customer complaint, and by then the damage is a support ticket, a refund, or a compliance question.
Anthropic's guidance on defining success criteria and building evaluations is blunt about this: criteria must be specific and measurable, and the example it gives is "an F1 score of at least 0.85 on a held-out test set" rather than "good accuracy". If your success criterion cannot be computed, it is a hope.
What a golden set actually is
A golden set is a versioned collection of examples. Each example holds an input, a reference output or an acceptance predicate, and metadata you can slice by later.
The metadata is the part teams skip and then regret. Order type, locale, channel, customer tier, message length, catalog category. Without those fields you can compute one aggregate score and nothing else.
The same shape appears in LangSmith's evaluation model, where a dataset is a collection of examples, an evaluator scores application output, and an experiment is one version of the application run against one dataset. Whatever tool you pick, insist on those 3 nouns.
What one row of a golden set looks like
Concreteness helps here more than another paragraph of principle. A returns-classification example, stored as one line of JSONL in the repository, looks like this.
{
"id": "ret-0142",
"input": {
"message": "the boots arrived scuffed on the left toe, i still want them but not at full price",
"order_id": "SO-88213",
"days_since_delivery": 6,
"item_final_sale": false
},
"reference": {
"outcome": "partial_refund_keep_item",
"reason_code": "DAMAGED_IN_TRANSIT_MINOR"
},
"slices": {
"order_type": "standard",
"locale": "en-US",
"channel": "web",
"money_movement": true
}
}
The grader for this row is 6 lines of code comparing 2 enum fields. It runs in microseconds, needs no model, and it will still be correct in a year.
The slices block is what makes the suite worth keeping. When a release drops the pass rate by 4 points, the per-slice report tells you it was gift orders, and you have a hypothesis before you have a meeting.
Golden sets for real retail workflows
Abstract advice about evals is why teams do not build them. Here is what the first 150 examples look like for the AI features retail teams actually ask for.
| Workflow | Example input | Reference | Grader |
|---|---|---|---|
| Product attribute extraction | Supplier description plus 3 images | Expected attribute object | Exact match on required keys, set overlap on optional |
| Returns classification | Customer message plus order record | Policy outcome and reason code | Deterministic enum match |
| Product copy generation | SKU attributes plus brand rules | No single reference; a rubric | Banned-term check, claim-invention check, length |
| Search query rewriting | Raw query plus catalog slice | Expected top-3 SKU set | Recall at 3 |
| Order-status answering | Question plus order graph | Required facts | Model-graded factual containment |
Notice how few of these need a model to grade them. 3 of the 5 are deterministic, which means they run in seconds, cost nothing, and never drift themselves.
Graders, in cost order
- Deterministic code. Exact match after normalization, schema validation, enum membership, regex assertions. Fastest, cheapest, and the grader itself cannot regress.
- Statistical. Cosine similarity over embeddings, ROUGE-L for summarization overlap, recall at k for retrieval. Useful where output is constrained but not exact.
- Model-graded. A judge model scoring against a rubric or a reference, either reference-free or reference-based. Flexible, expensive, and it needs calibration before you trust it.
- Human. Annotation queues and structured review. The most expensive option and the only one that defines what correct means in the first place.
A judge you have not validated against human labels is a random number generator with good manners. Label 50 examples by hand, run the judge on those same 50, and report agreement before the judge is allowed to gate anything.
If judge-human agreement is below about 80%, fix the rubric rather than the model. Most disagreement comes from a rubric that is ambiguous to a person too.
The pass bar is per-slice, never global
An aggregate pass rate of 92% is the most comfortable number in AI engineering and the least informative. It hides the slice where returns classification on gift orders runs at 61%.
Set a floor per slice and a floor overall. Ship when both clear, and treat any slice that falls below the floor as a blocking bug even when the aggregate looks fine.
Slices worth carrying from day 1 in retail: order type, locale and language, channel, customer tier, and whether the case involves money movement. That last one is the slice your risk team will ask about.
Drift is 3 different failures sharing one word
| Kind | What changed | Detection | Response |
|---|---|---|---|
| Model drift | The vendor shipped a new version, or you upgraded | Pin the version; re-run the full set on every bump | Gate the upgrade behind an eval run, never behind a date |
| Data drift | Inputs moved: new category, seasonal phrasing, new locale | Track input distribution and per-slice pass rate in production | Promote production examples into the golden set weekly |
| Policy drift | Your business rules changed and the expected outputs did not | Tie eval review to policy change control | Fail closed; a stale reference is worse than no reference |
Policy drift is the one that bites hardest in retail, because return windows, promotional rules and shipping cutoffs change several times a year. The golden set has to be owned by whoever owns the policy, not only by engineering.
Practically, that means the eval dataset lives in version control with a code owner from operations. A policy change that does not update the expected outputs fails review.
Where the human approval gate belongs
The gate question has a clean decision rule. Automate when the action is reversible, cheap to undo, and obviously wrong when it is wrong. Gate when the action moves money, permanently changes a customer record, or fails invisibly.
The OWASP Top 10 for LLM Applications names excessive agency and overreliance as separate risks, and both describe a missing gate. Excessive agency is the system being allowed to act; overreliance is the human waving it through without reading.
A gate that approves 100% of what it sees is not a gate. Measure approval rate at the gate, and if it exceeds roughly 98% for a sustained period, either the gate is theater or the automation has earned a wider bound.
On one AI product-description build, first-draft editor approval ran comfortably ahead of the target agreed before the work started.
The approval rate was not the interesting number. Being able to measure it per category, before merchandising committed 3 weeks of review capacity, was.
Start with a workflow, not an agent
Anthropic's building effective agents draws the line by predictability. Workflows suit well-defined tasks with known subtasks and give you consistency; agents suit open-ended problems where the number of steps cannot be predicted.
Most retail AI requests are workflows wearing agent clothing. Attribute extraction, returns triage, copy drafting and query rewriting all have a knowable step count, and a workflow with a fixed graph is dramatically easier to evaluate.
The article's own advice is to add complexity only when it demonstrably improves outcomes. Demonstrably means measured against a golden set, which is the argument for building the evals first, restated by the model vendor.
Wiring it into CI without a 40-minute pipeline
Three tiers keeps the feedback fast and the coverage honest.
- Pre-merge: a 40-example smoke subset, deterministic graders only, under 3 minutes. It catches prompt edits, schema breaks and obvious regressions.
- Nightly: the full golden set including model-graded evaluators, with a per-slice report and a diff against the previous run.
- Pre-release: full set, plus a human spot-check of 20 sampled outputs weighted toward the slices closest to their floor.
Store every run. An experiment history is what lets you answer "when did this get worse" without re-deriving it from memory, and it is the artifact an auditor will ask for.
The NIST AI Risk Management Framework organizes around Govern, Map, Measure and Manage. Measure is the function most teams skip, and it is the one that turns an AI feature from a pilot into something a risk committee will approve.
What we would keep if the budget halved
Keep the golden set and the deterministic graders. Drop the judge model, drop the dashboard, drop the vendor platform, and run the suite from a script in CI.
150 well-chosen examples with exact-match grading catch the majority of real regressions. Everything above that is refinement, and refinement is a poor reason to delay the thing that catches the regression.
FAQ
How big should the first golden set be?
100 to 300 examples covering every slice you care about. Coverage across slices matters far more than volume within a slice, and automated grading makes it cheap to grow later.
Can the model generate its own golden set?
It can draft candidate inputs. Each expected output still needs human sign-off, because a set graded against the model's own idea of correct will pass forever and detect nothing.
What pass rate is good enough to ship?
There is no universal number. Set it from the cost of a miss: a wrong product tag is cheap, a wrong refund decision is not, and the floor should reflect that difference per slice.
Do we need a vendor eval platform?
Not to start. A dataset file, a grader function and a CI job cover the first 6 months. Buy tooling when the human annotation workflow becomes the bottleneck.
How often should the golden set change?
Weekly additions from production traffic, and an immediate update whenever a business policy changes. Treat it as a living artifact with an owner and a review cadence.
Where does prompt injection testing fit?
As its own slice inside the same suite, with adversarial inputs drawn from the OWASP list. It runs pre-merge because it is deterministic and because a regression there is a security incident.