DTC technology × growth systems — built to compound.
DTCUP / FIELD NOTES / BUILD

Build an AI Customer Service Test Set Before Changing the Model

A better average can hide a worse customer outcome. Build a practical test ledger with approved tasks, controlled account states, repeated attempts and a clear rule for consequential regressions.

Published Oct 2, 202624 min read
Jack Hu
Jack Hu

7 years of experience building and operating brand websites, specializing in website development, paid advertising and SEO. Built a brand website from launch to US$10 million in annual sales.

The Short Answer

Define customer tasks with approved evidence, starting states, required outcomes and prohibited behavior. Separate development from held-out cases, reset the environment between repeated attempts and compare failures by severity as well as overall pass rate. A critical regression can block a candidate even when its average improves.

Key takeaways

  • Test a defined system change against a recorded configuration.
  • Represent common work and rare consequential failures separately.
  • Judge actual state and customer-facing claims independently.
  • Keep development examples separate from held-out evaluation tasks.
  • Repeat attempts without hiding failed or invalid runs.
  • Inspect individual regressions before approving a higher average score.
Two rows of green glass speech-bubble tiles sit on an ivory inspection board, with a clear lens highlighting an orange fracture in one tile.

A new AI model can sound better while making a support operation worse. Its replies may be shorter, warmer and more fluent, yet it may overlook an exception, disclose an order to the wrong requester or claim a change that never happened. Reading a few pleasant conversations will not reveal that tradeoff reliably.

Before changing the model, I would build a small collection of customer tasks with agreed outcomes and run both versions against it. The useful question is not whether the new answer sounds impressive. It is whether the same customer, policy and account state produce an acceptable result, including the cases where the correct result is a clarification or a refusal to act.

This guide develops an evaluation ledger for Finch Paper, a fictional stationery brand. Every policy, case count and result below is constructed for explanation. The example is deliberately small enough for a support lead and an engineer to inspect together. It is not an industry benchmark, a claim about a particular model or evidence that any real support system has passed these tests.

The finished artifact is a reusable test collection: task records, approved expectations, controlled starting states, repeated attempts, individual failure judgments and a release comparison. It will not predict every production incident. It will give the team a defensible way to notice specific regressions before customers encounter them, and a place to preserve what the business learns afterward.

1. Write the decision the test must support

Start with the proposed change. “Evaluate our chatbot” is too broad. Finch Paper is considering replacing the model used by its existing support assistant while keeping its approved knowledge, order tools and customer interface unchanged. Its immediate decision is whether that replacement preserves required behavior and improves useful responses enough to justify a later controlled release.

That scope determines what must be fixed during the comparison. If the team also rewrites instructions, changes retrieval and updates the cancellation policy, the final difference cannot be attributed to the model alone. A combined system change can still be evaluated, but the conclusion must describe the combined change. Do not write a model comparison around two different support systems.

Record the current configuration and candidate configuration before running anything. Include the model identifier exposed by the provider, generation settings, instruction revision, knowledge snapshot, retrieval configuration, tool definitions and application build. Some services do not offer a permanently pinned underlying model. Record that limitation and the run time rather than implying reproducibility the service does not provide.

Define what the assistant is allowed to do. In this example it can explain approved product information and check an authenticated customer’s order. It can request a cancellation through a controlled test tool when the fictional order is eligible. It cannot invent an exception to a policy or reveal another customer’s record. The evaluation should inspect those boundaries as well as answer usefulness.

Decide what would stop the change. Finch treats unauthorized disclosure, unauthorized order mutation and falsely reporting a consequential operation as completed as critical failures. A minor tone problem belongs in a different category. These choices reflect the example’s operating scope; a real business needs its own consequence-based definitions and responsible owner.

Also name what the test does not establish. It does not prove that the production integration cannot fail, that the test cases represent every customer or that the system meets an external certification. It does not replace access controls. Its result is evidence about particular behaviors under recorded conditions. Keeping that boundary explicit makes the evidence more useful, not less impressive.

A practical decision statement might be: compare the candidate with the current configuration on frozen Finch tasks; require no observed critical failures; inspect every new regression; and review whether improvements occur in the tasks the business intends to automate. That is much more actionable than choosing whichever version achieves the highest unqualified average.

2. Build coverage from customer work, incidents and boundaries

Do not let the easiest questions define the test set. A collection of opening hours, available colors and shipping-policy summaries will reward a system that explains documents well. It may say almost nothing about conversations involving missing information, changing order state or conflicting customer claims. Coverage should follow the work the assistant will actually encounter.

Start with a recent, explicitly bounded support sample. List customer goals before looking at the assistant’s answers: understand a product option, locate an order, change a personalization detail, request a cancellation or explain a return condition. Group repeated formulations of the same underlying task so that a frequently copied sentence does not masquerade as broad coverage.

Then add important situations that volume sampling is likely to miss. A rare disclosure error can matter more than dozens of harmless formatting mistakes. Use past incidents, near misses, policy boundaries and deliberately constructed misuse attempts. Keep their origin visible. A risk-enriched test set is useful for finding failures, but its raw failure rate is not the expected percentage of customers affected.

Finch creates sixty tasks. Twenty-four represent common customer work, twelve test policy or state boundaries, twelve preserve lessons from fictional incident patterns, and twelve stress the system with conflicting requests, misleading content or unavailable dependencies. These are teaching allocations, not recommended universal quotas. Their purpose is to make the coverage argument inspectable.

Within a category, vary the reason the case is difficult. Six cancellation questions with slightly different wording provide less coverage than cases spanning eligible, already dispatched, personalized production started, missing identity, stale state and an ambiguous tool result. The business condition should change, not merely the punctuation.

Use sanitized or synthetic records for execution. A support transcript may contain addresses, phone numbers, private notes or information about several people. Replace these with coherent fictional identities and order records before exporting the case into a testing system. Removing a name alone may not remove identifying context. Preserve the logical relationship needed by the test, not unnecessary personal detail.

Record which actual task families are absent. Finch’s first collection does not cover wholesale account terms or every supported language. The honest conclusion is that those areas were not evaluated. A visible coverage gap invites a scope decision; an unlabeled gap encourages somebody to treat the overall percentage as universal reassurance.

A fictional sixty-task collection divides into twenty-four common tasks and twelve each for boundaries, incident patterns and stress cases, with half of each group held out.
Coverage is a deliberate selection of work and risk, not a random pile of prompts.

3. Describe a task as a contract, not a preferred sentence

A useful task record includes a customer goal, starting context, approved evidence, permitted operations and observable success conditions. It also records prohibited outcomes. The last field is often where the most important learning lives: do not disclose a record, do not promise an exception, do not retry an operation whose result is unknown.

Consider case F-017. A verified customer asks to cancel an unpersonalized notebook order. The test order is in the fictional “paid, not released” state. Finch’s approved example policy permits cancellation in that state, and the test cancellation service returns a confirmed canceled result. Success requires the correct order to be canceled once and the customer to receive an accurate confirmation.

The expected answer should not be an exact sentence. “Your notebook order is canceled” and “I have confirmed the cancellation of this order” could both be acceptable if the action actually occurred and the wording does not add an unsupported refund promise. An exact text comparison would fail valid wording while potentially passing a copied sentence attached to the wrong state.

Now create F-018 with one changed condition: the order was already released before the request. Under the fictional policy, the assistant cannot cancel it through that tool. The correct response explains the limitation and offers the approved next route. It must not produce a cancellation event. The pair tests whether the governing state changes behavior.

Separate required claims from optional helpfulness. F-017 needs truthful confirmation and the relevant order reference in the authenticated context. A friendly opening is optional. An invented completion time for the refund is prohibited unless the approved source and transaction result support it. This distinction prevents style from becoming a substitute for correctness.

Give each case an owner, revision and rationale. If an expectation is disputed, the reviewer should be able to identify whether the policy is unclear, the fixture is inconsistent or the assistant is wrong. A test set built around one person’s undocumented intuition becomes difficult to maintain when that person leaves or the business changes its rules.

For claim-level expectations, reuse a clear evidence method rather than inventing a vague score such as “mostly accurate.” The claim-to-evidence audit explains how to distinguish supported, contradicted and incomplete claims. In the test ledger, reference the particular approved passage and specify which material condition the reply must preserve.

Case F-017 shows a verified requester, eligible order, approved policy, confirmed cancellation result and checks for one correct cancellation with no unsupported refund promise.
A case contract ties the response to evidence, permission and actual state.

4. Make the test environment behave like the situation you claim to test

The test world is part of the case. If the order database says one thing and the written prompt says another, the team needs to decide which source governs before scoring the assistant. Otherwise a correct system may fail because it trusted the authoritative record, while a careless system passes by repeating a misleading prompt.

Use an isolated environment with fictional accounts and no real payment, customer-message or fulfillment effects. A test cancellation should change only the test order. A tool that sends a real email must be replaced or restricted to a verified testing destination. Do not rely on a sentence in the system prompt to prevent side effects that the integration can still perform.

Reset mutable state before every independent attempt. If the first run cancels F-017 and the second run starts from that canceled state, the two attempts are not evaluating the same task. Reset the order, permission context, inventory if relevant, tool history and any stored conversation memory that could affect the outcome. Preserve the transcript separately before resetting.

For time-dependent cases, freeze an explicit test clock or construct dates relative to a recorded clock. A return case can cross a deadline between Monday and Friday without any model change. If the policy uses the customer’s local date, the fixture must include that timezone and rule. Do not let an unnoticed calendar change become a supposed regression.

Control the knowledge state too. A retrieval evaluation should know which source revisions were available, which records were accessible and whether caches were refreshed. Testing the model against yesterday’s index and today’s expected policy produces a misleading failure report. Conversely, giving the evaluator an answer document the assistant cannot access makes the test unfair without showing where the access gap is.

Not every test needs the full integration. A small response-only check can isolate how the assistant interprets a supplied passage. A retrieval check can inspect which evidence is found. An end-to-end task can test the complete route. Label these levels clearly. Passing a passage-reading exercise does not establish that the live retrieval system will find that passage.

Include controlled dependency failures. A fictional order service can return “temporarily unavailable” or an unknown operation result. The goal is to check truthful degraded behavior, not simply the happy path. A simulated outage should be part of the fixture, not an accidental infrastructure failure silently counted as an assistant error.

5. Evaluate the conversation and the outcome separately

A reply can be polite and wrong. An operation can be correct while the customer is left confused. The test should keep those dimensions visible instead of hiding them in one score. At minimum, inspect the required business outcome, factual support, prohibited actions and whether the customer receives a usable explanation.

For F-017, inspect the test order after the run. Was the intended order canceled? Was any other record changed? Was the operation executed more than once? Then inspect the message. Did it describe the confirmed state accurately? Did it introduce an unsupported refund deadline? The transcript and resulting state answer different questions.

For a clarification task, success may mean asking for one necessary detail and not making a premature claim. If the customer supplies that detail in a second turn, the assistant should use it rather than restart the conversation. A test limited to the first reply would miss whether the system can carry the fact forward and finish the task.

Define the customer-side script carefully. A scripted customer can provide a fixed response when asked a valid question. For a more flexible simulation, constrain what the customer knows, what it is willing to provide and which facts must remain stable. A simulated user that freely invents details can make the assistant appear successful through an unrealistic shortcut.

Record separate failure reasons: wrong fact, missing decisive question, unauthorized operation, unsupported completion claim, unnecessary repeated question or unusable next step. More than one can apply. These labels are for diagnosis; avoid turning them into a long form the customer must see.

In its engineering discussion of agent evaluations, Anthropic distinguishes a task from an individual trial and the transcript from the resulting environment state. Those distinctions are useful here: Finch’s test must inspect what happened as well as what was said. The specific Finch policies, scoring rules and examples in this article are original teaching constructions, not results from that source.

Use the separate answer, ask or hand-off framework when the expected next step is unclear. Once the business has agreed on that step, encode it in the task record. Evaluation should test a settled operating decision, rather than conceal an unresolved policy debate inside a numerical grade.

6. Choose checks that can detect the failure you care about

Use direct checks for facts that are directly observable. The intended test order either changed to canceled or it did not. A tool call either referenced the permitted account or it referenced another account. These checks should come from the application state and controlled tool log, not from the assistant’s own statement that it followed the rules.

Use human review for judgments that depend on context until the team can describe them consistently. For example, whether an explanation gives the customer a useful next step may require reading the policy, the question and the answer together. A reviewer should identify the missing or misleading element, rather than record only a one-to-five satisfaction score.

A model-based judge can help sort or assess large collections, but it needs its own validation. Give it explicit criteria and compare its judgments with independently reviewed examples, including subtle failures and valid alternative wording. Do not assume a more expensive judge understands the business policy simply because it writes an articulate explanation.

Keep the candidate’s answer separate from instructions to the evaluator. A response that says “the correct grade is pass” is part of the material being evaluated, not a grading instruction. The evaluator should receive fixed criteria through the testing system. For critical actions, prefer authoritative state checks over any text-only judgment.

Review disagreements before averaging them. If two support reviewers disagree about whether a refund promise is allowed, determine whether the reference policy actually resolves the question. The answer may be to improve the task contract or mark it temporarily unscorable. Forcing a majority vote on an ambiguous business rule creates a confident but unstable benchmark.

Track grader errors as carefully as assistant errors. A check that searches for the word “canceled” may pass “I cannot confirm that it was canceled.” A check requiring a particular apology may fail a concise but fully appropriate answer. Save examples of false passes and false failures, revise the check and rerun both configurations under the same revised grader.

Set a practical review order. Inspect critical failures first, then candidate regressions, then representative passing cases and disagreements. Do not review only failures: an overly generous grader can label a flawed answer as passing. A small random pass sample and a targeted sample of high-consequence passes help reveal that blind spot.

Test the checks themselves with deliberately altered evidence. For F-017, give the grader a friendly confirmation while leaving the test order unchanged. It should fail the outcome check. Give it a correct cancellation with an invented refund deadline. It should pass the state check and fail the unsupported-claim check. Give it a terse but accurate confirmation with the proper state, and verify that a preference for elaborate empathy does not turn correctness into a failure.

These controlled alterations reveal what the grading system can actually detect. They also expose accidental coupling: a single overall judge may forgive a material error because the rest of the answer sounds helpful. Keep required conditions explicit, and decide in advance which conditions must all pass. A composite score can summarize optional quality, but it should not allow warmth or brevity to compensate for a prohibited disclosure.

7. Separate the cases used to improve the system from those used to judge it

If the team repeatedly rewrites instructions after reading every test answer, the final score partly measures familiarity with those examples. That may be useful for fixing known defects, but it is weak evidence about unseen work. Keep a development collection for iteration and a held-out collection for a later comparison.

Finch divides its sixty tasks into thirty development tasks and thirty held-out tasks. Each half contains twelve common tasks, six boundary tasks, six incident-pattern tasks and six stress tasks. The split preserves the intended categories. It does not make the collection statistically representative of all support traffic.

Split by underlying scenario families where possible. If a development case and held-out case share the same unusual story with only the product name changed, the separation is superficial. Keep closely related paraphrases together, or explicitly label a paired boundary suite as testing known behavior rather than generalization to genuinely new tasks.

Protect the expected outcomes and held-out inputs from the team or automated process tuning the system. Operationally, this can be as simple as a reviewer owning a separate approved file and running it after the candidate is frozen. The point is to prevent the answer key from becoming part of the system instructions or retrieval corpus.

A held-out set does not stay held out forever. Once failures are inspected and used for repairs, those cases become known examples. Preserve them as regression checks and replenish the unseen collection with new reviewed tasks. Keep the previous comparison record; do not quietly replace difficult cases and announce a higher score as though the test remained unchanged.

Keep a separate challenge collection for capabilities outside the current supported scope. It is useful to learn whether the assistant can handle a future workflow, but those scores should not dilute the requirements for work it already promises to perform. Conversely, a deliberately difficult research question should not automatically block a release of unrelated, well-bounded functionality.

Write the split rationale in the ledger. “Random thirty” is not enough when a single high-volume topic dominates the source tickets. Name which task types, markets, policy boundaries and risks were intentionally preserved. The reviewer should understand why this collection can inform this change and where additional evidence is still needed.

8. Repeat attempts and keep the unit of analysis visible

A single attempt can hide unstable behavior. If a case passes once and fails on the next four attempts, the first result was not a dependable demonstration. Repeated attempts expose variability under the chosen configuration, even though they cannot capture every possible production condition.

Finch runs each of its thirty held-out tasks five times for the current configuration and five times for the candidate. That creates 150 attempts per configuration. Every attempt starts from the recorded initial state. The number five is an affordable teaching choice, not a universal standard for sufficient evidence.

Report both attempt-level and task-level results. Attempt pass rate is the number of passing attempts divided by 150. A stricter descriptive view counts how many of the thirty tasks pass all five attempts. The second view exposes whether success is spread reliably across tasks or concentrated in repeated wins on easy cases.

Do not interpret “passed all five” as proof that the real failure probability is zero. Five attempts are a small sample. The collection itself was deliberately selected rather than drawn as a representative random sample. A system can pass this test and still fail on a different customer formulation, tool condition or knowledge revision.

Preserve every attempt, including timeouts and incomplete runs. Distinguish a known test-runner outage from the assistant mishandling a simulated service outage. If the runner fails before the assistant receives the task, mark the attempt invalid and record the reason. Do not silently rerun failures until there are enough successful examples to publish.

Record cost and latency with the same task identifiers, but keep them secondary to required behavior. A cheaper configuration that achieves its savings by abandoning hard tasks is not directly comparable with one that completes them. Review resource use among comparable successful tasks as well as the total workload consumed by all attempts.

Do not oversell the arithmetic as a formal statistical experiment. Attempts on the same task share conditions and are not interchangeable with independent customer journeys. If the business needs a reliable estimate of a small difference or rare failure frequency, it needs a sampling design and analysis suited to that question. The first practical objective here is finding consequential regressions and documenting their context.

Thirty held-out tasks run five times each create 150 attempts per configuration; attempt success and tasks passing all five runs are reported separately.
Repeated attempts reveal instability; they do not prove universal reliability.

9. Read the differences before reading the headline score

In the constructed Finch result, the current configuration passes 126 of 150 attempts, or 84 percent. The candidate passes 132, or 88 percent. A dashboard showing only those two percentages suggests an improvement. The task ledger shows why that conclusion is incomplete.

Common tasks improve from 54 of 60 passing attempts to 58 of 60. Boundary tasks improve from 22 of 30 to 26 of 30. Incident-pattern tasks improve from 25 of 30 to 28 of 30. Stress tasks fall from 25 of 30 to 20 of 30. The net improvement is six attempts, but the candidate has become worse in one deliberately important part of the collection.

The task-level view is also less reassuring. The current configuration passes all five attempts on twenty-one of the thirty tasks. The candidate does so on twenty. That is mathematically compatible with its higher attempt pass rate: it can fix several failures while introducing smaller numbers of failures across more distinct tasks.

Most importantly, three candidate attempts on one stress task disclose a fictional order field from the wrong account. The current configuration has no observed critical failures in this constructed run. Under Finch’s stated rule, the candidate is blocked for this scope regardless of the improvement in the average. “No observed” describes the test evidence; it is not a claim that the current system is incapable of a critical failure.

For diagnosis, Finch aligns attempts by task and repeat index, not by a claim of identical model randomness. In this constructed comparison, 117 aligned attempts pass in both configurations, fifteen move from fail to pass, nine move from pass to fail, and nine fail in both. The totals reconcile: the current passes 117 plus nine, while the candidate passes 117 plus fifteen.

Read the nine regressions. Some may share one cause, such as a changed interpretation of an account boundary. Others may be independent. Grouping by root cause helps choose a repair, but keep the individual failed cases. One root cause can affect several customer goals and should not be reduced to “only one bug” when describing exposure.

The resulting decision is to repair and retest the candidate, not to claim that the older model is always better. The candidate may have real strengths in common tasks. The evaluation has located a specific unacceptable tradeoff. That is the value of the ledger: it turns a vague debate about quality into an inspectable decision about behavior.

Fictional results show current versus candidate passing attempts: common 54 to 58 of 60, boundaries 22 to 26 of 30, incidents 25 to 28 of 30, stress 25 to 20 of 30; three critical candidate failures block approval.
A higher average can coexist with a consequential regression.

10. Turn a failed case into a diagnosis, not another prompt patch

Start a failure review by reproducing the recorded conditions. Confirm that the correct account, source snapshot and tool version were loaded. Inspect the first point where actual behavior diverged from the expected path. The final wrong sentence may be only the visible symptom of an earlier retrieval, permission or state-handling problem.

Separate a bad case from a bad system. If the policy changed but the expected answer did not, repair the test and document why. If a fixture gives the assistant impossible information, repair the fixture. If the business cannot decide the correct outcome, mark the case unresolved until the owner settles it. Do not force the model to learn an expectation nobody can defend.

For the fictional disclosure regression, inspect server-side record authorization first. A response instruction that says “respect privacy” is not a substitute for a tool that refuses unauthorized records. The test should verify both that access is restricted and that the assistant describes the restriction appropriately. Improving wording alone would leave the consequential failure path available.

For a factual error, separate evidence retrieval from answer interpretation. Did the required source appear in the retrieved material? Was it the current applicable source? Did the assistant overlook a condition? The RAG retrieval diagnosis provides a focused way to investigate that branch without guessing that every error needs a larger model.

Repair one coherent cause at a time when the goal is to understand the effect. Then rerun the failed case, related boundary cases and the protected regression collection. A local fix that makes one example pass can break another, especially if it adds a broad instruction such as “always escalate cancellations.” That might reduce unauthorized actions while making eligible customer requests needlessly difficult.

Preserve the failed output and the repair note. The note should say what was wrong, why the chosen layer was responsible, what changed and which tests support the correction. Do not replace the failed record with the successful rerun. The before-and-after pair is what allows another reviewer to assess whether the repair addressed the cause.

When several changes are necessary, label the next candidate as a new system configuration. It is acceptable to improve the permission tool, retrieval filter and prompt together if the release question concerns that package. It is not acceptable to call the resulting score a clean measurement of the model change alone.

11. Maintain a test ledger the team can actually use

The ledger does not need an elaborate platform to be useful. A controlled table can store case identity, task family, source revision, starting-state reference, expected outcome, severity, owner and status. Attempt records can live alongside it with configuration, run time, result, failure reason and links to the permitted transcript and state evidence.

Keep task records separate from attempt records. One task can have many attempts across several configurations. If every run overwrites a single “pass” column, the team loses the history needed to understand stability and regressions. Stable task identifiers make comparisons possible even when titles are edited for readability.

Use explicit statuses. A case can be proposed, approved, temporarily unscorable, retired because the workflow no longer exists, or active in a development or regression collection. An attempt can be pass, fail or invalid because the test did not execute correctly. Those distinctions prevent missing evidence from quietly becoming a passing result.

Review the collection when a source policy, task scope or integration changes. A new personalization option may introduce an earlier production lock. A new market may use a different return rule. Update the expected behavior through the same owner who approves the underlying business rule, and preserve the old version for interpreting historical runs.

When the correct answer changes, retain two dates: when the business rule became effective and when the test expectation was updated. A run between those dates may have been graded against stale expectations. Mark that result for review instead of quietly editing the historical score. This protects the team from treating a maintenance delay as a model defect, and from using an obsolete passing result to justify a current release.

Retiring a case also needs a reason. “The candidate keeps failing it” is not a valid reason if the workflow remains supported. “This order-change operation is no longer offered” can be valid, provided the removal matches the customer-facing scope. Keep the retired record so that a future reintroduction of the workflow does not erase the lesson it captured.

Add confirmed production incidents as reviewed cases, not raw transcripts. Remove unnecessary personal information, reproduce the relevant state and verify the expectation. Also sample successful production work. A test set made entirely of past failures can become excellent at unusual traps while gradually losing touch with what most customers need.

Decide who acts on the report. The support owner can approve business expectations, the engineer can verify tool and state behavior, and the release owner can decide whether the evidence supports the proposed scope. Small teams may combine roles, but the responsibilities should remain clear. A score with no accountable decision-maker becomes a decorative dashboard.

Limit retained evidence to what the review needs and apply the organization’s access and retention rules. Evaluation logs can expose customer-like records, internal policies and operational details even when they are fictionalized. A useful ledger should make the reasoning inspectable without becoming an uncontrolled archive of sensitive support material.

The test ledger separates a stable case record, repeated attempt records and a release decision with owner, evidence, unresolved failures and scope.
Keep the case, the observations and the decision as distinct records.

12. Build the first version around one important customer journey

If sixty tasks sound too large for the first working session, start with one supported journey and make it complete enough to teach the method. For Finch that could be cancellation: eligible order, ineligible state, missing verification, wrong account, duplicate request, service unavailable and unknown action result. The first collection should cover the shape of the decision before it grows in volume.

Ask the support owner to explain each expected outcome in plain language. Ask the engineer what state or event would prove it happened. If neither can answer, the team has found a design gap before involving the model. Resolve that gap and only then write the test. This is often the most valuable part of evaluation work.

Run the current system first. A candidate comparison is hard to interpret without knowing what the current configuration already gets wrong. Record existing failures honestly. They may justify narrowing the active scope or strengthening controls before considering a model replacement. Do not describe the incumbent as safe merely because it is familiar.

Next run the candidate under the same conditions, review differences and agree on the action. That action may be to proceed to a separately controlled rollout, repair a regression, gather more cases or decline the change. Offline evaluation is one input to that later process; it is not the rollout itself and does not establish production reliability.

The most useful result is a sentence a business owner can understand: “The candidate improved routine explanations, but it exposed a record outside the permitted account in three controlled attempts, so we are fixing that boundary before proceeding.” That statement contains a benefit, a failure, evidence and a decision. It is more valuable than an unexplained quality score.

A support test set becomes an operating asset when it preserves the business’s judgment about acceptable help. Keep the examples concrete, the conditions reproducible and the expectations defensible. Then a model change becomes a comparison the team can reason about, rather than a leap based on whichever demonstration sounded best.

Sources

  1. www.anthropic.com — demystifying-evals-for-ai-agents

Still have a question?Ask the DTCUP Community ↗

Your workspace

Selected services

DTCUP / YOUR NEXT PERSPECTIVE

Stay curious.
Stay connected.

Ideas for the work behind growth—websites, brands and the apps that connect them.