Before you run a single live AI calling campaign, write one sentence: what does a passing call look like?
A passing conversation and a passing call are different events. A conversation can sound fluent, warm, and complete while nothing true happened afterward. A passing call means three things occurred and left evidence: the agent used the approved introduction, captured the required information accurately, and completed the agreed next step in a connected record you can open and inspect.
That sentence becomes your acceptance standard. Everything else in the evaluation hangs from it.
Why the Sentence Comes First
Deployment numbers show why this matters. Research on voice agent monitoring found that 87% of companies have deployed AI voice agents, yet only 12% report satisfaction with deployment quality. The difference sits in acceptance criteria. An agent that talks well clears the demo. An agent that executes to a written standard, with verifiable outcomes, clears production.
Scale sharpens the stakes. At 200 calls a month, a 1% failure rate means two bad calls. At 200,000 calls, it means 2,000 real people having a bad experience. Define the standard before volume arrives, because volume multiplies whatever you launched with.
Score Three Things Separately
Blend the scores and you lose diagnostic power. A single pass/fail grade tells you something broke without telling you what. Keep three independent scorecards.
1. The Conversation
Did the agent follow the approved task? Check the introduction, the questions asked, the accuracy of captured answers, and the handling of refusals and contact restrictions. The introduction is measurable, and it affects outcomes directly. Disclosing AI was associated with roughly 20% fewer hang-ups in the first ten seconds, and mentioning recording with roughly 30% fewer. Transparency belongs in your acceptance criteria as a tested behavior, with evidence in the recording.
2. The Audio
Listen to the recording while following the transcript. A transcript alone hides clipped speech, talk-over, unnatural pauses, and repeated introductions. Mark audibility verified only after a person has heard the call. Transcript text confirms words. It never confirms experience.
3. The Action
Open the connected record. Confirm the attendee, the selected time, the booking status, the follow-up state. The agent saying a meeting was booked is a claim. The calendar entry is evidence. When the record is missing or inconsistent, mark the result unresolved and assign an owner to investigate.
Never Mark Complete Without Evidence
This rule holds the entire framework together. A result reported as complete, without a record proving it, is the most expensive failure type in outbound calling. It poisons your funnel numbers, your forecast, and your expansion decision all at once.
Build the discipline into the workflow:
- Every scenario gets a specification. Input conditions, expected behavior, configuration version, and links to evidence.
- Every status is one of four states. Passed with evidence, failed with evidence, unresolved, or not yet tested. Nothing else exists.
- Every denominator is honest. If a count is zero, report zero. An undefined ratio is more useful than an invented one.
- Every configuration change triggers a recheck. A prompt edit, a model swap, or a new voice resets acceptance for the scenarios it touches.
Agree on Blockers Before Launch
Write the stop conditions before the pilot starts and name the person with authority to halt activity. Reasonable blockers include wrong caller identity, ignored contact restrictions, invented availability, repeated speech loops, and critical actions reported complete without evidence.
Setting these in advance removes negotiation under pressure. When a blocker fires mid-campaign, the decision was already made. The team executes it.
Take the Framework Into Procurement
The same sentence that governs your pilot governs your buying process. Bring the acceptance definition to every vendor demonstration and ask to see the evidence paths for your specific calling job. Ask which recordings, transcripts, and connected records the workspace retains, and who owns exceptions when the evidence is missing.
A vendor comfortable with that question has built for inspection. A quiet-room demo answers whether the agent can talk. Your written standard answers whether the system can execute, and execution you can verify is the only kind worth scaling.
Write the sentence. Score the conversation, the audio, and the action separately. Accept nothing without a record. The campaign that survives that discipline deserves your budget.