Your agent will meet real customers soon. Meet it first.
Agent Shakedown stress-tests AI agents — voice first — before your customers do.
We run your agent through realistic scenarios, record what actually happens, and score it on the things that break in production: whether it finishes the job, whether it delivers the required statements, how it handles being interrupted, and how long people wait for it to answer.
Built for agents running on Retell, Vapi, or a custom stack.
Join the waitlistLaunching soon. Join the list and we'll tell you when the first shakedown slots open.
The problem this solves
Most agents get tested by the person who built them, with test calls that go exactly as planned.
Real callers don't do that.
They interrupt. They give half an address, then correct it. They say yes before you've finished the sentence. They ask the one question the script didn't cover. They go quiet.
A shakedown is a trial run under those conditions — before a paying customer is on the other end of the line.
How a shakedown works
1. You tell us what the agent is supposed to do
Its job, in plain language: book the appointment, take the order, qualify the lead, answer the questions, deliver the required disclosures, stay inside the policies you set. If it isn't written down, we help you write it down — an agent can't be scored against a job nobody defined.
2. We run scenarios against it
Scripted and unscripted calls that cover the normal path and the edges:
- The straightforward job, start to finish
- A caller who interrupts, talks over the agent, and answers early
- A caller who gives information out of order, or corrects themselves mid-call
- A caller who pushes on price, policy, or an exception the agent isn't allowed to grant
- A caller the agent should not try to handle — and should hand off or close out cleanly
Voice agents are called, not just prompted. What matters is what happens on a real call: the audio, the timing, the transcript.
3. You get a scored report
Every run is scored against the job definition, with the recordings and transcripts to back it up.
The report covers:
- Task completion — did the agent finish the job it was given, end to end?
- Fee and policy delivery — did it state the required prices, terms, and disclosures, completely and once?
- Interruption handling — when the caller talked over it, did it listen, adapt, and continue — or restart, repeat, and frustrate?
- Latency — response times reported as percentiles (p50 and p90), not averages, so the slow tail is visible.
- Fallback behavior — when it hit something outside its job, did it fail cleanly?
Each finding points at the moment in the call where it happened. No score without a receipt.
4. You fix, and re-run
Change the prompt, the settings, or the flow — then run the same scenarios again and compare reports. A shakedown you can repeat is how you know a change made the agent better instead of just different.
Don't want to do the fixing yourself? A Fix Pack will be available as an add-on: we implement the report's findings — prompt repairs, settings, flow fixes — and the verification re-run is included, so the improvement is proven, not promised.
Who it's for
Agent Shakedown is built for independent builders and small operators running customer-facing agents:
- You built a voice or chat agent for your own business, and customers will talk to it
- You build agents for clients, and you need to show your work before go-live
- You run an agent on Retell, Vapi, or your own stack, and the platform's built-in testing isn't telling you what a real caller will do to it
- You're about to change a prompt, a model, or a voice — and you want a before-and-after, not a guess
It is not an enterprise compliance suite, and it doesn't replace your platform. It sits in front of launch, and in front of every change after launch.
Why cross-platform
Your agent doesn't run on our platform, and the shakedown doesn't need it to.
Platforms test agents inside their own walls, against their own definitions. A shakedown is run from the caller's side, the same way your customers will meet the agent — on Retell, on Vapi, or on a custom build.
Where this comes from
Agent Shakedown grew out of a working lab that grades real agent calls — our own — scored on task completion, required-statement delivery, interruption handling, and latency. The scorecard on this page is the one we use. You can read it in full here: AI agent evaluation checklist
We are opening it up to other builders. That's what the waitlist is for.
Launching soon
Agent Shakedown is not open yet. We are finishing the first version and will open a small number of shakedown slots to the waitlist first.
Joining the list means:
- One email when slots open
- Early access in the order people joined
- No newsletter, no sequence, no sales calls
FAQ
What kinds of agents can be tested?
Voice agents first — that's where timing, interruption, and audio problems live. Chat, workflow, and personal agents are on the list too; if you run a personal assistant (Grok, Meta AI, Dots, Duet, or another), tell us which one in the form.
Which platforms?
Retell, Vapi, and custom stacks. If your agent can receive a call or a message, it can be shaken down.
What do I get at the end of a shakedown?
A scored report: task completion, fee and policy delivery, interruption handling, latency percentiles, and fallback behavior — each finding tied to a recording or transcript.
Can you fix what the report finds?
Yes — that's the Fix Pack add-on. We implement the fixes from your report (prompt repairs, settings, flow changes) and re-run the same scenarios to verify them. Fixes are bounded to what the report found; rebuilding an agent from scratch is not part of it.
Why not just test it myself?
You should. A shakedown adds the part self-testing usually misses: callers who don't follow your script, scored the same way every time, so runs can be compared.
When does it launch?
In the near future. The waitlist gets the date first, and gets in first.
What does it cost?
Pricing will be announced to the waitlist when slots open. Joining costs nothing and commits you to nothing.