To test an AI agent, give it a set of real tasks with known good outcomes, score every run, and run the whole set again after each change to the prompt, the model or a tool. To monitor it after launch, record every step it takes (a trace), watch cost, speed and failure rates, and have a person review a sample of conversations each week. People call this practice AgentOps, after DevOps, and AgentOps is also the name of one of the tools.
An agent is harder to test than ordinary software because it chooses its own steps. Ask it the same thing twice and it may call different tools in a different order and still reach the right answer, or reach a wrong one with great confidence. This post covers what to test before launch, what to watch after it, and which tools fit each job.
Two different jobs that share a name
Searches for "AI agent testing" mix two things, so it helps to separate them first.
The first is testing your agent: checking that the agent you built answers well, uses its tools correctly and stays safe. AgentOps and Hamming do this.
The second is using an agent to test your software: an AI that clicks through your web app or calls your API the way a QA engineer would, and reports what broke. Momentic, Kusho, Posium, Owlity and TestDriver do this. Both are useful, and many teams end up with one of each.
Before launch: build a task set
Collect 30 to 50 real tasks from the work the agent will do: support tickets, booking requests, research questions. For each one, write down what a good outcome looks like. Some outcomes are exact ("the refund is issued for order 1042"). Others need judgement ("the reply is polite, correct and under 120 words").
Score exact outcomes with code. Score judgement outcomes with a written rubric, applied by a person or by a second model acting as a judge. If you use a model as the judge, have a person score 20 of the same runs and compare, so you know how far to trust it.
Add the hard cases on purpose: a missing order number, an angry customer, a request outside the agent's job, and an attempt to make it ignore its instructions. Prompt injection is the most common attack on agents that read web pages, emails or documents, so include a document with hidden instructions in your set.
Run the set after every change
Changing one sentence in a prompt, switching to a newer model or adding a tool can fix one task and break three others. Run the full task set after every change, compare the scores with the last run, and read the tasks whose score dropped. Keep the results, so you can see the trend over months.
Run each task more than once. An agent that passes four times out of five has a problem that one run would hide.
After launch: trace every step
A trace records what the agent saw, which tools it called with which inputs, what came back and what it answered, with the time and cost of each step. When a customer complains, the trace shows where it went wrong.
OpenTelemetry, the open standard most monitoring tools use, has conventions for recording model and agent calls, so your traces can move between tools. On top of traces, watch four numbers each day: task success rate, cost per task, time per task, and how often the agent hands over to a person. Set an alert when any of them moves sharply.
Then read. Pick 20 conversations a week at random, plus every one a customer rated badly, and add the failures to your task set. That loop is what makes the agent better over time.
Tools for each job
Tool | What it does | Fits best when |
|---|---|---|
Traces, costs and session replays for agents | You built an agent in code and want to see every step | |
Simulated calls to test voice agents, plus production monitoring | You run a voice agent and need to test hundreds of calls | |
AI testing for web apps, with a low-code editor and tests that adapt as the UI changes | Your QA team wants regression tests that survive redesigns | |
An AI agent that turns a description into ready-to-run tests for web interfaces and APIs | You want a test suite fast, front end and back end | |
AI agents that map user journeys and write end-to-end tests for web and mobile | You want tests generated from how users move through your app | |
Autonomous AI-driven QA, from test design to bug finding | You want the whole QA process automated | |
A QA agent that uses AI vision to test browsers, desktop apps and more | You test more than a browser | |
An open source framework for browser automation with AI steps | Your developers prefer to write the tests in code |
Kusho and Owlity each have a 4.3 rating on AI Agents Live, and Momentic has a 4.0 rating.
A plan for your first two weeks
- Write 30 real tasks with good outcomes, and score your agent on them today. That is your baseline.
- Add tracing, with a tool or with OpenTelemetry, before the next release.
- Run the task set before every change goes live, and block the release if the score drops.
- Read 20 real conversations every Friday and add the failures to the set.
Questions people ask
What is AgentOps?
AgentOps is the practice of running AI agents in production: testing them before release, tracing what they do, watching cost and quality, and improving them from real failures. It is also the name of a tool that records agent sessions for developers.
What is the difference between Momentic and Stagehand?
Momentic is a testing product: you build tests in its low-code editor and it runs them against your web app, adapting them as your interface changes. Stagehand is an open source framework that developers use in code to drive a browser with AI steps, for tests or for any other automation.
How do you test a voice agent?
Simulate calls at scale with different accents, noise, interruptions and goals, score each call against what should have happened, and listen to a sample yourself. Hamming is built for this.
The Software Testing category on AI Agents Live lists more testing agents with their reviews.