AI QA agent: what it does and what stays human
An AI QA agent is software that does QA work rather than assisting with it. Given a plain-language description of what your app should do, it runs the check itself in a real browser, reading each page at run time and choosing its next action from what it finds, then hands the recorded run to a separate judge for a verdict. It executes everything. It decides very little.
The word agent is carrying real weight in that definition. In the sense Anthropic's engineering guidance uses it, an agent is a model directing its own actions in a loop toward a goal, not a script replaying steps somebody recorded. We build TestAutomate, whose working half is exactly that kind of agent, so we're not neutral about the term. What we can be is specific. This page walks through what the worker does across a QA workflow, where its authority stops on purpose, and what that leaves for the engineers it works beside.
How does an AI QA agent run a regression check?
In a real browser, against the real app, with nothing scripted in between. A regression check here is two declarations: a plain-language prompt describing the flow, and an expected outcome describing what should be true afterward. Everything between those two is supplied at run time. The agent operates the app through the same interface your users do, in a bounded read-decide-act loop that ends when the flow is done or truly stuck, never in an infinite hang.
The interesting parts are the absences. There's no stored locator behind any of this. The agent reads the page and finds elements the way a person would, so there are no selectors to maintain, and a relocated control changes the agent's route rather than breaking your test. And the agent's opinion of how the run went isn't the result, because a separate judge grades the recorded run against the outcome you declared. For that mechanism a step at a time, how a computer use agent turns a described flow into a graded verdict takes the loop apart from the inside.
What does a QA agent verify that other checks miss?
The layer your user actually inhabits. A backend can keep every contract it made while the screen in front of a real person quietly stops working, because a working product is more than a stack of correct responses. Checks that stop below the browser can't see that gap. An AI QA agent works above it by construction. It exercises the rendered app end to end, so a pass means the flow worked in the place users experience it, not merely in the layers underneath.
None of that makes the lower layers redundant. Fast checks below the browser should stay exactly where they are, and the agent's job is the slice of reality they can't reach. The full argument for that split, green dashboards and stuck users included, is made from a practitioner's seat in why browser agent testing catches what API suites miss.
Can an AI agent for QA draft the tests too?
Yes, and the drafts come from an unglamorous source, your documentation. Point TestAutomate at a docs site and it crawls the pages, extracts the articles, and classifies them into the procedures your product says it supports. A URL alone is enough to start. Nothing gets drafted from all that until the first time someone hits Run suite, at which point the procedures are clustered into topics and the strongest model drafts one to three end-to-end flows per topic.
Every draft is plain natural language by rule. The generation prompt is blunt about it: "The prompt and expected_outcome MUST be plain natural language. Do NOT include code, JSON, YAML, or selectors." Drafting is then followed by pruning, because generated tests have a failure mode of their own, the test that fails for reasons other than a defect. A deterministic lint drops any flow that assumes state some earlier test left behind, or that grades against a resource nothing guarantees, on the code's own principle that a bad test is worse than a missing one. What survives is a candidate in the honest sense of the word. It's readable, it's reviewable, and it hasn't earned trust yet. That's what running it is for.
What happens when a QA agent thinks it found a bug?
It writes the finding down and stops. A judged failure goes to triage first, where anything that looks like a test or environment problem rather than a real application defect gets filtered out. What remains becomes a pending bug candidate carrying the judge's reasoning and an itemized account of which expectations went unmet, which is the difference between a report you can act on and a red X you have to reproduce. A run that was blocked by its environment never reaches this stage at all, so a broken test bed can't manufacture defects.
Pending is the operative word. The agent can't file a ticket, full stop. Filing is a person clicking File in the dashboard, and before that click creates anything, the candidate is checked against a pool of up to fifteen similar existing issues. If a strict judge decides one of them is the same defect, you're shown the duplicate instead of a new ticket. The agent's ambition ends at a well-argued draft. Your tracker stays yours.
How much should an autonomous QA agent decide on its own?
Less than it technically could, and the gaps are load-bearing. Inside a run, the agent has full authority over its route. At the run's edges, authority changes hands. The agent may conclude that a run met its declared expectations or didn't, and it may conclude that the run never validly happened at all. A test whose precondition couldn't be established comes back blocked, an environment problem on the record rather than a red mark against your app. And when the app changed in a way that still satisfies the intent, a renamed control, a dialog that wasn't there last week, the run passes and the difference is flagged for a person to confirm it was deliberate.
What the agent may not do is act on any of those conclusions beyond recording them. Confirming a flag, filing a candidate, deciding what deserves a test in the first place, deciding what correct even means, each of those waits for a human. Where every one of those lines sits, and why they hold when a run gets weird, is a subject of its own. The guardrails that decide what a QA agent may conclude on its own draws the full map.
What stays human when a QA agent joins the team?
The judgment, which was always the actual job. Testing has a useful old distinction, sharpened by James Bach and Michael Bolton, between testing and checking. Checking is running a defined observation against a defined decision rule, and it can be mechanized. Testing is the wider act of learning the product, questioning it, and deciding what matters, and it can't. An AI QA agent is a tireless checker with a very good notebook. It doesn't know your users, your risk appetite, or which of two correct-looking behaviors is the one you meant to ship.
Drawn as a division of labor rather than a body count, the split looks like this.
| The work | The AI QA agent | The humans |
|---|---|---|
| Deciding what deserves a test | Nothing | Own it outright |
| Drafting new tests | Proposes plain-language flows from your docs | Review what it proposed |
| Running the regression pass | Runs every flow in a real browser | Read verdicts instead of traces |
| Grading a run | Hands the recorded run to a separate judge | Spot-check the reasoning on close calls |
| A change that still works | Passes the run and flags the difference | Confirm the change was intended |
| A failure worth reporting | Stages a pending candidate with the reasoning attached | Click File, or don't |
The middle column flatters the agent if you read it lazily, so treat the right-hand column as the load-bearing one. Drafted tests inherit the quality of the docs they came from, and a flag queue nobody reviews is just a pass with extra steps. The honest claim isn't that QA work disappears. It's that the hours move up a level, from performing checks to deciding what the checks should be and what their results mean. Execution moves. Judgment stays.
How does TestAutomate's AI QA agent work?
As the worker inside a wider practice. In TestAutomate, you describe a flow and its expected outcome in your own words, our agent runs it in a real browser, and a separate model judges the recorded run against what you declared. The practice around that worker, the verdict vocabulary, the retry rule, the question of where determinism lives, has a page of its own. What agentic testing is and how it works covers the full mechanism, and the same page covers the no-markup, bring-your-own-key cost model. TestAutomate is pre-launch.
Frequently asked questions
What is an AI QA agent?
An AI QA agent is software that performs QA work itself: it takes a plain-language description of expected behavior, drives the real application in a browser to check it, and records what happened for a separate judge to grade. Unlike scripted automation, nobody writes its steps. Unlike a human tester, it doesn't decide what matters.
What does a QA agent actually do?
Across a QA workflow, a QA agent runs regression checks in a real browser, drafts candidate tests from product documentation, and turns judged failures into pending bug reports that carry the judge's reasoning. Each output stops short of a decision: humans confirm flagged changes, review generated drafts, and click to file any bug.
Can an AI agent replace a QA engineer?
No. An AI agent for QA replaces the running of checks, not the judgment around them. Deciding what needs testing, what correct means, whether a flagged change was intended, and whether a failure is worth a ticket all stay human calls. The realistic outcome is a QA engineer who executes less and decides more.
Do QA agents need test scripts?
No. A QA agent works from a plain-language prompt and an expected outcome instead of a script, deciding each step at run time by reading the page in front of it. There are no user-authored selectors to maintain, so a renamed button or a redesigned form changes the agent's route, not your test.
Is autonomous QA really autonomous?
Within a run, yes: the agent chooses its own route, and a bounded loop stops it from wandering forever. Around the run, deliberately not. A separate model grades the result against a fixed expectation, environment problems are recorded as blocked rather than failed, and a bug is filed only when a person clicks to file it.
How does an AI QA agent know if a test passed?
It doesn't get to decide. After the run, a separate judge model reads the recorded trajectory and grades it against the declared expected outcome, and it doesn't take the agent's word for anything. That separation is the point: the agent never gets to grade its own run.
Can a QA agent write its own test cases?
Yes, as drafts. Point it at your product's documentation and it crawls the pages, extracts the procedures, and drafts end-to-end test flows in plain natural language. A deterministic lint then drops drafts that assume leftover state or grade against resources nothing guarantees, on the principle that a bad test is worse than a missing one.
Watch one run before you decide
Definitions are cheap, and this page is full of them. A recorded run is not. You can see what the agent tried, what the judge concluded, and where it stopped to wait for a person. TestAutomate is pre-launch; the waitlist is for teams that want an AI QA agent working the regression layer while they keep the last word.