AI agents and the future of software testing

01The asymmetry nobody planned for

Code generation got cheap. Code verification didn’t. A coding agent can emit a thousand-line diff in minutes; a human reviewer needs an hour to read it properly, and a test suite that predates the agent can only check what someone thought to specify. The result is an asymmetry: throughput of change is exploding while confidence per change is flat or falling.

Traditional test automation doesn’t fix this, because it shares the bottleneck — tests are also written by people, one scenario at a time. When the authoring side is machine-speed, the checking side has to be too. That’s the actual meaning of “autonomous QA”: not AI-flavored assertions, but a closed loop where generation and verification run at the same tempo.

02LLMs at authoring time; determinism at runtime

The design choice that separates serious systems from demos is where the model sits. LLMs are excellent at the parts of testing that benefit from judgment: reading an unfamiliar app, mapping flows, proposing assertions, diagnosing why a selector broke. They are terrible at the parts that need repeatability: deciding pass/fail on Tuesday the same way they did on Monday.

So put the model at authoring and healing time, never in the runtime path:

• Authoring: an exploration agent crawls your app and emits real Playwright specs from plain-English scenarios — the mechanics we detailed in AI Playwright test generation.
• Healing: when a locator breaks, the model proposes a patch against the new DOM; a human or policy layer approves it — the discipline described in self-healing test automation.
• Runtime: the emitted spec executes deterministically. Same input, same verdict. No inference call decides whether your checkout test passes.

# Runtime path contains zero LLM calls:
#   spec (committed) → browser → assertions → pass/fail
# Model touches only:
#   crawl → generate → propose heal → human review → commit

03Coding agents should verify their own work

The most underrated shift is protocol-shaped. Via MCP, a coding agent doesn’t just write the feature — it invokes the QA platform itself, gets structured results back, reads the failure evidence, fixes the code, and re-runs. No human relay in between:

$ molar verify "signup confirms with OTP" --url http://localhost:3000 --json
{
  "verdict": "fail",
  "step": 4,
  "reason": "OTP email never arrived within 30s",
  "evidence": {
    "trace": "https://app.molar.it/t/run_8f21",
    "cloned_calls": ["POST /v1/emails → 503 (mailbox quota)"]
  },
  "hint": "MailHole clone rejected send; check SMTP pool config"
}

Structured output with evidence links changes the economics of agent-driven development: the agent self-corrects from the trace instead of a developer pasting screenshots into a chat. This is why we built Molar’s MCP server as a first-class interface — the coding agent becomes the primary consumer of QA, and the QA platform becomes part of its toolbelt rather than a downstream department.

04What stays stubbornly human

Agents are converging on execution. Three things remain yours:

• Intent. What should be true? Which flows matter enough to gate a merge on? An agent can discover flows, but only you know which ones would end your business if they broke.
• Judgment calls. Is this pixel drift acceptable? Is this edge case a bug or a feature? These are product decisions wearing engineering costumes.
• Review of proposed changes. Including proposed test changes — a healing patch that quietly weakens an assertion is a vulnerability, not a convenience.

The role shift is real but familiar: less time authoring steps, more time specifying outcomes and auditing what the machines proposed. It mirrors what happened to compilers — nobody misses hand-writing assembly, and nobody reviews the machine code either.

05The failure mode worth designing against

Every automation wave in testing has produced the same pathology: tests that pass while the product breaks, because the suite was optimized for green rather than truth. Agent-driven QA can amplify this dramatically — an agent that both writes and heals its own tests can, without any malice, converge on assertions that match whatever the app currently does.

Three structural defenses: keep runtime verdicts deterministic so healing requires a reviewable diff; version scenarios as source files so weakening an assertion shows up like any other change; and sample agent-modified suites against known-broken builds periodically to confirm they still catch what they claim to catch. Mutation-style spot checks are cheap insurance here — break the checkout total on purpose once a month and confirm something red appears within one run.

06Where this lands

Our working prediction for the next few years: plain-English scenarios become the interface for specifying behavior (reviewed like code, versioned like code); autonomous agents author, heal, and execute the suites; deterministic replay makes every verdict auditable; and required checks become negotiations between two machines, with humans reading summaries instead of triaging failures. The teams that get there first won’t hire fewer QA engineers — they’ll point them at the problems automation genuinely can’t judge.

If you want to feel the loop before believing the essay, Molar is the shortest path: describe a flow in English, get Playwright-grade tests, gate PRs with them, and let your coding agent call the whole thing through MCP.