Dual-track AI assurance

Why

Teams putting AI systems into production fail in one of two directions. Some retrofit the application-era test phases onto the AI and miss the risks that live in the model: data that was never fit for purpose, behaviour that shifts under adversarial pressure, confident answers with no grounding. Others treat the AI as so special that it escapes the ordinary discipline of system delivery, and the boring defects (a broken integration, a missing rollback plan) ship unexamined next to a well-evaluated model.

The method refuses both. The AI gets its own assurance track with its own evidence classes, and it still has to pass through the same delivery gates as everything else. One standard of quality, reached by two different kinds of proof.

How

Two tracks run in parallel until the system is whole:

• Track 1, application testing. The conventional spine: unit and component testing, then system testing with a regression baseline, performance, and stability. Exit evidence: the components work and the system holds together.
• Track 2, AI assurance. Three stages, each with a plain-language claim it must earn. Data readiness (provenance, bias and representation, privacy): the data is fit for purpose. Model evaluation (evals, hallucination rate, adversarial resistance): we know how the model behaves. AI system testing (prompt injection, agentic flow validation, latency under load, guardrail effectiveness, graceful degradation): the behaviour holds under real conditions.

The tracks converge at the first gate, and both certificates are required to pass it. From there the system is tested as one thing: integrated end-to-end testing through a solution gate, business scenario validation and a confirmed rollback plan through a production gate. After release the track keeps running: drift monitoring, hallucination trending, regression monitoring, shadow testing against a baseline window. A production breach does not raise a ticket and disappear; it re-enters the AI track and earns its way back through the gates.

The method is deliberately maturity-graded. Five levels run from getting the data and model ready, through behavioural QE and system-level integration QE, to continuous QE in production and finally autonomous QE: AI testing AI, with automated red-teaming at scale, meta-evaluation of the evaluators, and agentic exploratory testing. Most organisations sit at the first two levels. The framework is designed to be run manually first and automated one level at a time, so the discipline arrives before the tooling.

What

What the shape buys is a sign-off authority that receives evidence in both languages at once: the delivery evidence it already knows how to read, and the AI evidence it has never been given in readable form. The certificates make the handshake explicit. No one has to guess whether the model was evaluated, because integration testing cannot start until it was.

Track 2 is scoped, not exhaustive. Which quality attributes matter for a given system (fairness, groundedness, drift, latency, compliance alignment) is read off a quality attribute taxonomy spanning trust and compliance, performance, operational readiness, and resilience concerns, so the AI track tests what is material for this system rather than everything equally hard.