Across a month of public engineering discussion on social media and technical forums, our internal research counted 204 separate threads asking a version of one question: “How Do I Know This Is Good Enough?”

Nobody was asking whether simulation works. That argument finished years ago. They were asking whether the work already done amounts to proof. Those threads drew heavier engagement than almost every other topic we tracked, which tells you the question is both live and unresolved. Very few of them ended with an answer anyone accepted.

If you run validation for an autonomous system, you have probably asked it yourself, most likely in front of someone who wanted a number.

The most rigorous attempt to answer that question comes from road vehicles, which is simply where validation has been argued in public for longest. RAND calculated that an autonomous vehicle would need roughly 11 billion miles of driving to statistically demonstrate it is safer than a human driver. No test program reaches that figure physically, and no maritime, aviation, or defense program reaches its equivalent either.

So look at what the leading autonomous-driving developer does instead:

• Logged nearly 200 million real autonomous miles, with billions more run in simulation
• At peak, run as many as 25,000 virtual vehicles, covering up to 10 million simulated miles in a single day
• Simulation is the larger part of that program rather than a supplement to it
• Those figures, the RAND calculation above, and their original sources are collected in our 2026 statistics review.

The lesson transfers cleanly. Test volume measures effort. It says nothing about which parts of your operating envelope you actually touched.

Gartner expects 60% of data and analytics leaders to hit critical failures in managing synthetic data by 2027. That forecast is worth sitting with, because it is not a prediction about generation. Producing scenarios stopped being the hard part some time ago.

The difficulty moved downstream. Knowing which regions of your operating envelope have been sampled, and which have never been touched at all, is where the program now comes apart. A team can generate ten thousand scenarios and still be unable to say where their system stops working.

Run 100 randomly chosen scenarios, pass them all, and you have learned that your system passed 100 scenarios. You still cannot say where performance begins to degrade, because random sampling spreads runs evenly across a space where the interesting behavior sits in narrow bands.

Intelligent System Testing samples adaptively. Each run is chosen on the basis of what earlier runs revealed, so tests cluster around the boundaries where behavior changes rather than scattering across territory that was never in doubt. In our coverage matrix, 25 intelligently selected runs define those boundaries precisely, where 100 random runs leave them unknown.

The output is the part that matters commercially. You do not receive a pass rate. You receive a description of where your system works, where it fails, and which combination of conditions moved it from one to the other. That is a document someone can act on and build a release decision around.

Here is the shape of it in practice. You start from your own operational design domain and validation strategy, which gives you a set of base scenarios. You plug in your own system, an autopilot or a perception stack. You choose the parameters worth exploring: weather, environment, initial conditions, physical properties, sensor defects. The run orchestrates at scale on our cloud simulation service, and an interactive report comes back.

Now the part that sells itself. Picture an auto-docking approach that holds in calm water and holds under every individual disturbance you have tested. Then a crosswind at a particular strength, an ebb current across the berth, and a low sun angle behind it all arrive together, and the approach drifts. None of the three causes it alone. The combination does.

A few hundred simulated approaches will surface that combination. A season of field trials probably will not, because you cannot order the weather.

Three objections come up more than any others.

• We already have a simulator. Good, keep it. Intelligent System Testing sits on top as a test selection and coverage layer, and it answers a different question: which scenario should you run next?
• Our system is too specific for a preconfigured pack. The preconfigured part is the scenario scaffold. From step two onward, the system under test is yours.
• Thirty hours will not cover our domain. It is not meant to. Thirty hours is sized to answer one question, which is whether adaptive coverage surfaces failures your current suite misses. If it does not, you have found that out cheaply.

Four weeks. Thirty hours of simulation, unlimited editing time, and two preconfigured maritime scenarios covering auto-docking and obstacle detection and avoidance. Inside that window you explore thousands of parameter combinations, identify where and why the system fails, measure performance across conditions you would struggle to arrange in reality, and receive an interactive report on all of it. Fixed price, agreed before you start.

The report is the point. It is built to survive an engineering review, a customer conversation where the questions are harder than they were last year, and an internal go or no-go where somebody has to sign. If your job is to defend a validation position rather than describe one, that is the artefact you are buying.

The IST Starter Pack is live now, alongside three others covering counter-drone, airport ground operations, and interceptor unmanned surface vessels. Enrolment is open on the AILiveSim website, and if you would rather talk through which pack fits before committing, get in touch.

If you want the underlying numbers first, the figures in this article and their sources are collected in our 2026 statistics review.

Four weeks. Thirty hours of simulation, unlimited editing time, two preconfigured maritime scenarios, and your own autopilot or perception stack under test from the second step onward.

You get a fixed price, agreed before you begin, with no custom proposal to negotiate and no proof of concept to fund first. At the end you hold an interactive report showing where your system passes, where it fails, and which conditions triggered each result.

See the full scope, what is included, and how to enroll, on the Intelligent System Testing Starter Pack page.