At a high level, Autonomous Driving System’s (ADS) safety standards are aligned in their intent. Whether in the U.S., Europe, or Asia, the expectation is the same: systems must demonstrate safe behavior across a wide range of real-world scenarios.
But while the destination is consistent, the path to getting there is not. Standards like ISO 21448 (SOTIF) and the ISO 3450X series, which details scenario-based testing, define what must be achieved, but not the how. This leaves teams the task of determining how to translate those guidelines into concrete testing workflows, measurable coverage-driven validation workflows, and defensible safety evidence.
This gap between guidelines, whether expressed in safety standards or internal safety cases, and practical, traceable testing and validation workflows is where compliance becomes difficult to scale. In this blog we will explore how structured methodology and AI-assisted workflows help teams close that gap and turn standards into executable and repeatable validation processes.
The Gap Between Standards and Execution
Across markets, ADS developers, safety, and compliance teams face a shared challenge: turning high-level safety requirements into a structured, executable validation process.
Standards define the overall safety objectives, but translating those objectives into day-to-day verification and validation (V&V) activities is where complexity begins to grow. Teams must decide how to break down the Operational Design Domain (ODD) into meaningful scenarios, prioritize testing, evaluate both system behavior and test coverage, and ultimately build a defensible argument that the system is ready for deployment.
Without a structured methodology connecting these steps, validation efforts can become fragmented, difficult to scale, and heavily dependent on manual expertise.
Frameworks like the Automated Driving Safety Evaluation Framework, which was developed by the Japan Automobile Manufacturers Association (JAMA), go further by providing a structured approach to scenario-based testing. JAMA introduces ways to break down the ODD, categorize disturbance scenarios, and define benchmark models such as the “competent and careful driver.” This gives teams a concrete starting point.
However, even with this level of detail, critical gaps remain. Teams still need to define how to quantify coverage of the existing testing with respect to what is prescribed in the standards and how to argue that their validation effort is complete.
Building a Structured V&V Methodology
To bridge this gap, teams need a reusable methodology that can be applied to different requirement frameworks that connect requirements to the execution of a highly automated testing and validation workflow. In practice, this means structuring V&V as a repeatable, iterative and highly automated toolchain, the V&V loop, rather than a one-off exercise.
A structured V&V loop connects safety goals, testing, coverage, and decision-making into a repeatable process.
This loop captures the essential steps:
- Define the safety goal, scope, and applicable standards
- Design the test campaign and coverage strategies (fleet driving data, SiL, HiL)
- Execute and measure results across testing environments
- Assess system performance and coverage against defined targets, and triage each result as a system failure or as a criterion that needs revision
- Identify and prioritize coverage gaps and performance failures
- Evolve the verification and validation plan, close coverage gaps through synthetic data generation or targeted data collection, and resolve identified failures
- Generate supporting evidence for the safety case and determine when validation is sufficient
The process is a closed loop and closes at multiple points. Step 6 feeds back into step 3. New and regenerated data goes into execution, failures get fixed, and the campaign runs again. Step 4 feeds back into step 2. Triage often shows that the system behaved as intended while the evaluation criterion misjudged it, so the metric or the pass/fail threshold gets revised before any further testing. Both loops keep running until the evidence from step 7 supports the safety case. System behaviour and coverage have to be addressed together.
The system performance path of the diagram automatically determines how the ADS performs against the acceptance criteria. Coverage answers a second question. It measures how much of what a standard or an internal safety case asks for has been exercised by the tests that ran. The reference for that measurement is the defined operational design domain and the requirement space. A defensible safety argument requires evidence of both.
Defining meaningful performance criteria takes real work. Teams have to settle what counts as a pass, which severity thresholds matter, and how to treat borderline cases. As the loop above shows, those decisions get revised as results come in.
Coverage splits into two parts. One part can be measured. It is coverage of a defined space, built from the ODD decomposition and the scenario classes that a standard or framework prescribes. The other part is an argument, namely that this space represents the conditions the system will meet in the field. A coverage claim holds up when it carries both, the number and the argument for the space it was measured in.
This is the work a methodology has to deliver. Structure the coverage space, prioritize the gaps by safety relevance, and keep the completeness argument explicit and traceable. Standards require these arguments and leave the method open. The rest of this blog shows how we close that gap.
Teams therefore need to decide how to structure the coverage space, prioritize testing gaps, and ultimately justify that their validation effort is sufficiently complete. Standards require these arguments to be made, but do not provide a concrete methodology for how to make them.
Using Safety Evaluation Frameworks as a Starting Point
Beyond high-level standards, teams must also look to evaluation frameworks and regulations that define the “how” of validation. Frameworks like the JAMA Automated Driving Safety Evaluation Framework provide a structured approach to scenario-based testing by offering detailed ODD-based scenario classes and benchmark models. Similarly, regulatory frameworks such as UNECE R157 (ALKS), UN R171 (DCAS), and the recently adopted UN R185 (ADS), translate safety objectives into specific, executable requirements for system behavior and validation. These frameworks serve as a vital implementation layer, bridging the gap between abstract safety goals and the practical task of creating safety evidence.
The JAMA Framework provides a concrete implementation of scenario-based testing that connects high-level standards to structured evaluation workflows.
However, even these frameworks do not fully solve the problem. They provide structure, but not a complete methodology for coverage measurement, gap prioritization, or safety argumentation. Teams still need to operationalize these elements within their own workflows.
Turning Methodology into Execution
Foretellix Foretify addresses this gap through its scenario-based platform that turns standards and frameworks into structured, executable workflows for ADS validation, data curation, and generation.
Rather than leaving teams to interpret standards independently, Foretellix provides a structured V&V methodology that connects standards to executable workflows, including scenario-driven test design, coverage-driven verification and validation, gap prioritization, and the development of a traceable safety argument. For regulations such as UNECE WP.29 R185, this provides a practical way to operationalize requirements for scenario generation and selection, Operational Design Domain (ODD) coverage, and traceable safety evidence.
Structuring the coverage space starts with the ODD. The methodology breaks it down along the dimensions a standard or framework prescribes and turns the result into a coverage model the platform measures against, so each gap comes out as a named item a team can act on. Prioritization then works on the safety relevance of each gap, along criticality, exposure and complexity. The weighting stays with the customer, because it depends on their ODD, their claim and their risk acceptance. Every scenario keeps its link back to the requirement it serves and forward to the result it produced, and that link is what makes the completeness argument reviewable by someone outside the team.
A key component of this approach is standards-aligned starter packages. For example, Foretellix offers a dedicated JAMA scenario and evaluation V-Suite that gives customers a significant head start in implementing the methodology. This package provides a predefined scenario framework, benchmark models, and evaluation criteria that teams can use as an immediate foundation. From there, they can adapt the workflow to their specific ODD and system, allowing them to begin with a structured foundation rather than a blank page.
Additionally, Foretellix’s platform enables a hybrid approach to data. It allows teams to automatically generate new, targeted synthetic scenarios address identified coverage gaps, while simultaneously providing the intelligence to determine exactly where real-world data collection is most critical. This ensures that validation teams optimize their resources, focusing their efforts only on the scenarios that provide the highest safety impact whether that means creating data synthetically or prioritizing the acquisition of data from the real world.
The Role of AI in Scaling Compliance
As ADS programs expand across markets, the challenge becomes scaling this methodology. Foretellix integrates AI directly into the workflow as a practical enabler in the form of the Foretellix AI assistant and the Foretellix MCP for use with popular AI engines. AI serves as an expert-guided interface that connects standards, methodology, and existing validation data. Instead of relying on individual experts to interpret standards, teams can interact with Foretellix’s AI assistant which is grounded in a curated and validated knowledge base.
This allows teams to:
- Receive guidance on how to apply standards in practice
- Tailor workflows to specific markets and ODDs
- Connect requirements directly to testing and evaluation
- Reuse existing validation data when expanding to new regions
For example, when expanding to a new market such as Japan, teams can ask the AI assistant how JAMA requirements map to their existing SOTIF-based test campaigns, and receive a structured response grounded in the validated knowledge base, rather than reinterpreting standards from scratch.
This is not a separate consulting layer. It is embedded within the Foretify workflow and connected to the data and tools already in use. As a result, teams can extend their validation efforts across markets without starting over.
From Interpreting Standards to Executing Them
As ADS development matures, compliance is no longer limited by access to standards. It is limited by the ability to execute them consistently. Standards like SOTIF and frameworks like JAMA provide essential direction, but they do not replace the need for structured implementation. By combining a clear V&V methodology, standards-aligned starter packages, and AI-guided workflows, Foretellix enables teams to move from interpreting standards to executing them. The shift is clear: from defining safety to demonstrating it with measurable, data-driven confidence.
If your team is spending more time interpreting standards than executing against them, Foretellix can help you change that. Our platform helps teams turn standards into measurable, scalable validation processes without starting from scratch. Reach out to explore how this approach could support your program.
