The challenge
The growing complexity of automated driving systems such as Lane Keep Assist, Lane Centering and Adaptive Cruise Control challenge existing verification and validation (V&V) methodologies used in the automotive industry. As these systems become more prevalent, bugs surface and failures occur. A collision example took place earlier this month, on June 2nd. A Tesla Model 3 using an automated driving function collided with a stationary truck on a Taiwanese highway.
Video 1 shows a simulated reproduction of the Tesla accident, based on released footage and using Foretify™ and Carla Simulator
Per user request, Foretify can generate hundreds of thousands of scenarios in which a truck or a random stationary object resides in random locations, orientations, lanes, color, and so on.
- Mixing scenarios and risk dimensions – Foretify allows mixing numerous vehicle maneuvers and risk dimensions to achieve the next level of thoroughness. The constraint solver can take sub-scenarios and find multiple proper locations, speeds and circumstances in which they can co-exist. Also, since bugs typically come in clusters, Foretify users can create multiple scenarios and variations of an already discovered bugs.
- Test table for requesting cross combinations of values – For adding project specific tests, Foretify uses a simple interface Test tables constitute a productivity tool that allows requesting thousands of executions with all cross-value combinations.
- Powerful Coverage Driven Analytics – Generating a large number of tests requires powerful data analytics and management tools. Foretify provides a dashboard that displays the executed conditions and KPIs in a simple, multi-hierarchical view. The dashboard reflects what was actually executed (given the unpredictable AV responses) and allows analysis of what was tested and verified. The tool also allows removing test duplications by creating a minimal set of tests that achieves maximum V&V coverage. Utilizing functional coverage and metrics to guide the verification efforts is a proven and productive approach to pragmatically explore most of your ODD in a minimal amount of time.
Figure 2 shows an automatically generated metrics report, including both coverage metrics and KPIs.
Foretify introduces an innovative approach with scalable random scenario creations, scenario combinations and mixing, cross combination values, and data-analytics to meet the ADAS and AV industry challenge of identifying the unknown unknowns.
