The transition to AI-centric stacks and, specifically, end-to-end stacks in autonomous vehicles (AVs) promises to revolutionize mobility. While large foundation models can tackle the immense complexity of achieving higher levels of autonomy, its “black box” nature can introduce unique challenges. Building such autonomy requires the creation of a massive data flywheel to create diverse datasets for effective training, comprehensive validation, and evaluation of the AV stack’s performance. Achieving widespread coverage and diversity of sensor data is essential to large-scale AV deployment.
Effective training and testing of AI-centric AV stacks require massive amounts of varied and realistic data. These stacks need closed-loop validation as part of their Continuous Integration / Continuous Development (CI/CD) pipelines to identify gaps and regressions in AV stack performance throughout the development lifecycle. Furthermore, developers need hyper-realistic scenarios from the mundane, to the edge cases, and to situations too dangerous to test in the real world, to close these gaps and comprehensively test new versions of the stack.
To address these challenges, Foretellix is leveraging the fidelity and flexibility of the NVIDIA Omniverse platform into the Foretify™ toolchain to deliver a first-of-its-kind solution that transforms how AV developers train, test, validate, and accelerate the development and deployment of their systems.
Solving Industry Challenges with High-Fidelity Sensor Simulation
Physically Accurate Sensor Simulation for Generating Datasets at Scale
The Foretellix Foretify™ toolchain’s scalable scenario generation integrates the NVIDIA Omniverse Blueprint for AV simulation to provide developers with:
- Efficient AI Validation:
Support the shift to AI-centric end-to-end stacks by evaluating the performance of the entire system from sensor input to control output to detect gaps, edge cases, and risks. - Realistic Scenarios at Scale:
Generate any number of training and test scenarios under diverse environmental, weather, and traffic conditions along with hazardous scenarios too dangerous for real-world testing. - High-Fidelity Sensor Simulation:
Utilize the NVIDIA Omniverse Sensor RTX APIs to render physically based sensor data for camera, radar, and lidar to create both training and testing datasets, and enable closed-loop testing to accelerate development pipelines.
Solution Overview
- Foretify Evaluate:
Foretify Evaluate enables the aggregation and comprehensive analysis of real-world driving logs and virtual/Sim drive data for training and validation, ensuring robust AV system development. Stack developers and V&V engineers can quantify scenario coverage, systematically identify gaps, and address critical edge cases such as urban intersections and adverse weather conditions. Tailored Key Performance Indicators (KPIs) provide actionable insights to ensure AV systems meet safety standards and requirements. The solution automates the evaluation of physical and virtual driving scenarios, integrating seamlessly with Foretify Generate to connect synthetic scenario generation, big data analytics, and sensor simulation workflows for training and testing. Scalable to handle fleet-wide data, this end-to-end solution accelerates iterative improvements, enabling developers to close coverage gaps effectively and ensure large-scale, safe AV deployment. - Scalable Scenario Generation by Foretellix:
Foretellix Foretify’s scenario-based data-driven development toolchain empowers AV developers to create and run any number of synthetic testing scenarios simultaneously. Through Foretify’s Smart Replay capabilities developers can generate variations of the original drive log, including perception parameter variations and maneuver modifications, to improve the validation of new AV software versions and evaluate the completeness of their AV systems while providing the fidelity and scale necessary to train, test, and close critical gaps for large-scale, safe deployment. - High-Fidelity Sensor Simulation by NVIDIA:
The NVIDIA Omniverse Blueprint for AV simulation is a reference workflow that includes the physics, animation, and behaviors to enable physically accurate sensor simulation. It uses NVIDIA Omniverse Sensor RTX APIs to render the camera, radar, and lidar data necessary for AV training, testing, and validation. Foretellix will be one of the first partners to adopt the recently announced NVIDIA Cosmos, a platform of open world foundation models for generating physics-based videos and environments to enhance Foretellix’s unique scenario search technology.
Unique Benefits and Exclusive Features
- Comprehensive AI Evaluation Tools:
Ensure consistency and prevent degradation across trained AI systems by evaluating and comparing performance metrics between versions. - Unparalleled Realism:
The integration offers developers an exclusive, market-first capability to test their systems with high-fidelity sensor data. - Shorter Time to Market:
Efficiently identify and address gaps earlier in the development cycle, reducing time and costs. - Confidence in Deployment:
Enable safer and more reliable launches of autonomous vehicles by thoroughly validating systems in a virtual environment.
See Innovation in Action
Are you ready to see the Foretellix Foretify development toolchain with high-fidelity sensor simulation in action? Visit us at CES or book a personalized demo today to discover how this groundbreaking integration can supercharge your autonomous vehicle development process and help you launch with confidence.
