#United States #Boston #Autonomous Vehicles #Motional #nuReasoning

Motional Opens New Horizons for Autonomous Vehicle Reasoning

Motional, a frontrunner in the autonomous driving sector, has announced an exciting development in the form of nuReasoning—a pioneering open dataset tailored to enhance the reasoning capabilities of autonomous vehicles (AVs). This dataset is the first of its kind, focusing specifically on long-tail scenarios that challenge AVs to emulate human-like decision-making. With the intention of advancing the entire AV industry, the nuReasoning dataset aims to provide the necessary training for end-to-end autonomous systems to develop common-sense intuition needed for navigating complex situations.

Beyond Simple Recognition

Today's state-of-the-art artificial intelligence (AI) is evolving beyond just recognizing visual inputs. It's moving towards a deeper comprehension of the physical world, where reasoning about possible future events and making informed decisions become crucial. To support this progression, the newly introduced Vision-Language-Action (VLA) models require training data that not only showcases correct actions but also discusses the underlying logic behind these actions. The nuReasoning dataset addresses this gap by endowing models with the ability to comprehend spatial relationships, deliberate on driving decisions, foresee risks, and evaluate alternate outcomes.

The nuReasoning dataset comprises a staggering 20,000 long-tail scenarios derived from Motional's extensive driving history, amounting to millions of miles on the road. Each scenario is enriched with video clips lasting at least 20 seconds and high-quality, human-verified reasoning annotations. This structure allows researchers and developers to not only see what an AV perceives but also to comprehend the rationale behind specific actions taken and the reasons alternatives were deemed unsafe.

Real-World Applications

An example scenario featured in nuReasoning depicts a nighttime construction zone, illustrating the AV's halting behavior before proceeding. Initially, one might interpret this as a sign that the AV struggled to navigate through the area. However, the detailed annotation clarifies that the vehicle stopped due to a small animal crossing the road ahead, with alternative paths blocked by construction barriers, further obscured by the unpredictability of the animal's speed and direction.

Phil Michel, Senior Vice President of Autonomy and AI at Motional, emphasizes the importance of this endeavor: "To safely expand AV fleets, autonomous driving systems must be capable of reacting to rare and chaotic edge cases with the same logic as an experienced human driver. By prioritizing transparent AI, we can better evaluate real-time decision-making and risk assessments. With nuReasoning being made openly accessible, we're providing a foundation for the entire industry to confront edge cases and work towards scalable autonomous operations."

The nuReasoning Challenge

To further catalyze global research in this field, Motional is launching the nuReasoning Challenge at the European Conference on Computer Vision (ECCV) in Sweden. This competition encourages researchers to innovate and assess planning and reasoning using the dataset through evaluations on 1,000 private-test scenarios.

Participants will engage in tasks such as:

• - Explainable Trajectory: Evaluating physical motion planning accuracy alongside safety compliance driven by explicit counterfactual reasoning.
• - Long-Tail Visual Question Answering: Testing a model's ability to infer spatial relationships and discern causal decision-making traces, as well as identifying risk factors within edge cases.

The Challenge winners will be recognized during the Conference on Neural Information Processing Systems (NeurIPS) in December, making it a significant event in the AV research community.

A Commitment to Open Data

nuReasoning builds upon Motional's commitment to revolutionizing autonomous vehicle research through open data, an effort that began with nuScenes in 2019—the first multi-modal public AV dataset. The continuous expansion of their datasets aims to empower both academic and commercial engineers to establish shared safety standards around AV perception, trajectory planning, and explainable AI frameworks.

Motional's datasets are available for free academic use and can also be licensed commercially under specific conditions. For interested parties, more information can be found through direct contact with the company.

About Motional

Founded in 2020 and predominantly owned by Hyundai Motor Group, Motional seeks to pave the way for a future where driverless vehicles are not just a concept but a tangible, safe, and reliable reality. Currently, they operate a commercial robotaxi service on the Uber network in Las Vegas, with plans to further their autonomous operations by the end of 2026. With headquarters in Boston and operations extending to Singapore, Motional is at the forefront of the autonomous vehicle revolution. For more information, visit Motional.com.