#Japan #Tokyo #Nexdata #Physical AI #Ego-centric Data

Introduction

As the field of Physical AI progresses, the challenge of obtaining diverse and high-quality multimodal data that closely represents real-world environments becomes paramount. For robotic systems to perform autonomously in real settings, data must not only be visual but also involve comprehensive ego-centric, multi-sensory inputs along with precise operational data. Addressing this challenge, Nexdata has unveiled a cutting-edge multimodal dataset focused on Ego-centric data starting from August 26, 2026, aiming to boost the development of Physical AI foundational models.

The Significance of Ego-Centric Data

Ego-centric data refers to information collected from a first-person perspective, providing necessary context and details that traditional datasets may overlook. In Physical AI research, combining various data sources like General Data, Simulation Data, Ego Data, UMI Data, and Robot Data delivers a powerful “data recipe” essential for training robust algorithms. It enables systems to better understand complex environments and interactions.

The Newly Released Dataset

The recently released dataset comprises extensive resources that cover everything from foundational model development to intricate manipulation learning. This includes:

• - Video Data: High-quality video from various perspectives.
• - IMU Data: Information derived from Inertial Measurement Units for motion tracking.
• - SLAM and Depth Data: Simultaneous Localization and Mapping data that provides spatial awareness.
• - Hand Key Points: Precise data about hand movements.
• - Semantic Annotations: Rich descriptions to facilitate machine understanding in diverse scenarios.

All these components are synchronously aligned, enabling immediate integration into research pipelines.

Addressing Bottlenecks in Data Collection

One significant bottleneck in the practical introduction of Physical AI is the challenges related to data collection and preprocessing. Nexdata addresses this issue by offering a holistic support structure based on extensive global data collection experiences. This includes assistance in hardware procurement and real-world validation processes, catering specifically to the diverse needs of AI development.

Technical Hurdles in Real-World Data Collection

In Physical AI research, it's crucial to construct a data ecosystem that does not rely solely on a single source. The intricate synchronization of various sensor data, especially in ego-centric contexts, poses substantial technical challenges. Managing multi-sensory data to create a cohesive operational framework often demands extensive in-house resources, making it difficult for many organizations.

Nexdata's expertise in the Physical AI domain allows the company to leverage its vast dataset collection for global enterprises and research institutes. This includes a rich variety of collection methods, such as teleoperation and comprehensive ego-centric data gathering, leading to the delivery of synchronized and high-precision datasets incorporating multiple modalities.

New Dataset Specifications

The released datasets cater to critical research themes such as Multi-View Geometry, Whole-body Control, and Sim2Real validations. Examples of the datasets include:

• - 1,000 Hours of PICO Collection Dataset: This dataset focuses on intricate coordination of full-body movement required for fine manipulation learning, ensuring high compatibility across various research pipelines.
• - 1,000 Pieces of 6-Camera Ego-Centric Data: Essential for 3D reconstruction and SLAM research, this data provides strict temporal synchronization and spatial information across varied real-world environments. The rigorous hardware-triggered synchronization across six cameras guarantees accuracy, crucial in advancing research outcomes.

Supplementary Datasets

In addition to the primary datasets, Nexdata offers many other collections suitable for model building and simulation phases, such as:

• - 135,000 Hours of Ego-Centric Data: Comprising multiple modalities including video and semantic annotations across varying environments.
• - 150,000 Sets of Hand Manipulation Data: Including RGB-D data that comprehensively documents joint states and force/torque feedback.
• - 11,600 Sets of 3D Hand Gesture Data: Fully synchronized depth, mask images, and 3D key points supporting gesture recognition tasks.
• - 288 Million Sets of 3D Model and Scene Data: Essential for applications in Sim2Real and developing digital twins.

Nexdata's Support Infrastructure

Nexdata’s commitment extends beyond just data provision. The organization boasts a dedicated Physical AI data collection facility measuring 8,000 square meters, housing over 300 robots including humanoids, dual-arm robots, and quadrupeds, enabling a large-scale data collection capability of over 5,000 hours monthly. They facilitate:

• - Space Data Creation Services: Employing cutting-edge 3D scanning to develop high-fidelity simulated environments tailored for Physical AI learning and validation.
• - Annotation Support: Specialized in Vision-Language-Action models, ensuring complete synchronization of multimodal data and annotating action intentions effectively.

Conclusion

Nexdata is committed to resolving the data challenges throughout the development and implementation phases of Physical AI. For those interested in specific domain-related data collection needs or technical consultations on hardware selection and specialized sensor synchronization, reach out to Nexdata to leverage their expertise and advanced solutions.

About Nexdata

• - Company: Datatang Co., Ltd. (Nexdata)
• - Location: 2-105 Kanda Awajimachi, Chiyoda City, Tokyo, Japan
• - Established: February 2020
• - Capital: 50 million yen
• - Business Overview: Providing AI learning data services, including custom data solutions.
• - Website: Nexdata Website