CLARU
nvidia2025nvidia-noncommercial-license

R4D-Bench

A region-level 4D video question answering benchmark with 1,419 region-prompted multiple-choice VQA pairs built from dynamic real-world videos. It challenges models to track, reason about depth, and understand temporal dynamics of specific regions in video.

Downloads26
Episodes1419
Likes4

Why This Matters for Physical AI

This dataset advances physical AI by enabling models to understand 4D spatial-temporal dynamics at the region level, critical for tasks requiring precise depth perception, motion tracking, and reasoning about object kinematics in real-world scenarios.

Technical Profile

Modalities
rgblanguage
Environment
outdoorurban
Task Types
visual_question_answering3d_groundingspatial_reasoningmotion_understanding
Episodes
1419
Data Format
json
Annotation Types
language_instructionsbounding_boxesaction_labels
License
nvidia-noncommercial-license
Part of the R4D-Bench family

Access

View on HuggingFaceRead Paper

Need custom rgb data?

Claru builds purpose-built datasets for outdoor applications with dense human annotations and quality assurance.

Request a Sample Pack

Related Datasets

Xperience-10M

A large-scale egocentric multimodal dataset of human experience containing 10 million interactions and 10,000 hours of synchronized first-person recordings with six video streams, audio, stereo depth, camera pose, hand mocap, full-body mocap, IMU, and hierarchical language annotations for embodied AI, robotics, and world modeling research.

rgbaudiodepthproprioception+3
2.3M downloadsother

Open-H-Embodiment

A multi-embodiment community-driven dataset of paired kinematics and video for training AI autonomy models in surgical robotics and ultrasound applications. The dataset includes tabletop exercises, clinical procedures, and simulations of healthcare robotics tasks.

rgbproprioception
118K downloads2026cc-by-4.0

Open-H-Embodiment

A community-driven, multi-embodiment dataset of paired kinematics and video for training and evaluating AI autonomy models in surgical robotics and ultrasound applications, including tabletop exercises, clinical procedures, and healthcare robotics simulations.

rgbproprioception
76K downloads2026cc-by-4.0

LingBot-Depth Dataset

Self-curated RGB-D dataset for training masked depth modeling approaches, containing real-world indoor scenes, VLA robot manipulation tasks, and simulated data across multiple camera types and robot platforms.

rgbdepth
71K downloadsCC BY-NC-SA 4.0

PhysicalAI-Robotics-Manipulation-Kitchen-Demos

A large-scale dataset of 600 hours of human-teleoperated demonstrations across 316 different kitchen manipulation tasks, totalling 55k trajectories collected using a Franka Panda robot with an Omron mobile base.

rgbproprioception
57K downloads2025cc-by-4.0

ManiTwin-100K: Manipulation-Ready Digital Object Twins

A large-scale dataset of 100K manipulation-ready digital object twins with simulation-ready 3D meshes, physical properties, functional point annotations, grasp configurations, and language descriptions validated through physics-based simulation.

3d_meshlanguage
45K downloads2026cc-by-nc-4.0