kaimingyang2024CC BY 4.0 (RoboCasa assets); Apache-2.0 (Dream.exe code); mixed upstream terms for generated outputs
Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?
A benchmark of 101 curated robot-manipulation task instances in simulation that evaluates whether video generation models can produce videos convertible into executable 3D robot trajectories. The dataset includes benchmark cases, ground-truth references, and published results from multiple video-generation and world-action models.
Downloads57
Episodes101
Why This Matters for Physical AI
Dream.exe establishes a benchmark protocol for evaluating whether video generation models can produce actionable robot manipulation trajectories, bridging generative modeling and embodied AI execution in simulation.
Technical Profile
- Modalities
- rgbdepthproprioception
- Robot Embodiments
- manipulator
- Action Space
- end_effector_delta
- Environment
- simulation
- Task Types
- manipulationpick_and_placesingle_object_manipulationmulti_object_interactionlong_horizon_tasks
- Episodes
- 101
- Data Format
- JSON with video bundles and HDF5-style hierarchical directory structure
- Annotation Types
- language_instructionsaction_labelssegmentation
- License
- CC BY 4.0 (RoboCasa assets); Apache-2.0 (Dream.exe code); mixed upstream terms for generated outputs
Access
Need custom rgb data?
Claru builds purpose-built datasets for simulation applications with dense human annotations and quality assurance.
Request a Sample Pack