CLARU
paulpacaudapache-2.0

Guardian Failure Detection Dataset

A vision-language dataset for detecting robotic planning and execution errors, containing annotated manipulation failures across simulation and real-robot data with procedurally generated failure modes and step-by-step reasoning traces.

Downloads93
Episodes32,621

Why This Matters for Physical AI

Guardian enables vision-language models to detect and reason about robotic failures during execution and planning, which is critical for improving robot reliability and enabling human-robot collaboration in real-world manipulation tasks.

Technical Profile

Modalities
rgblanguage
Robot Embodiments
Franka PandaUR5gripper
Environment
simulationlab
Task Types
manipulationgraspingpick_and_placevisual-question-answering
Episodes
32,621
Data Format
JSONL
Annotation Types
language_instructionsfailure_labelsfailure_categoriesreward_labelsreasoning_tracesobject_detection
License
apache-2.0
Part of the Guardian family

Access

View on HuggingFace

Need custom rgb data?

Claru builds purpose-built datasets for simulation applications with dense human annotations and quality assurance.

Request a Sample Pack

Related Datasets

LingBot-Depth Dataset

Self-curated RGB-D dataset for training masked depth modeling approaches, containing real-world indoor scenes, VLA robot manipulation tasks, and simulated data across multiple camera types and robot platforms.

rgbdepth
380K downloadsApr 2026CC BY-NC-SA 4.0

Xperience-10M

A large-scale egocentric multimodal dataset of human experience containing 10 million interactions and 10,000 hours of synchronized first-person recordings with six video streams, audio, stereo depth, camera pose, hand mocap, full-body mocap, IMU, and hierarchical language annotations for embodied AI, robotics, and world modeling research.

rgbaudiodepthproprioception+3
188K downloadsApr 2026other

Open-H-Embodiment

A multi-embodiment community-driven dataset of paired kinematics and video for training AI autonomy models in surgical robotics and ultrasound applications. The dataset includes tabletop exercises, clinical procedures, and simulations of healthcare robotics tasks.

rgbproprioception
98K downloadsMar 2026cc-by-4.0

OmniAction

A large-scale multimodal dataset for proactive robot manipulation comprising 141,162 episodes with cross-modal contextual instructions derived from spoken dialogue, environmental sounds, and visual cues rather than explicit commands.

rgbaudiolanguage
63K downloadsMar 2026cc-by-nc-4.0

Egocentric-100K

The largest dataset of manual labor with 100,405 hours of egocentric video from head-mounted fisheye cameras, featuring state-of-the-art hand visibility and active manipulation density.

rgbvideo
61K downloadsFeb 2026apache-2.0

OmniAction

A large-scale multimodal dataset for proactive robot manipulation comprising 141,162 episodes across 112 skills and 748 objects, enriched with audio, visual, and contextual instruction data for cross-modal intention recognition.

rgbaudiolanguage
48K downloadsMar 2026cc-by-nc-4.0