RoboNet Alternative: Targeted Training Data for Production Robotics

RoboNet was a pioneering effort in multi-robot data aggregation, and its vision of shared robot data across institutions anticipated the Open X-Embodiment project by several years. However, the field has evolved significantly since 2019. Modern robot learning revolves around imitation learning with structured action labels, multi-modal observations, and language conditioning -- none of which RoboNet provides. Its unstructured video data, collected for video prediction research, is a poor fit for training the behavioral cloning, diffusion policy, and VLA models that represent the current state of the art. Claru provides data that is designed for how robots learn today: expert teleoperated demonstrations with standardized actions, multi-modal sensor streams, and natural language annotations, collected on your specific robot in your deployment environment. For teams that have explored RoboNet's multi-robot video for visual pretraining, Claru delivers the structured, task-directed, quality-controlled demonstrations needed to train policies that actually deploy. We deliver in RLDS, HDF5, zarr, or LeRobot format with the standardized schemas that modern training frameworks expect, bridging the gap between legacy video datasets and production-ready training data.

RoboNet is a large-scale multi-robot video dataset developed by Sudeep Dasari, Frederik Ebert, Stephen Tian, and colleagues at UC Berkeley, Stanford, and Google. Published in 2019 (CoRL), RoboNet was one of the earliest efforts to aggregate robot manipulation data across multiple institutions and robot platforms with the explicit goal of enabling cross-robot visual transfer. The dataset contains approximately 15 million video frames from over 162,000 robot interaction trajectories collected on 7 distinct robot platforms.

The robots in RoboNet span a range of configurations: Baxter, Sawyer, Franka, and KUKA arms with various end-effectors, collected across 4 institutions with 113 different camera viewpoints. The primary data format is video: RGB frames at 64x64 to 256x256 resolution recorded at variable frame rates. Each trajectory captures a robot performing free-form interactions with objects on a tabletop -- pushing, prodding, and occasionally grasping -- without structured task definitions or success criteria.

RoboNet's design philosophy centered on video prediction as a path to robot learning. The idea was that a model trained to predict future video frames from diverse robots would learn physics priors and visual dynamics that could transfer to new robots and tasks. This approach was influential in the era before large-scale imitation learning became dominant, and RoboNet served as a pretraining corpus for video prediction models like SVG, SV2P, and early visual model-predictive control systems.

The dataset is released under the MIT License. While RoboNet was groundbreaking for its time, the field has largely moved beyond video-prediction-based control toward direct imitation learning with structured action labels (behavioral cloning, diffusion policies, VLAs). This shift has reduced RoboNet's relevance as a primary training resource, though it remains historically important and useful for visual pretraining research.

RoboNet's most significant limitation for modern robot learning is its lack of structured action labels. The dataset records raw motor commands that differ across robot platforms with no standardized action representation. Modern imitation learning methods (behavioral cloning, Diffusion Policy, ACT, VLAs) require consistent, standardized action labels -- typically end-effector deltas or joint position targets at a fixed control frequency. Converting RoboNet's raw motor logs into structured action labels is a major engineering effort, and for some robot platforms in the dataset, the logged commands may not contain sufficient information to reconstruct meaningful action trajectories.

The dataset contains unstructured free-form interactions, not task-oriented demonstrations. Robots in RoboNet push objects randomly, wiggle in place, or perform open-loop scripted motions with no task intent. There are no task definitions, success criteria, or language instructions. This was intentional for video prediction (where the model learns visual dynamics, not task completion), but it makes RoboNet largely unsuitable for imitation learning, which requires demonstrations of intentional, successful task execution.

Image resolution ranges from 64x64 to 256x256 pixels -- far below what modern visuomotor policies expect. State-of-the-art manipulation policies operate on 256x256 or higher resolution inputs, and many leverage 480p or 720p for fine-grained object perception. RoboNet's low-resolution video was sufficient for video prediction architectures of its era but limits the visual information available for direct policy learning.

RoboNet predates the multi-modal revolution in robot learning. It contains no depth data, no force/torque measurements, no proprioceptive state (for most subsets), and no language annotations. The dataset is RGB-only, which was the standard in 2019 but is insufficient for the multi-modal policies that represent the current state of the art.

Data quality is unfiltered -- the dataset includes failed grasps, no-contact episodes, robot-idle periods, and scripted random motions alongside successful interactions. There is no quality labeling or demonstration filtering, so using RoboNet for training requires substantial curation to separate useful interactions from noise.

RoboNet still has value for visual pretraining research. If you are studying how video prediction models learn physical dynamics, testing whether visual representations transfer across robot morphologies, or developing self-supervised learning methods that consume unlabeled robot video, RoboNet's 15M frames of diverse robot interactions provide useful raw material. Its multi-robot, multi-viewpoint nature remains a differentiator for visual representation learning.

However, for any task that involves training a policy -- whether behavioral cloning, reinforcement learning with demonstrations, or VLA fine-tuning -- RoboNet is the wrong data source. It lacks the structured action labels, task definitions, quality filtering, and multi-modal observations that modern policy learning requires. For these applications, purpose-collected demonstrations are far more effective.

Claru provides data that is designed from the ground up for policy training. Every demonstration is a successful execution of a defined task, with standardized action labels, multi-modal observations, language descriptions, and quality validation. This purpose-built design makes Claru data immediately useful for the training pipelines that RoboNet's unstructured video cannot serve.

For teams that have used RoboNet for visual pretraining, Claru provides the structured fine-tuning data that converts learned visual representations into deployable manipulation policies. Where RoboNet gives you visual dynamics, Claru gives you task-directed behavior: demonstrations of specific tasks on your robot with standardized actions, language annotations, and multi-modal sensor coverage.

Claru addresses every gap in RoboNet's design. We provide standardized action labels at your control frequency, task definitions with success criteria, multi-modal observations (RGB + depth + force/torque + proprioception + tactile), and validated language descriptions. Every demonstration is quality-controlled and represents a successful task execution, eliminating the curation overhead that RoboNet requires.

Where RoboNet spread thinly across 7 robot platforms, Claru focuses deeply on your specific platform. We collect thousands of demonstrations on your exact robot with your end-effector, in your deployment environment, producing data that is directly relevant to your policy rather than requiring transfer across embodiments.

Data is delivered in RLDS, HDF5, zarr, or LeRobot format -- standard formats that modern training pipelines expect, rather than the raw video dumps that RoboNet provides. Integration into your training workflow is immediate, with no preprocessing or format conversion required.