Training Data for BridgeData V2

Claru supplements BridgeData V2 with additional real-world demonstrations in environments and object categories not covered by the original dataset. We collect on WidowX 250-compatible hardware with identical 7-DoF EE delta actions at 5 Hz and deliver in RLDS format for direct concatenation with the BridgeData V2 corpus. Our environment coverage extends to commercial kitchens, retail shelves, warehouses, and other production-relevant settings that go beyond BridgeData V2's Bay Area apartment distribution. Each demonstration includes quality-validated language instructions, and our QA pipeline verifies action normalization ranges, camera framing consistency, and trajectory smoothness to ensure clean integration with existing co-training pipelines.

BridgeData V2 is a large-scale robot manipulation dataset created by Homer Walke, Kevin Black, and colleagues at UC Berkeley's RAIL lab, published in 2023. It extends the original BridgeData (2022) with significantly more demonstrations, environments, and skill coverage. The dataset contains 60,096 teleoperated demonstrations across 13 manipulation skills (pick-and-place, pushing, stacking, folding, sweeping, etc.) performed in 24 distinct real-world environments (kitchen counters, desks, shelves, toy kitchens).

What makes BridgeData V2 distinctive is its role in the ecosystem. Unlike model-specific datasets collected for a single paper, BridgeData V2 was explicitly designed as a shared pretraining resource for the community. It was one of the founding datasets in the Open X-Embodiment (OXE) project and is the primary pretraining source for Octo, the first open-source generalist robot policy. Nearly every major generalist manipulation policy published since 2023 has either pretrained on or evaluated against BridgeData V2.

The data was collected on a WidowX 250 6-DoF robot arm with a parallel-jaw gripper, controlled via a 3Dconnexion SpaceMouse for teleoperation. Each episode includes an RGB observation from a single third-person camera (256x256 resolution), the end-effector delta action (6-DoF position/rotation delta + 1-DoF gripper), and a natural language instruction describing the task. Episodes average 30-50 timesteps at 5 Hz control frequency.

BridgeData V2 is stored in RLDS (Reinforcement Learning Datasets) format, the TensorFlow-based dataset format adopted by the OXE consortium. Each episode is a tf.data.Dataset of steps, where each step contains observation images, actions, language instructions, and metadata. This format integrates directly with the training pipelines of Octo, RT-X, and other OXE-compatible models.

BridgeData V2 was collected through a distributed effort across multiple lab spaces and student apartments at UC Berkeley over approximately 18 months. The key design principle was environment diversity over per-task demonstration count. Rather than collecting 10,000 demonstrations of a single pick-and-place task, the team collected 50-500 demonstrations per task-environment combination across many combinations. This encourages policies trained on the data to generalize across environments rather than memorize specific configurations.

Teleoperation used a 3Dconnexion SpaceMouse that maps 6-DoF hand movements to end-effector velocity commands. This is a lower-bandwidth interface than leader-follower teleoperation (as in ALOHA), but it is dramatically cheaper and allows non-roboticists to collect data after minimal training. The trade-off is that demonstrations tend to be slower and less dexterous — BridgeData V2 tasks are primarily tabletop pick-and-place rather than fine-grained bimanual manipulation.

The 13 skill categories were chosen to cover the manipulation primitives most commonly needed in household and kitchen environments: pick and place, push, pull, stack, unstack, open/close containers, fold, sweep, wipe, pour, screw/unscrew, and tool use. Each episode has a language annotation describing the task in natural language, enabling language-conditioned policy learning. These annotations were written by the demonstrators during collection, not post-hoc labeled.

Quality control relied on success labels: each demonstration was marked as successful or failed by the demonstrator. The published dataset includes only successful demonstrations. Failed attempts (approximately 15-20% of all collection attempts) were discarded. No additional annotation or filtering was applied, which means the dataset contains some demonstrations with suboptimal trajectories — the object was placed successfully but the motion was not the smoothest possible path.

When people talk about training data requirements for BridgeData V2, they typically mean one of two things: how to add your own data to BridgeData V2 for co-training, or how to fine-tune a policy that was pretrained on BridgeData V2. Both have specific format requirements.

For co-training (adding new data to the BridgeData V2 corpus), your data must be in RLDS format with matching observation and action specifications. Observations should be 256x256 RGB images from a similar third-person viewpoint. Actions must be 7-dimensional end-effector deltas in the same coordinate frame convention: xyz position delta in meters, rotation delta as axis-angle in radians, and binary gripper command. Each episode needs a language instruction string. The RLDS conversion requires writing a dataset builder class using TensorFlow Datasets; the OXE project provides templates.

For fine-tuning an Octo model pretrained on BridgeData V2, the requirements are more flexible. Octo's architecture supports variable observation configurations and action dimensions through its task-specific adapter layers. You can fine-tune with as few as 20-50 demonstrations on a new task, provided the task is within the distribution of manipulation behaviors seen during pretraining. For tasks that are far out-of-distribution (e.g., deformable object manipulation, which BridgeData V2 barely covers), you may need 200-500 demonstrations.

Action normalization is critical. BridgeData V2 actions are recorded as small deltas at 5 Hz — typical position deltas are 1-3 centimeters per step, rotation deltas are 0-0.1 radians per step. If your robot operates at a different control frequency, you need to resample actions to match. If your robot uses joint position control instead of EE delta control, you need to convert through forward/inverse kinematics and account for the different action semantics.

Image preprocessing matters more than you might expect. BridgeData V2 images were collected with consumer webcams in natural lighting. If your environment has dramatically different lighting, background clutter, or camera viewpoint, the visual gap can be larger than the task gap. Augmentation (random crops, color jitter) during training helps, but collecting 20-50 demonstrations in your target environment for fine-tuning is usually necessary.

Claru supplements BridgeData V2 with additional real-world demonstrations in environments and object categories not covered by the original dataset. BridgeData V2's 24 environments are predominantly Bay Area student apartments and lab spaces — Claru provides demonstrations from commercial kitchens, warehouses, retail shelves, and other settings that better represent production deployment scenarios.

We collect on WidowX 250-compatible hardware with identical action space conventions (7-DoF EE delta at 5 Hz) and deliver in RLDS format, ready for direct concatenation with the BridgeData V2 corpus. Each demonstration includes a natural language instruction following BridgeData V2's annotation conventions. We also support Franka Emika and other platforms for teams using DROID-style pretraining.

For teams fine-tuning Octo or CrossFormer on custom tasks, we provide targeted fine-tuning datasets of 50-500 demonstrations per task in the exact format the model expects. We handle the RLDS conversion pipeline, action normalization, and image preprocessing so your team can focus on training rather than data engineering.

Our quality pipeline goes beyond BridgeData V2's success/failure labeling. We compute trajectory smoothness metrics, check for consistent camera framing, validate action magnitude ranges, and verify language instruction quality. This additional QA layer means our supplementary data integrates cleanly without introducing distributional noise that could degrade policy performance during co-training.