Wrist Camera Training Data

Claru's collection rigs are equipped with wrist-mounted Intel RealSense D405 cameras (RGB-D, 640x480, 60 Hz capable) on all teleoperation arms, synchronized via PTP hardware triggers to achieve sub-1 ms temporal alignment with third-person cameras and proprioception streams. Precision-machined aluminum mounting brackets ensure < 0.1 mm positioning repeatability, and hand-eye calibration is performed and verified at the start of each collection session with residuals logged as metadata. Our quality pipeline automatically flags motion blur, lens flare, and gripper occlusion issues, replacing affected episodes in subsequent collection batches. We support wrist camera data collection across all manipulation tasks in our catalog and provide both raw and rectified images with full calibration data. For cross-embodiment datasets, we collect on multiple robot platforms with standardized wrist camera configurations. Delivered in RLDS (OpenVLA/Octo compatible), HDF5 (Diffusion Policy compatible), or custom formats with hand-eye calibration matrices, camera intrinsics, distortion coefficients, and synchronization verification logs.

Wrist-mounted cameras provide an egocentric, eye-in-hand view of the manipulation workspace that third-person cameras cannot replicate. When a robot reaches into a cluttered shelf, behind an obstacle, or inside a container, fixed external cameras lose line of sight — but the wrist camera maintains a close-up view of the end-effector-object interaction throughout. This makes wrist camera data essential for contact-rich tasks: insertion, connector mating, fine-grained grasping, and any manipulation where millimeter-level visual feedback determines success.

The impact of wrist camera data on policy performance is well-documented. Brohan et al. (2023) showed that adding a wrist camera view to RT-2 improved success rates by 15-25% on tasks involving small objects and tight clearances compared to third-person cameras alone. The ALOHA system (Zhao et al., 2023) uses dual wrist cameras as a primary observation modality, and ablation studies confirm that removing wrist views degrades bimanual manipulation success by 20-30%. The Open X-Embodiment dataset (Padalkar et al., 2024) found that policies trained on datasets with wrist views transfer more effectively than those without, likely because wrist views provide a more robot-invariant observation of the task-relevant workspace.

Despite its importance, wrist camera data is scarce in public datasets. Most academic datasets use only overhead or third-person views because wrist cameras add complexity: they move with the arm (requiring ego-motion compensation), have narrow fields of view, and suffer from motion blur during fast movements. Collecting high-quality wrist camera data requires careful camera selection (wide-angle, low-latency, vibration-resistant), mounting design (rigid attachment without obstructing gripper), and synchronization with other modalities. This infrastructure cost is why wrist camera data remains the highest-value gap in the manipulation data landscape.

The proliferation of foundation models for robotics has made wrist camera data even more critical. Models like OpenVLA, Octo, and Pi-zero are designed to consume multi-view inputs where each view provides complementary information. The third-person view gives spatial context and scene layout, while the wrist view gives precise hand-object geometric relationships needed for fine manipulation. Training or fine-tuning these models without wrist camera data produces policies that succeed on coarse tasks (pick up a large object) but fail on fine tasks (insert a USB plug) — precisely the tasks with the highest commercial value. The DROID dataset (Khazatsky et al., 2024) established wrist camera inclusion as a standard by collecting 76,000 demonstrations across 564 tasks with wrist views from 90 institutions, creating a de facto expectation that serious manipulation datasets include this modality.

Modern visuomotor policies have converged on using wrist cameras as a primary or co-primary observation modality. The RT-2 system processes wrist and third-person views through a shared vision encoder, learning to attend to whichever view is most informative at each timestep — third-person for approach, wrist for fine manipulation. Octo (Team, 2024) explicitly supports multi-view inputs including wrist cameras, and their ablations show that wrist views are most valuable in the final 30% of manipulation episodes when the end-effector is close to the target object. Pi-zero (Physical Intelligence, 2024) processes wrist camera inputs through a flow-matching architecture, demonstrating that the wrist view is critical for the high-precision phase of manipulation where the policy must servo to sub-centimeter accuracy.

Diffusion Policy with wrist camera observations has become the default setup for fine manipulation tasks. Chi et al. (2023) demonstrate that wrist-view Diffusion Policy achieves 90%+ success on USB insertion (a task requiring sub-millimeter alignment) where third-person-only policies achieve only 50-60%. The key advantage is that the wrist camera provides a consistent, high-resolution view of the insertion point regardless of the global robot configuration. This invariance to arm pose is why wrist camera data improves cross-embodiment transfer — a policy trained with wrist views on a Franka can transfer to a UR5 more easily than one trained with fixed third-person cameras, because the wrist view looks similar across embodiments while third-person views change completely with robot morphology.

The emerging direction is wrist camera-based foundation models. By training on large corpora of wrist camera video from diverse tasks and robots, these models learn general-purpose representations of hand-object interactions that transfer across manipulation tasks. The DROID dataset (Khazatsky et al., 2024) collected 76,000 demonstrations across 564 tasks from 90 institutions, with wrist camera views as a standard modality. This scale of data enables zero-shot policy deployment on new tasks when paired with language conditioning. HPT (Hierarchical Policy Transformer, Wang et al., 2024) demonstrated that training on 200K+ episodes with wrist views from heterogeneous robot platforms produces a policy backbone that requires only 50-100 demonstrations for fine-tuning on new tasks — a 10x reduction compared to training from scratch.

Cross-embodiment transfer is the most commercially valuable property of wrist camera data. Companies building manipulation products need to deploy across different robot arms (Franka, UR, Kinova, custom hardware) without recollecting entire datasets for each platform. Wrist camera views are the most transferable modality because the view depends primarily on the gripper-to-object relationship, not on the arm kinematics or workspace geometry. Open X-Embodiment results show that wrist view data from one robot type improves performance on a different robot type by 10-15%, while third-person data provides near-zero cross-embodiment benefit due to the completely different camera viewpoints.

Wrist camera selection is critical. The camera must be wide-angle (120+ degree FOV) to capture the workspace despite being mounted close to the gripper, low-latency (< 50 ms end-to-end), and vibration-resistant (fixed-focus or hardware-stabilized). Popular choices include the Intel RealSense D405 (RGB-D, compact form factor, 87-degree depth FOV), the Arducam UC-698 (170-degree fisheye, ultra-compact), and custom GoPro-class cameras (high resolution, long latency). For RGB-D wrist cameras, the D405 is currently the gold standard due to its 42 mm minimum depth range (close enough to capture the grasp zone) and 640x480 resolution at 30 Hz. Mounting must be rigid — any relative motion between camera and end-effector invalidates the hand-eye calibration. We use precision-machined aluminum brackets with dowel pin alignment to ensure < 0.1 mm mounting repeatability.

Temporal synchronization is the biggest technical challenge. Wrist camera frames must be synchronized with third-person cameras, proprioception, and force/torque data within 5 ms. At 60 Hz, a 5 ms offset means the wrist camera frame corresponds to a position error of up to 3 mm during fast arm movements — enough to confuse a policy learning fine manipulation. Hardware triggering (using a shared pulse signal to trigger all sensors simultaneously) is strongly preferred over software synchronization. Claru's collection rigs use PTP (Precision Time Protocol) synchronized camera triggers for sub-1 ms alignment across all modalities. We verify synchronization quality at the start of each collection session using a checkerboard visibility test where a checkerboard is simultaneously visible in wrist and third-person cameras, confirming temporal alignment through synchronized motion.

Wrist camera data has unique quality challenges: motion blur during fast movements, lens flare from overhead lighting, and gripper finger occlusion in the field of view. Our quality pipeline automatically flags frames with blur scores above threshold (Laplacian variance < 100), over-exposure from specular reflections (histogram saturation > 15% of pixels), and occlusion events where the gripper blocks more than 40% of the field of view. Flagged episodes are reviewed and either repaired (by adjusting crop) or replaced in the next collection batch. For wide-angle and fisheye lenses, we provide both the raw distorted images (for compatibility with architectures that learn their own spatial representations) and rectified images (with calibrated distortion correction applied) as parallel data channels.

Hand-eye calibration is performed per collection session using the eye-in-hand method: the robot moves through 20+ poses while observing a fixed calibration target, and the resulting pose data is used to compute the rigid transformation between the camera frame and the end-effector frame. This calibration is critical for any downstream processing that requires mapping wrist camera pixels to 3D workspace coordinates. The calibration residual (reprojection error) is recorded and included in the dataset metadata — typical values are 0.5-1.5 pixels for well-calibrated setups. Sessions with reprojection error above 2 pixels are re-calibrated before collection proceeds.

Claru's collection rigs are equipped with wrist-mounted Intel RealSense D405 cameras (RGB-D, 640x480, 60 Hz capable) on all teleoperation arms, with precision-machined aluminum mounting brackets ensuring < 0.1 mm positioning repeatability. All wrist cameras are synchronized via PTP hardware triggers to achieve sub-1 ms temporal alignment with third-person cameras and proprioception streams. Hand-eye calibration is performed and verified at the start of each collection session, with calibration residuals logged as dataset metadata.

Our quality pipeline automatically flags motion blur, lens flare, and gripper occlusion issues, replacing affected episodes in subsequent collection batches. We provide both raw fisheye/wide-angle images and rectified images as parallel channels, along with camera intrinsics and distortion coefficients for custom processing. For cross-embodiment datasets, wrist camera data is collected on multiple robot platforms (Franka, UR5e, Kinova Gen3) with standardized camera mounting and calibration procedures to maximize transfer learning utility.

We support wrist camera data collection across all manipulation tasks in our catalog — from fine insertion and connector mating to kitchen tasks, assembly, and warehouse bin picking. Datasets include synchronized wrist RGB-D, third-person RGB-D, 100 Hz proprioception, optional force/torque, and per-frame quality metadata (blur score, exposure quality, occlusion percentage). Delivered in RLDS (OpenVLA/Octo compatible), HDF5 (Diffusion Policy compatible), or custom formats with full hand-eye calibration matrices, camera intrinsics, distortion coefficients, and synchronization verification logs.