Painting Task Training Data

Claru provides painting data collection with precision-instrumented workstations featuring calibrated multi-view RGB-D cameras (2 fixed + 1 wrist-mounted, synchronized at 30 Hz), 6-axis force/torque sensing at the wrist (500 Hz, 0.01 N resolution), and proprioceptive recording at 50-100 Hz. Our operators are trained on client-specific painting procedures with qualification protocols before production collection. We systematically vary configurations, conditions, and strategies to maximize diversity. Deliverables include synchronized multi-view video, force profiles, proprioceptive streams, per-episode annotations, and formatted outputs for Diffusion Policy, ACT/ALOHA, RT-2, OpenVLA, Octo, or custom architectures.

Painting is a critical robotic manipulation capability with growing demand across industrial automation, logistics, and service robotics. The task requires precise coordination of perception, planning, and control — making high-quality demonstration data essential for training policies that generalize beyond scripted trajectories. Traditional automation approaches rely on hand-coded programs that must be reprogrammed for each product variant or environment change, creating a bottleneck that learning-based approaches can overcome with sufficient training data.

The data challenge in painting is multifaceted. Success depends on precise force control, accurate state estimation, and adaptive behavior in response to environmental variation. Demonstrations must capture not just the nominal execution trajectory but the full range of corrective behaviors, error recovery strategies, and edge cases that arise in real operations. Without this diversity, learned policies overfit to idealized conditions and fail when deployed in the variability of production environments.

Research has consistently shown that multimodal data — combining vision, force/torque sensing, and proprioception — dramatically improves policy performance for painting tasks. Vision-only policies typically achieve 60-75% success rates on contact-rich variants of this task, while multimodal policies reach 85-95% by leveraging force feedback for precise contact state estimation and compliance control. This 15-30 percentage point improvement establishes multimodal demonstration data as essential for production-grade painting systems.

The commercial applications of robotic painting span manufacturing, logistics, healthcare, and service industries. As labor costs rise and workforce availability tightens, automation of painting operations becomes increasingly economically compelling. The primary barrier to adoption is not hardware capability but the availability of training data that captures the full complexity of real-world painting scenarios — including material variation, environmental uncertainty, and the diverse strategies that experienced human operators employ.

Recent advances in robot learning have dramatically improved the capabilities of learned painting policies. Diffusion Policy (Chi et al., 2023) has emerged as a leading architecture for contact-rich manipulation tasks, achieving state-of-the-art results on benchmarks that include painting components. The key advantage of diffusion models for this task is their ability to represent multimodal action distributions — when multiple valid execution strategies exist, the policy generates diverse high-quality actions rather than averaging across strategies.

ACT (Action Chunking with Transformers, Zhao et al., 2023) has shown particular promise for painting tasks that require smooth, temporally coherent trajectories. By predicting action sequences of 10-50 timesteps rather than individual actions, ACT produces the continuous motions needed for contact-rich tasks without the jerky transitions that plague frame-by-frame policies. On real-robot benchmarks involving similar manipulation skills, ACT achieves 85-96% success rates with only 50-100 teleoperated demonstrations.

Foundation models like RT-2 (Brohan et al., 2023) and OpenVLA (Kim et al., 2024) have demonstrated that pretrained vision-language models can be fine-tuned for painting with significantly less task-specific data than training from scratch. RT-2 achieves 60-75% zero-shot success on novel task variants described in natural language, suggesting that internet-scale pretraining provides transferable understanding of physical interactions. However, precision-critical aspects of painting still require task-specific demonstrations to achieve production-grade reliability.

The Open X-Embodiment initiative (Padalkar et al., 2023) has aggregated 970K robot episodes across 60+ datasets, providing a large-scale pretraining corpus that improves downstream performance on painting by 50-100% compared to training on individual datasets. Models pretrained on OXE and fine-tuned with 1K-5K task-specific demonstrations consistently outperform models trained from scratch on 10K demonstrations, establishing the pretrain-then-fine-tune paradigm as the most data-efficient approach for new painting deployments.

Data collection for painting requires a workspace instrumented for high-fidelity demonstration capture. The standard setup includes 2-3 calibrated RGB-D cameras covering the workspace from multiple viewpoints (overhead, angled, wrist-mounted), a 6-axis force/torque sensor at the robot wrist for contact force measurement, and proprioceptive recording from robot joint encoders at 50-100 Hz. Camera placement should ensure continuous visibility of the manipulation target throughout the entire task execution, including during phases where the robot hand may occlude direct overhead views.

Teleoperation is the primary collection method, with bilateral leader-follower systems (ALOHA-style) or VR controller interfaces enabling operators to perform natural task motions. Collection throughput depends on task complexity: simple single-action tasks yield 100-200 demonstrations per hour, while multi-step sequences with precision requirements drop to 30-60 per hour. Operators should be trained on the specific task procedures and complete a qualification round (20-50 successful demonstrations) before production collection begins. Rotate operators every 60-90 minutes to prevent fatigue-induced quality degradation.

Annotation requirements include: task phase segmentation (approach, contact, execute, verify, retract), contact state labels at key frames, success/failure classification with failure mode taxonomy, natural language task description (1-3 sentences), and any task-specific measurements (force profiles, dimensional accuracy, completion metrics). For learning-based architectures that process raw sensor streams (RT-1, Diffusion Policy), lightweight annotations (success label + language description) suffice. For modular approaches or curriculum learning, richer per-frame annotations enable more targeted training.

Data diversity drives generalization more than data volume. Systematically vary: object/part configurations (position, orientation, variant), environmental conditions (lighting, background clutter), operator strategies (encourage multiple valid approaches per task), and difficulty levels (easy baseline through challenging edge cases). Include 10-20% of demonstrations with intentional variation from nominal conditions — slightly different starting poses, minor obstacles, tool wear — to teach robust policies that handle the real-world variability absent from pristine lab setups.

Claru provides painting data collection with precision-instrumented workstations designed for high-fidelity manipulation demonstrations. Each station features calibrated multi-view RGB-D cameras (2 fixed + 1 wrist-mounted, synchronized at 30 Hz), a 6-axis force/torque sensor at the robot wrist (500 Hz, 0.01 N resolution), and proprioceptive recording at 50-100 Hz. We support both kinesthetic teaching for force-sensitive tasks and bilateral teleoperation for complex multi-step sequences.

Our operators are trained on painting procedures specific to each client application, completing a qualification protocol before production data collection begins. We systematically vary task configurations, environmental conditions, and execution strategies to maximize data diversity. Each demonstration captures the complete task cycle with automatic phase segmentation, contact state annotations, success labels with failure mode classification, and natural language task descriptions.

Claru delivers painting datasets formatted for direct ingestion by Diffusion Policy, ACT/ALOHA, RT-2, OpenVLA, Octo, or custom architectures. Standard deliverables include synchronized multi-view video, force/torque profiles, proprioceptive streams, per-episode annotations, and train/validation/test splits. Our daily throughput enables rapid scaling to the demonstration volumes that modern foundation models and task-specific policies require for production-grade reliability.