Mobile Manipulation Training Data

Claru collects mobile manipulation data using instrumented mobile platforms with synchronized multi-view cameras, LiDAR, arm proprioception, and base odometry. Our operators use whole-body teleoperation interfaces to produce natural coordinated base-arm demonstrations in real homes, offices, retail spaces, and other indoor environments across our global network of 100+ cities. We leverage environment diversity — the single most important factor for mobile manipulation generalization — by collecting in 10-20+ distinct indoor spaces per dataset. Standard delivery includes whole-body kinematic recordings, environment semantic maps, task phase annotations, and manipulation outcome labels in RLDS format compatible with Mobile ALOHA, NaVILA, and foundation model architectures. For teams using co-training, we can provide matched static manipulation datasets collected on the same robot arms.

Mobile manipulation combines navigation and manipulation into a single system: a robot that can move through an environment, approach objects, and manipulate them. This capability underlies virtually every useful household and commercial service robot application — fetching objects from shelves, clearing tables, loading dishwashers, organizing rooms. The global service robot market is projected to exceed $130 billion by 2030, and mobile manipulation is the core technical capability that unlocks this market.

The fundamental challenge is whole-body coordination. A fixed-base manipulator has 6-7 degrees of freedom (DoF); a mobile manipulator has 10-15+ DoF when combining the base (2-3 DoF), arm (6-7 DoF), and gripper (1-2 DoF). These must be coordinated to perform tasks that require simultaneous base repositioning and arm movement — for example, walking up to a table while reaching for an object, or repositioning the base mid-task when the arm reaches its workspace limit. Classical motion planning decomposes this into sequential navigate-then-manipulate, losing the fluid whole-body coordination that makes human task execution efficient.

Mobile ALOHA (Fu et al., 2024) demonstrated that whole-body teleoperation followed by behavioral cloning produces surprisingly capable mobile manipulation policies. Using a bimanual mobile platform teleoperated by a human at walking speed, Mobile ALOHA trained Diffusion Policies on 50-100 demonstrations per task that achieved 80-90% success on complex tasks like opening a two-door cabinet while repositioning, loading dishes into a drying rack, and pushing chairs under a table. The key insight was co-training with static manipulation data: combining 50 mobile demos with 3,000 static tabletop demos improved mobile task success by 34% compared to training on mobile data alone.

The data challenge for mobile manipulation is environment diversity. A policy trained in one apartment fails in another because the spatial layout, furniture arrangement, and object locations differ completely. NaVILA (Chiang et al., 2024) showed that internet-scale vision-language pre-training provides the perceptual generalization needed for mobile manipulation in novel environments, but the manipulation policies themselves still require real demonstration data across at least 10-20 distinct environments to handle the diversity of object positions, approach angles, and cluttered spaces encountered in real deployment.

Mobile ALOHA (Fu et al., 2024) set the current standard for learned mobile manipulation. The system uses a bimanual mobile platform with two 6-DoF arms mounted on a wheeled base, teleoperated by a single human operator using a matching leader-follower interface. Trained on 50 demonstrations per task, Mobile ALOHA achieved 90% success on table clearing, 85% on cabinet opening, and 80% on dishwasher loading. The breakthrough finding was that co-training with 3,000 static tabletop manipulation demonstrations from the ALOHA dataset improved mobile task success by 34 percentage points — the static data teaches generalizable manipulation skills that transfer to the mobile context.

NaVILA (Chiang et al., 2024) tackled cross-environment mobile manipulation by combining a vision-language foundation model with learned navigation and manipulation policies. NaVILA can execute open-vocabulary instructions ('bring me the water bottle from the kitchen counter') in apartments it has never seen by leveraging VLM-based scene understanding for navigation target identification and approach planning. In zero-shot evaluations across 10 novel apartments, NaVILA achieved 68% task success versus 31% for Mobile ALOHA without VLM guidance, demonstrating the importance of semantic understanding for mobile manipulation in diverse environments.

TidyBot (Wu et al., 2023) demonstrated that LLM-based preference learning enables personalized mobile manipulation. Given 5-10 examples of how a specific user organizes their home, TidyBot's LLM-based planner generalizes to organize novel items following the same personal preferences with 85% accuracy. The manipulation primitives are trained on 200-500 demonstrations per skill (pick, place, open, close), while the high-level planning leverages GPT-4's category reasoning to determine where new items should go.

For industrial mobile manipulation, Spot from Boston Dynamics combined with learned manipulation policies demonstrates warehouse-relevant capabilities. Hawkins et al. (2023) showed that a Spot robot with a 6-DoF arm trained on 2,000 demonstrations achieves 93% pick success from shelving units at varying heights, with the learned policy coordinating base approach angle and arm reach to maximize the reachable workspace. The critical data requirement was demonstrations at 30+ different shelf configurations spanning the full range of heights, depths, and clutter levels.

Mobile manipulation data collection requires a teleoperation-capable mobile platform and diverse indoor environments. The standard approach uses a leader-follower teleoperation interface (Mobile ALOHA style) where the human operator walks alongside or behind the robot, controlling arm and base simultaneously through mirrored joints. This produces the most natural whole-body coordination patterns. Alternative approaches include VR teleoperation (more intuitive but higher latency) and waypoint-based collection (lower quality but higher throughput for simple tasks).

The sensor suite must capture the full state of the mobile manipulator and its environment. At minimum: head/body-mounted RGB cameras (2-3 views at 30 Hz) for scene understanding, wrist-mounted cameras for close-up manipulation views, base odometry (50+ Hz) for position tracking, arm joint positions and velocities (100+ Hz), and optionally a 2D LiDAR (10-20 Hz) for obstacle avoidance context. All sensors must be hardware-synchronized and calibrated to a common robot body frame. Record base velocity commands alongside odometry to capture the intended motion, not just the achieved motion.

Environment diversity is the primary driver of mobile manipulation policy generalization. Collect data in at least 10-20 distinct indoor environments (apartments, offices, retail spaces) with naturally varied furniture arrangements, lighting conditions, and clutter levels. For each environment, record 5-10 episodes of each target task from different starting positions to ensure coverage of the approach angle distribution. Randomize object placement between episodes (move cups to different counter positions, change which shelves contain target objects) to prevent the policy from memorizing fixed locations.

Claru collects mobile manipulation data using instrumented mobile platforms with synchronized multi-view cameras, LiDAR, proprioception, and base odometry across our global network of collection sites. Our operators use whole-body teleoperation interfaces to produce natural coordinated base-arm demonstrations in real homes, offices, and retail spaces. We deliver datasets with full kinematic recordings, environment metadata, task phase labels, and manipulation outcome annotations in RLDS format compatible with Mobile ALOHA, NaVILA, and foundation model architectures.