Training Data for Agility Robotics
Agility Robotics is deploying Digit in real warehouses at Amazon and GXO. Here is how purpose-collected data addresses the sim-to-real challenges of commercial bipedal logistics.
About Agility Robotics
Agility Robotics builds Digit, the first humanoid robot designed and built for real-world warehouse work. Founded in 2015 as a spin-out from Oregon State University by Jonathan Hurst and Damion Shelton, Agility has raised over $179 million and opened RoboFab — the world's first factory purpose-built for manufacturing humanoid robots — in Salem, Oregon. Digit is the only bipedal robot with active commercial deployments in Amazon fulfillment centers and GXO logistics warehouses.
Agility Robotics at a Glance
Core Data Requirements
Warehouse Navigation
Bipedal locomotion on real warehouse floors with varied friction, obstacles, dock plates, and congestion patterns captured with body-worn sensors and ground-truth paths.
Tote Manipulation
Thousands of pick-and-place demonstrations with real warehouse totes varying in weight, fill level, surface condition, and stacking configuration.
Human-Robot Interaction
Proximity and interaction data from shared human-robot workspaces for training safe co-working behaviors and human intent prediction.
Environmental Diversity
Sensor data from dozens of real warehouses capturing the long-tail distribution of lighting, floor, racking, and layout conditions.
Known Data Requirements
Agility's deployment of Digit in Amazon and GXO warehouses creates acute demand for real-world warehouse navigation data, tote manipulation recordings, and human co-worker interaction patterns. Unlike research humanoids chasing general-purpose intelligence, Agility's commercial focus means they need data that captures the long-tail distribution of real warehouse conditions — varied lighting, floor conditions, obstacle configurations, and workflow patterns — at the fidelity required for safety-critical bipedal locomotion.
Warehouse navigation trajectories on real floors
Source: Agility Robotics partnership announcements with Amazon and GXO Logistics, 2023-2024
Navigation paths through real warehouse aisles with varying congestion, floor conditions (polished concrete, dock plates, slope transitions, seams between floor sections), and dynamic obstacle layouts. Must capture the distributional diversity of floor friction coefficients, liquid spills, and uneven surfaces that simulation cannot faithfully reproduce for bipedal locomotion.
Tote and bin manipulation recordings at scale
Source: Digit product specifications and Amazon BOS27 deployment documentation
Pick-and-place demonstrations for standard warehouse totes (Amazon's yellow and black totes, GXO containers) with varying weights (1-25 kg), fill levels, orientations, and stacking configurations. Multi-viewpoint recordings with force sensing to capture grasp stability and tote-to-tote variation across thousands of picks.
Human proximity and interaction data in shared workspaces
Source: Agility safety documentation, ANSI/RIA safety standards for collaborative robots
Recordings of humans working near bipedal robots in warehouse environments — approach patterns, handoff interactions, collision-avoidance scenarios, and human intent prediction data. Safety-critical: Digit shares workspace with human packers and must predict human trajectories to avoid collisions while maintaining throughput.
Dynamic locomotion on varied terrain and floor surfaces
Source: Agility CTO Jonathan Hurst's published work on bipedal locomotion and DARPA Robotics Challenge heritage
Real-world locomotion data on commercial warehouse floors, loading docks, dock plates with slope transitions, and outdoor surfaces between warehouse buildings. Synchronized IMU, foot-contact, and visual data for validating sim-to-real transfer of locomotion controllers trained in Isaac Gym or MuJoCo.
Conveyor and shelving interaction recordings
Source: Digit deployment videos showing conveyor-loading and shelf-stocking tasks
Manipulation recordings of loading totes onto conveyors, placing items on shelves at varying heights, and retrieving items from deep shelving — tasks that combine locomotion (positioning the body), reaching (arm extension), and manipulation (grasping and placing) in a coordinated sequence.
How Claru Data Addresses These Needs
| Lab Need | Claru Offering | Rationale |
|---|---|---|
| Warehouse navigation trajectories on real floors | Custom Egocentric Collection in Warehouse Environments | Claru's collector network can deploy body-worn cameras and IMUs in partner warehouses across its network of 100+ cities to capture hundreds of hours of navigation data with ground-truth paths, floor surface characterization, obstacle annotations, and environmental metadata. Multiple warehouse facilities provide the floor-condition diversity that single-site data collection misses. |
| Tote and bin manipulation recordings at scale | Manipulation Trajectory Dataset + Custom Warehouse Collection | Claru's manipulation data captures multi-camera object interaction recordings with temporal annotations. Targeted collection campaigns in warehouse facilities produce tote manipulation data with the weight, fill-level, and stacking variation that Digit encounters in production — not the clean, uniform objects found in lab demonstrations. |
| Human proximity and interaction data in shared workspaces | Egocentric Activity Dataset + Custom Human-Robot Interaction Collection | Claru's egocentric video captures humans performing activities in real-world settings, providing visual diversity for training human detection and proximity estimation models. Targeted collection in warehouse environments with natural human traffic patterns provides the interaction data Digit needs for safe co-working in shared spaces. |
| Dynamic locomotion on varied terrain and floor surfaces | Custom Locomotion Data Collection with Body-Worn Sensors | Claru can coordinate body-worn IMU, foot-force sensor, and camera data collection on diverse real-world surfaces — warehouse floors, loading docks, outdoor terrain — across dozens of locations, providing the ground-contact distributional coverage that sim-to-real locomotion transfer requires. |
Technical Data Analysis
Agility Robotics occupies a unique position in the humanoid landscape: rather than pursuing general-purpose intelligence, they have narrowed Digit's initial deployment to warehouse logistics — a domain with predictable task structures but highly variable environmental conditions. This strategic focus means their data needs are simultaneously more specific and more demanding than general humanoid research. They do not need data covering every possible household task, but they need exhaustive coverage of the environmental variation within warehouses.
The core technical challenge for Digit in warehouse settings is robust bipedal locomotion on real warehouse floors. Commercial warehouses have polished concrete with varying friction coefficients, occasional liquid spills, uneven seams between floor sections, and dock plates with slope transitions. Digit's spring-loaded legs — a design inherited from Jonathan Hurst's PhD research at CMU on series-elastic actuators — give it compliance advantages over rigid-legged humanoids, but this compliance also means the locomotion controller is more sensitive to ground-contact dynamics. Simulated locomotion policies trained on flat terrain in Isaac Gym transfer poorly to these conditions. Agility needs hours of real locomotion data collected in actual warehouses with ground-truth foot contact measurements, IMU recordings, and environmental surface characterization.
RoboFab represents a critical inflection point. When Agility opened the world's first humanoid robot factory in Salem, Oregon in 2023, they announced capacity to produce 10,000 Digits per year. This manufacturing scale creates an urgent data scaling challenge: each deployed Digit needs policies robust enough for the specific warehouse it operates in, while benefiting from shared representations learned across the full fleet's experience. The ratio of training data to deployed units must increase dramatically as production scales.
Manipulation presents a different but equally demanding challenge. Digit's end-effectors are designed for grasping standardized warehouse totes — a seemingly simple task that becomes complex when totes vary in weight (1-25 kg), fill level, surface friction (dry versus condensation-covered plastic), and stacking configuration. A tote at the bottom of a stack behaves differently than one on top. A tote packed with heavy items has different inertial properties than one with lightweight goods. Real-world manipulation data must capture these variations across thousands of picks to build policies that generalize beyond the narrow distributions seen in laboratory demonstrations.
The human-robot interaction dimension adds a safety-critical data requirement. Digit operates in shared spaces with human workers at Amazon's BOS27 facility and GXO warehouses, requiring accurate perception of human proximity, intent prediction, and safe motion planning. This demands training data collected in authentic mixed human-robot work environments — something that cannot be ethically or practically simulated at scale.
Key Research & References
- [1]Agility Robotics. “Digit: A Platform for Legged Robot Research and Deployment.” Company Technical Report, 2023. Link
- [2]Radosavovic et al.. “Real-World Humanoid Locomotion with Reinforcement Learning.” Science Robotics, Vol 9, 2024. Link
- [3]Dao et al.. “Sim-to-Real Transfer for Mobile Manipulation in Warehouse Settings.” ICRA 2023, 2023. Link
- [4]Hurst, J.W.. “The Role and Implementation of Compliance in Legged Locomotion.” PhD Thesis, Carnegie Mellon University, 2008. Link
- [5]Agrawal et al.. “Legged Locomotion in Challenging Terrains using Egocentric Vision.” CoRL 2022, 2022. Link
Frequently Asked Questions
Digit requires warehouse navigation trajectories on real floors with varied friction and surface conditions, tote manipulation demonstrations with diverse weights and container types, human-robot interaction data from shared workspaces, and multi-site environmental recordings. The focus is on capturing the distributional diversity of real commercial warehouse conditions that simulation cannot reproduce — especially for bipedal locomotion where ground-contact dynamics are safety-critical.
Commercial warehouses have unique floor friction properties, surface compliance characteristics, lighting conditions, obstacle layouts, and human traffic patterns that simulators approximate poorly. Digit's spring-loaded bipedal locomotion is particularly sensitive to ground contact dynamics — small simulation errors in friction or surface compliance compound into unstable walking on real hardware. The gap between simulated and real warehouse floors is the primary cause of locomotion failures during deployment.
Claru deploys collectors with standardized sensor packages (body-worn cameras, IMUs, force-sensing arrays) in partner warehouse facilities across its network of 100+ cities. Data is collected during actual warehouse operations to capture authentic environmental conditions, workflow patterns, and human activity. Multiple warehouse sites provide the cross-facility variation that single-site collection misses — different floor coatings, racking systems, lighting, and ambient conditions.
RoboFab is the world's first factory purpose-built for manufacturing humanoid robots, opened by Agility in Salem, Oregon in 2023 with capacity to produce 10,000 Digit units per year. This production scale dramatically increases data requirements: each deployed Digit needs policies robust enough for its specific warehouse, while benefiting from shared representations trained across fleet-wide data. The ratio of training data to deployed units must increase with manufacturing volume.
Boston Dynamics' Stretch is a wheeled robot specialized for case unloading. Digit is bipedal, allowing it to navigate stairs, step over obstacles, and operate in spaces designed for human workers. This bipedal capability comes with greater data requirements — locomotion on two legs is fundamentally more complex than wheeled navigation, and Digit must handle the full range of terrain humans traverse in warehouses, including dock plates, ramps, and uneven floor seams.
Support Digit's Real-World Deployment
Discuss purpose-collected warehouse data for Agility Robotics' bipedal logistics applications.