Workplace Egocentric Video Data for General-Purpose Robotics
Challenge:Robotics training datasets are overwhelmingly collected in controlled lab environments or staged settings that fail to represent real-world work conditions — cluttered spaces, time pressure, improvised tool use, and the contextual decision-making that workers perform unconsciously.
Solution:We embedded data capture directly into real-world business operations across multiple countries and 10 workplace categories.
Result:The program established a fundamentally new data source for robotics research: active workplaces as scalable, cost-efficient contributors of egocentric training data.
Robotics training datasets are overwhelmingly collected in controlled lab environments or staged settings that fail to represent real-world work conditions — cluttered spaces, time pressure, improvised tool use, and the contextual decision-making that workers perform unconsciously. Existing egocentric video datasets capture household tasks performed by research participants, but the distribution gap between a researcher's kitchen and a commercial kitchen during service is enormous. The lab needed a partner who could embed data collection into actual business operations without disrupting workflows, while meeting the quality and diversity requirements of frontier robotics research across multiple industries and geographies.
We embedded data capture directly into real-world business operations across multiple countries and 10 workplace categories. Business owners and workers were onboarded as contributors through a lightweight side-revenue model that kept participation voluntary and minimally disruptive to normal workflow.
Workplace categories spanned food service (barista, cooking), skilled trades (carpentry, tailoring, screen printing), repair services (phone repair, tool repair), textile work (clothing shop, ironing), and assembly (furniture assembly, paper cutting). Tasks were designed for handheld smartphone recording at 4K 60fps — no specialized hardware required. Research-level specifications for camera angle, framing, and activity coverage were translated into practical instructions that respected workplace realities: space constraints, safety requirements, hygiene protocols, and varying levels of technical comfort.
Activity coverage and task diversity were tracked continuously through a real-time monitoring dashboard. We balanced collection across workplace types and across task complexity levels within each type. QA validation focused on the characteristics that distinguish genuine workplace data from staged alternatives: natural pacing, contextual tool selection, environmental adaptation, and multi-step task sequencing under real constraints.
Workplace footage on Explore
Public clips from Claru Explore's work-activity folders, shown muted.
- Natural pacingWork happens at real speed under real time pressure, not slowed down for the camera.
- Contextual tool selectionWorkers pick and switch tools the way they do on the job.
- Environmental adaptationFootage shows work around clutter, space limits and safety constraints.
- Multi-step sequencingTasks run as connected steps under real constraints, not isolated motions.
- Coverage balanceCollection balanced across workplace types and task complexity, tracked on a live dashboard.
Formatted egocentric datasets per workplace category.
- Smartphone egocentric video
- Handheld 4K, 60 fps recordings made during normal workflow.
- Workplace-type metadata
- Each clip tied to its workplace category.
- Activity labels
- Per-clip labels for the work performed.
The program established a fundamentally new data source for robotics research: active workplaces as scalable, cost-efficient contributors of egocentric training data. The first 10 workplace categories proved the model generalizes beyond food service; the program has since grown to 17 categories each carrying 100+ accepted hours — retail, construction, agriculture, personal care, restaurants, carpentry, textile, warehouse, and more — with accepted capture in 42 countries from 2,300+ approved contributors. The behaviors captured — improvisation under time pressure, adaptation to cluttered and constrained spaces, contextual tool selection — represent precisely the distribution that lab-collected datasets lack, and the operational model (side-revenue for businesses, smartphone capture, lightweight onboarding) remains economically viable for sustained collection at scale.
General-purpose robotics requires training data from the environments where robots will actually be deployed. The behaviors captured in active work environments — the improvisation, the physical constraints, the contextual decision-making — cannot be replicated in staged settings. This program showed that real workplaces across 17 categories are a scalable, cost-efficient dataset source without requiring specialized hardware or disrupting operations. For robotics labs, it opens a data collection channel that scales with the number of participating businesses rather than the number of available research facilities.
Representative record from the annotation pipeline.
Espresso prep, milk steaming, order assembly
Ingredient prep, stove work, plating
Sawing, sanding, joint assembly
Cutting, stitching, fitting
Screen prep, ink application, drying
Disassembly, component swap, testing
Diagnosis, part replacement, calibration
Fabric handling, folding, display
Steam pressing, garment finishing
Part alignment, fastening, hardware install
- Explore: Metal fabrication & weldingPublic first-person fabrication clips
- Explore: Auto service & repairPublic first-person repair-shop clips
- Egocentric Workshop Video DatasetWorkshop and maker-space footage
- Egocentric Restaurant Video DatasetFood-service environments
- Welding Task Training DataWelding tasks and data requirements
- Tool Use Training DataTool selection and handling
Ready to build your next dataset?
Tell us about your project and we will scope a plan within 48 hours.