Workplace Egocentric Video Data for General-Purpose Robotics

Robotics training datasets are overwhelmingly collected in controlled lab environments or staged settings that fail to represent real-world work conditions — cluttered spaces, time pressure, improvised tool use, and the contextual decision-making that workers perform unconsciously. Existing egocentric video datasets capture household tasks performed by research participants, but the distribution gap between a researcher's kitchen and a commercial kitchen during service is enormous. The lab needed a partner who could embed data collection into actual business operations without disrupting workflows, while meeting the quality and diversity requirements of frontier robotics research across multiple industries and geographies.

We embedded data capture directly into real-world business operations across multiple countries and 10 workplace categories. Business owners and workers were onboarded as contributors through a lightweight side-revenue model that kept participation voluntary and minimally disruptive to normal workflow.

Workplace categories spanned food service (barista, cooking), skilled trades (carpentry, tailoring, screen printing), repair services (phone repair, tool repair), textile work (clothing shop, ironing), and assembly (furniture assembly, paper cutting). Tasks were designed for handheld smartphone recording at 4K 60fps — no specialized hardware required. Research-level specifications for camera angle, framing, and activity coverage were translated into practical instructions that respected workplace realities: space constraints, safety requirements, hygiene protocols, and varying levels of technical comfort.

Activity coverage and task diversity were tracked continuously through a real-time monitoring dashboard. We balanced collection across workplace types and across task complexity levels within each type. QA validation focused on the characteristics that distinguish genuine workplace data from staged alternatives: natural pacing, contextual tool selection, environmental adaptation, and multi-step task sequencing under real constraints.

The program established a fundamentally new data source for robotics research: active workplaces as scalable, cost-efficient contributors of egocentric training data. Ten distinct workplace categories — from barista stations to carpentry workshops to screen printing studios — demonstrated that the approach generalizes across industries, not just food service. The behaviors captured — improvisation under time pressure, adaptation to cluttered and constrained spaces, contextual tool selection — represent precisely the distribution that lab-collected datasets lack. The operational model (side-revenue for businesses, smartphone capture, lightweight onboarding) is economically viable for sustained collection at scale.