CLARU
Data Collection

Narrated Egocentric Video for Language-Grounded Learning

4,700+ hoursNarrated egocentric video accepted
summary.md

Challenge:Language-conditioned robot policies and grounded video models need data where action and description co-occur: first-person video in which the demonstrator narrates intent, steps, and object interactions in natural spoken language.

Solution:We ran a dedicated narration program alongside our household egocentric collection: 698 approved contributors wear cameras and narrate their own activities in their own words — what they are doing, what they pick up, what happens next.

Result:The corpus pairs visual demonstration with natural language at a scale usable for language-conditioned imitation learning and grounded instruction following: every clip is action-synchronized narration rather than post-hoc captioning, and cohort-level language tagging makes the accent and language mix a controllable training variable.

4,700+ hoursAccepted narrated egocentric hours
18,417Accepted RGB + spoken-narration clips
26 countriesCountries with accepted narration capture
2,830 hrsPeak accepted hours in one week
// THE CHALLENGE

Language-conditioned robot policies and grounded video models need data where action and description co-occur: first-person video in which the demonstrator narrates intent, steps, and object interactions in natural spoken language. Collecting this is harder than silent egocentric video — narration must actually describe the action, stay synchronized with it, and remain audible and natural rather than read from a script. The buyer needed thousands of hours of it, across varied English and regional accents, under commercial licensing.

// OUR APPROACH

We ran a dedicated narration program alongside our household egocentric collection: 698 approved contributors wear cameras and narrate their own activities in their own words — what they are doing, what they pick up, what happens next. Spoken audio is recorded on the same timeline as video, so narration can be transcribed and aligned to actions without manual syncing.

Every clip passes an acceptance gate that checks narration quality, not just video quality: the description must match the visible action and stay synchronized with it. Cohorts are organized by narration language — US/CA/AU/NZ/EU cohorts narrate in English, while India and Norway cohorts narrate in local languages — so downstream teams can control language mix explicitly rather than inheriting it.

Delivery ran in weekly waves that peaked at 2,830 accepted hours, with per-cohort and per-country tracking so the buyer could rebalance accent and activity coverage as the corpus grew.

  1. 01RecruitOnboard narrating contributors by language cohort with consented capture agreements
  2. 02CaptureFirst-person video with live self-narration on a shared timeline
  3. 03GateAcceptance checks on narration quality, action match, and synchronization
  4. 04DeliverWeekly waves with per-cohort tracking; peak week 2,830 accepted hours

One narrated clip, one clock

Video
first-person RGB
Action
step 1
step 2
step 3
Narration
speech
pause
speech
speech

Schematic of a single clip. Speech is recorded on the same timeline as the video, so each stretch of narration lines up with the action it describes without manual syncing.

// FOOTAGE

Household footage on Explore

Public household clips from Claru Explore, shown muted. Explore previews carry no audio; the narration program's spoken track is part of its delivery, not of these previews.

Measuring and stirring ingredients into a pot
Folding a ribbed garment, overhead view
Arranging saucers on a lace tablecloth
Preparing a drink next to a gas stove
Browse egocentric footage on Explore
// QUALITY GATES
  1. Narration matches the actionThe spoken description must describe what is visible in the clip.
  2. Narration stays in syncSpeech must line up with the action as it happens, not trail behind it.
  3. Natural, unscripted speechContributors describe their own activity in their own words, not a read-aloud script.
  4. Video quality still appliesNarration is checked on top of the visual review, not instead of it.
// RESULTS
4,700+ hoursAccepted narrated egocentric hours
18,417Accepted RGB + spoken-narration clips
26 countriesCountries with accepted narration capture
2,830 hrsPeak accepted hours in one week
// WHAT THE CUSTOMER RECEIVED

Accepted first-person clips with the narrator's spoken audio recorded on the same timeline.

RGB video + spoken audio
18,417 accepted clips across four narration projects, video and speech captured together.
Language cohort
Each clip carries its cohort, so language and accent mix can be set when assembling training sets.
Country
Accepted capture tracked across 26 countries.
Weekly waves
Per-cohort and per-country tracking; the peak week delivered 2,830 accepted hours.
// IMPACT

The corpus pairs visual demonstration with natural language at a scale usable for language-conditioned imitation learning and grounded instruction following: every clip is action-synchronized narration rather than post-hoc captioning, and cohort-level language tagging makes the accent and language mix a controllable training variable.

// WHY IT MATTERS

Silent demonstration video teaches what happened; narrated video teaches why, in language a policy or model can condition on. Data where the demonstrator explains their own actions in their own words is among the scarcest training data for embodied AI — it cannot be scraped, must be consented and captured live, and loses its value the moment narration drifts from the action. That is exactly what the acceptance gate protects.

// DECISIONS WE MADE
Narrate live, not after the factwhyNarration recorded during the action carries intent and order that post-hoc captions miss.
Organize cohorts by languagewhyDownstream teams control language mix explicitly instead of inheriting it.
Gate on narration, not only videowhyA clip whose narration drifts from the action loses its value as language supervision.
Record audio on the video's clockwhyTranscripts align to actions without manual syncing.
// KEEP EXPLORING
// RELATED
// FAQ

Ready to build your next dataset?

Tell us about your project and we will scope a plan within 48 hours.