Active Learning: Smarter Annotation for Robot Training Data

Active learning is a semi-supervised machine learning framework in which the learning algorithm can interactively query an oracle — typically a human annotator — to label selected data points from a pool of unlabeled examples. The core principle is that not all data points are equally informative: some lie near decision boundaries, in underrepresented regions of the feature space, or in areas where the model is uncertain, and labeling these strategically yields more model improvement per annotation dollar than uniform random sampling.

The standard active learning loop operates in four steps. First, the model is trained on the currently labeled dataset. Second, the model evaluates all unlabeled data points according to a query strategy (uncertainty sampling, query-by-committee, expected model change, or information density). Third, the top-k most informative points are sent to human annotators for labeling. Fourth, the newly labeled points are added to the training set and the model is retrained. This cycle repeats until the annotation budget is exhausted or performance targets are met.

In robotics, active learning operates at two levels. At the annotation level, it selects which frames or trajectory segments from an already-collected dataset need human labels. A robot that has collected 100,000 frames of kitchen manipulation does not need every frame labeled — active learning identifies the 10,000-20,000 frames where labels would be most informative (novel objects, unusual configurations, ambiguous contact states). At the data collection level, active learning can guide which environments, tasks, or scenarios need additional demonstrations, directing collection resources toward the gaps in the current dataset.

The effectiveness of active learning depends on the query strategy, the model architecture, and the data distribution. For classification and detection tasks, uncertainty sampling (selecting points where the model is least confident) is the simplest and most widely used approach. For regression tasks common in robotics (predicting continuous action values), expected model change and Bayesian methods that estimate epistemic uncertainty tend to perform better because output confidence is less meaningful for continuous predictions.

Implementing active learning for a robotics data pipeline requires decisions about the query strategy, batch size, annotation interface, and stopping criteria.

For perception annotation (bounding boxes, segmentation masks, keypoints), pool-based active learning with batched queries is the standard approach. The model scores all unlabeled frames, selects a batch of 100-500 frames for annotation, and retrains after each batch. Batch sizes smaller than 50 create excessive overhead from model retraining; batches larger than 1,000 reduce the benefit of active selection because the model does not update frequently enough. In practice, 200-500 frames per batch balances annotation throughput with selection quality.

For trajectory annotation (labeling robot demonstrations as successful/failed, segmenting actions within a trajectory, or scoring trajectory quality), the unit of selection is a full trajectory or trajectory segment rather than a single frame. Query strategies should account for trajectory-level diversity — selecting 100 trajectories that all involve the same object provides less information than 100 trajectories spanning different objects, grasp types, and failure modes.

The cold-start problem is a practical challenge: active learning requires an initial model to estimate informativeness, but the initial model is trained on a small random seed set. In robotics, teams typically label 5-10% of the dataset uniformly at random as the seed set, train the initial model, then switch to active selection for the remaining budget. Using pretrained feature extractors (CLIP, DINOv2) to compute embedding-space coverage for the seed selection improves the initial model quality.

Stopping criteria matter for budget management. Common approaches include monitoring validation performance curves and stopping when the marginal gain per annotation batch drops below a threshold, or using a held-out test set and stopping when target performance is reached. For production robotics, the performance threshold is typically defined by the minimum success rate needed for deployment.

Claru integrates active learning into our annotation pipelines to maximize the value of every labeled frame. When clients deliver large pools of unlabeled robot data, our pipeline uses model-guided selection to prioritize the frames and trajectories where annotation will have the highest impact on downstream model performance.

This approach allows Claru to deliver annotation projects 40-60% faster than exhaustive labeling while maintaining or improving model accuracy. For teams on tight budgets or timelines, active learning means getting to a deployable model with less annotation spend — directing human attention where it matters most rather than uniformly across redundant data.