Protocol Buffers for Robotics: Complete Guide for Robotics Data

Protocol Buffers (protobuf), developed by Google and open-sourced in 2008, define data schemas in .proto files using a compact Interface Definition Language (IDL). A robotics observation message might be defined as: message Observation { bytes image = 1; repeated float joint_positions = 2; double timestamp = 3; Pose gripper_pose = 4; }, where each field has a unique numeric tag used for binary encoding. The proto3 syntax (current standard since 2016) uses variable-length encoding for integers (1-10 bytes depending on value), fixed-width encoding for floats and doubles, and length-delimited encoding for strings, bytes, and nested messages. This produces serialized payloads that are 3-10x smaller than equivalent JSON and 20-100x faster to parse.

In robotics, protobuf serves three distinct roles: real-time message passing (via gRPC with sub-millisecond serialization), data logging (via MCAP containers or raw binary files), and dataset distribution (as used by the Waymo Open Dataset). The schema-first approach ensures that producer and consumer always agree on the data structure, with backward compatibility maintained through field numbering rules: new fields can be added without breaking old readers, and deprecated fields are never reused. For robotics teams managing data across collection robots, processing servers, and training clusters, this schema evolution capability is critical because sensor configurations change across data collection campaigns.

The protobuf compiler (protoc) generates language-specific code from .proto files for C++ (libprotobuf), Python (protobuf package), Go, Java, C#, Rust (prost), and more. In a typical robotics pipeline, the same .proto file generates C++ code for the real-time robot controller (zero-allocation serialization at 1 kHz+), Python code for data processing and ML training, and Go/Java code for cloud services. The proto3 Any type allows embedding arbitrary message types with self-describing type URLs, which MCAP uses to store heterogeneous message streams in a single container. The proto3 map type provides efficient key-value storage, useful for per-object annotations in scene descriptions.

The standard workflow for protobuf-based robotics data begins with schema definition. A well-structured robotics .proto file defines messages for each data type: SensorFrame (timestamp, sensor_id, data as oneof { Image, PointCloud, IMU }), RobotState (joint_positions, joint_velocities, gripper_state), Action (joint_commands or end_effector_deltas), and Episode (repeated SensorFrame observations, repeated Action actions, metadata). Using protobuf's oneof for polymorphic sensor data avoids the overhead of storing empty fields for unused sensor types. The repeated keyword defines variable-length arrays, and bytes fields store opaque binary blobs (JPEG images, compressed point clouds) without protobuf overhead.

For data logging during robot operation, MCAP (developed by Foxglove) has emerged as the standard container format for protobuf messages. MCAP wraps timestamped protobuf messages with per-channel metadata and a chunk-based index structure that enables O(log n) time-range queries without reading the entire file. A typical MCAP file from a robot logging session contains channels for each sensor (camera images at 30 Hz, LiDAR at 10 Hz, joint state at 100 Hz, force/torque at 1 kHz), with each message serialized as protobuf. The MCAP specification supports chunk compression (LZ4 or ZSTD), achieving 2-4x compression on typical robotics data while maintaining random access through the chunk index.

Converting protobuf-logged data to ML training formats is a common pipeline step. For Waymo Open Dataset, the official processing pipeline reads protobuf TFRecords (tf.train.Example containing serialized Waymo-specific protobuf messages), extracts camera images, LiDAR range images, and 3D annotations, and converts to framework-specific formats (KITTI, nuScenes, or custom). For MCAP-logged data, the mcap Python library (pip install mcap mcap-protobuf-support) provides an iterator-based reader: for schema, channel, message in reader.iter_messages(), then MyProtoMessage.FromString(message.data) deserializes each message. This read-convert-write pattern is the standard approach for turning protobuf-logged operational data into training-ready datasets.

In production robotics deployments, protobuf serves as the universal data language connecting all system components. The robot controller publishes joint states, sensor readings, and status messages as protobuf over gRPC at 100-1000 Hz. A logging service subscribes to these streams and writes timestamped protobuf messages to MCAP files for post-hoc analysis. A perception service receives camera and LiDAR protobuf messages, runs inference, and returns detections as protobuf response messages. This uniform serialization layer means that any component can be replaced or upgraded independently as long as the .proto contract is maintained.

The Waymo Open Dataset demonstrates protobuf at scale for ML training data. Each TFRecord file contains serialized tf.train.Example messages wrapping Waymo-specific protobuf messages (Frame, CameraImage, RangeImage, Label). A single Frame message encodes data from 5 cameras and 5 LiDAR sensors with full 3D annotations, serializing to approximately 10 MB. The protobuf schema defines nested structures for camera calibration (CameraCalibration message), LiDAR calibration (LaserCalibration), and 3D labels (Label with Box message containing center_x/y/z, width, length, height, heading). This level of schema formalization makes it possible to process the 1000+ hour dataset programmatically without ambiguity in field interpretation.

For teams transitioning from development (where data is often stored in ad-hoc formats) to production (where schema consistency is critical), protobuf provides a migration path. Starting with a simple .proto file defining your robot's observation and action spaces, you can progressively add fields for new sensors, new annotation types, and operational metadata. Protobuf's backward compatibility rules guarantee that data written with older schemas remains readable with newer schemas, and tools like buf (buf.build) provide linting, breaking change detection, and registry services for managing .proto files across teams.

Claru provides complete .proto schema definitions alongside data deliveries for teams using protobuf-based pipelines. Schemas are designed following protobuf best practices: field numbering reserves ranges for future extension, oneof types handle polymorphic sensor data, and nested messages represent complex structures like camera calibration and 6D poses. All .proto files are compatible with proto3 syntax and validated with buf lint for style consistency.

Data can be delivered as serialized protobuf messages within MCAP containers (with LZ4 or ZSTD chunk compression, indexed for time-range queries), as standalone binary protobuf files (one per episode or one per frame), or as TFRecord files with protobuf payload for TensorFlow ecosystem compatibility. For teams using gRPC-based robot communication, we provide service definitions (.proto files with service and rpc declarations) that enable streaming data delivery directly into your robot's existing gRPC infrastructure. Every delivery includes a Python example script demonstrating message deserialization and conversion to NumPy arrays.