KITTI Format: Complete Guide for Robotics Data

KITTI format uses a file-based directory structure developed by the Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago for their 2012 vision benchmark suite. The format organizes data by modality: images in /image_02/ (left color camera) and /image_03/ (right color camera), LiDAR point clouds in /velodyne/ as binary float32 files (each point stored as 4 floats: x, y, z, reflectance), calibration data in /calib/ as text files containing projection matrices, and labels in /label_02/ as space-delimited text files. Each frame is identified by a 6-digit zero-padded index (000000.png, 000000.bin, 000000.txt), and all modalities for a given frame share the same index number.

The KITTI label format encodes 3D bounding boxes as a single line per object with 15 space-delimited fields: type (class name like Car, Pedestrian, Cyclist), truncated (float 0-1 indicating how much of the object is outside the image), occluded (integer 0-3 from fully visible to fully occluded), alpha (observation angle in radians), 2D bbox (left, top, right, bottom in pixels), 3D dimensions (height, width, length in meters), 3D location (x, y, z of the bottom center in camera coordinates), and rotation_y (yaw rotation around the vertical Y-axis in radians). The 3D bounding box is parameterized as a 7-DoF representation: (x, y, z, h, w, l, ry), where the center is defined as the bottom face center of the box. This convention means the Y-axis points downward (camera convention), which is a common source of confusion when interfacing with LiDAR-centric systems that use Z-up conventions.

The calibration files contain four projection matrices (P0-P3 for each camera) and two transformation matrices (Tr_velo_to_cam for Velodyne LiDAR to camera 0, and Tr_imu_to_velo for IMU to Velodyne). The projection matrices are 3x4 and encode both camera intrinsics and the rigid transformation from the reference camera (camera 0) to each target camera. To project a LiDAR point to the left color image (camera 2): first apply Tr_velo_to_cam to transform from Velodyne to camera 0 coordinates, then left-multiply by the rectification matrix R0_rect (3x3 stored as 3x4 with zero translation), and finally by P2 (3x4) to get homogeneous pixel coordinates. This multi-step projection pipeline is the most implementation-intensive aspect of working with KITTI format.

Reading KITTI point clouds is straightforward: points = np.fromfile('000000.bin', dtype=np.float32).reshape(-1, 4) yields an (N, 4) array of [x, y, z, reflectance]. Images are standard PNG files loadable with cv2.imread() or PIL.Image.open(). Label files are parsed line-by-line, splitting on whitespace: each line produces a dictionary with type, truncated, occluded, alpha, bbox (4 floats), dimensions (3 floats), location (3 floats), and rotation_y. The calibration file contains key-value lines like P2: followed by 12 floats representing the 3x4 projection matrix.

Writing KITTI-format data requires careful attention to coordinate conventions. Point clouds must be in the Velodyne coordinate frame (X-forward, Y-left, Z-up) and written as contiguous float32 binary: points.astype(np.float32).tofile('000000.bin'). Labels must use the camera 2 coordinate frame (X-right, Y-down, Z-forward), which means applying the Velodyne-to-camera transformation to any LiDAR-centric annotations before writing. The 2D bounding box must be the tight axis-aligned enclosure of the projected 3D box corners in image space, and the alpha angle is the observation angle (not the global yaw), computed as alpha = rotation_y - arctan2(location_z, location_x).

The KITTI evaluation toolkit (devkit provided by the benchmark organizers) computes Average Precision (AP) at three difficulty levels: Easy (minimum bbox height 40px, max occlusion 0, max truncation 0.15), Moderate (25px, occlusion 1, truncation 0.30), and Hard (25px, occlusion 2, truncation 0.50). For 3D detection, the IoU threshold is 0.7 for cars and 0.5 for pedestrians and cyclists. The evaluation computes both BEV (Bird's Eye View) AP and 3D AP. Many 3D detection papers report the 40-point interpolated AP (R40) rather than the original 11-point interpolated AP (R11), a change introduced by the KITTI benchmark in 2019 that produced different numerical values for the same detections.

Although KITTI format was designed for autonomous driving, its simplicity has led to widespread adoption in other robotics domains. Indoor navigation systems use KITTI format for RGB-D SLAM evaluation by substituting depth camera data for LiDAR point clouds. Agricultural robotics teams use the label format for 3D crop detection with modified category names. Warehouse robotics systems adopt KITTI format for forklift and pallet detection because the extensive ecosystem of pre-trained 3D detectors (PointPillars, SECOND, CenterPoint) can be fine-tuned on domain-specific KITTI-format data with minimal code changes.

The KITTI format's main limitation is the lack of native temporal linking between frames. In the original KITTI benchmark, tracking sequences are stored in a separate split with consecutive frame indices, but there is no explicit mechanism to link annotations across time (no tracking IDs in the detection label format, though the tracking split adds them). Newer formats like nuScenes address this with linked sample tokens, and Argoverse 2 uses unique track UUIDs. For teams that need temporal consistency in KITTI format, the standard workaround is to add a track_id column to the label files (position 16), which most KITTI-compatible loaders ignore but can be parsed by custom code.

Despite these limitations, KITTI format remains the most broadly supported format in 3D detection research as of 2026. The OpenPCDet framework alone supports over 20 different detection architectures that all consume KITTI-format data, and nearly every new 3D detection paper includes KITTI benchmark results. This means that training a model on KITTI-format data gives you access to the largest collection of pre-trained checkpoints, training recipes, and community-maintained codebases of any 3D detection format.

Claru delivers data in KITTI format with complete calibration files ensuring geometric consistency between camera images, LiDAR point clouds, and 3D annotations. All labels follow the standard 15-field format with accurate 3D bounding boxes, occlusion levels, and truncation estimates. Point clouds are delivered as binary float32 files in the Velodyne coordinate frame with reflectance values, and calibration matrices are computed from rigorous multi-target camera-LiDAR calibration procedures.

For teams migrating from or benchmarking against the original KITTI dataset, Claru maintains strict format compatibility verified by running the official KITTI evaluation toolkit against delivered data. For applications beyond standard automotive categories, we define custom class taxonomies (e.g., pallet, forklift, shelf for warehouse robotics) while maintaining the same label file structure. Large deliveries include pre-computed train/val splits following KITTI's Moderate difficulty distribution, and we provide OpenPCDet-compatible configuration files for immediate training with any of the 20+ supported detection architectures.