Open3D Format: Complete Guide for Robotics Data

Open3D works with three primary 3D data formats, each with distinct strengths. PCD (Point Cloud Data) files, originally defined by the Point Cloud Library (PCL), store point clouds with a structured header specifying fields (e.g., x, y, z, rgb, normal_x, normal_y, normal_z), their sizes and types, the total point count, and the data encoding (ascii, binary, or binary_compressed). The binary_compressed variant uses LZF compression, reducing file sizes by 30-50% for typical robotics point clouds while maintaining fast decompression. PLY (Polygon File Format, also called Stanford Triangle Format) supports both point clouds and polygon meshes, with a header defining element types (vertex, face) and their properties. PLY is the preferred format when both point data and mesh topology are needed, such as for reconstructed 3D scenes or object models used in grasp planning.

Open3D's native dataset format for RGBD reconstruction pipelines stores registered sequences as a directory structure: trajectory.json (camera poses as 4x4 transformation matrices for each frame), a depth/ directory with 16-bit PNG depth images, and a color/ directory with RGB images. The trajectory.json file contains an array of 4x4 homogeneous transformation matrices representing the camera-to-world transform for each frame, enabling TSDF (Truncated Signed Distance Function) volumetric integration to reconstruct dense 3D surfaces. This format is used by Open3D's reconstruction pipeline (open3d.pipelines.integration.ScalableTSDFVolume) and is interoperable with ElasticFusion, BundleFusion, and other real-time reconstruction systems.

Open3D 0.18+ introduced a tensor-based data model (open3d.t.geometry) that stores point clouds and meshes as dictionaries of named tensors on CPU or GPU. This tensor geometry system supports arbitrary per-point and per-vertex attributes stored as named Open3D tensors, enabling seamless interoperability with PyTorch and TensorFlow via DLPack zero-copy transfer. For ML training on 3D data, the tensor API eliminates the conversion overhead between Open3D's legacy geometry objects and framework-specific tensor types, achieving throughput of 50-100 million points per second for operations like voxel downsampling and normal estimation on modern GPUs.

Open3D provides unified I/O for all supported 3D formats. Reading a point cloud is a single call: pcd = o3d.io.read_point_cloud('scene.pcd') or o3d.io.read_point_cloud('scene.ply'), returning an open3d.geometry.PointCloud with .points (Nx3 float64), .colors (Nx3 float64, 0-1 range), and .normals (Nx3 float64) attributes. For meshes: mesh = o3d.io.read_triangle_mesh('model.ply') returns an object with .vertices, .triangles, .vertex_normals, and .vertex_colors. The format is auto-detected from the file extension, and Open3D supports reading PCD (ascii/binary/binary_compressed), PLY (ascii/binary), OBJ, STL, OFF, and GLTF formats.

Writing follows the same pattern: o3d.io.write_point_cloud('output.pcd', pcd, write_ascii=False, compressed=True) writes a binary compressed PCD file. For best compatibility with PCL-based C++ code, use PCD format with binary encoding. For exchange with Blender, MeshLab, or web viewers, PLY binary is preferred. The key performance consideration for large point clouds (10M+ points) is to avoid ASCII encoding, which is 5-10x slower to read and 3-5x larger on disk than binary. For RGBD sequences, o3d.io.read_pinhole_camera_trajectory('trajectory.json') loads the camera poses, and the reconstruction pipeline is initialized with o3d.pipelines.integration.ScalableTSDFVolume(voxel_length=0.005, sdf_trunc=0.04).

For ML workflows, Open3D's tensor-based API (open3d.t.io) provides zero-copy conversion to PyTorch tensors. Loading a point cloud as tensors: tpcd = o3d.t.io.read_point_cloud('scene.ply'), then tpcd.point.positions.to(o3d.core.Device('cuda:0')) moves points to GPU, and tpcd.point.positions.to_dlpack() provides DLPack interop with PyTorch via torch.from_dlpack(). This pipeline achieves sub-millisecond conversion overhead for point clouds up to 1 million points, making it practical for on-the-fly data loading in training loops. Open3D-ML extends this with dataset classes for S3DIS, ScanNet, SemanticKITTI, and Toronto3D that handle format-specific loading and provide standardized train/val/test splits.

Open3D serves as the 3D processing backbone for many robotic manipulation systems. In a typical pick-and-place pipeline, depth cameras capture RGBD frames that are converted to point clouds using Open3D's create_from_rgbd_image() with the camera intrinsic matrix. These point clouds are then processed through a chain of Open3D operations: statistical outlier removal (remove_statistical_outlier) to clean noise, plane segmentation (segment_plane) to remove the table surface, DBSCAN clustering (cluster_dbscan) to segment individual objects, and ICP registration (registration_icp) to align detected objects against known 3D models for pose estimation. This entire pipeline runs at 5-15 Hz on a modern CPU depending on point cloud density.

For 3D scene reconstruction in robotics, Open3D's TSDF integration pipeline converts a sequence of RGBD frames with known camera poses into a dense 3D mesh or point cloud. The ScalableTSDFVolume class uses a voxel hashing approach that allocates voxels only where data exists, enabling memory-efficient reconstruction of large scenes. A typical reconstruction with 5mm voxel resolution and 500 RGBD frames at 640x480 produces a mesh with 1-5 million vertices, consuming 2-4 GB of memory. The resulting mesh is directly usable for collision checking in motion planning (via Open3D's raycasting scene), navigation map generation, and digital twin creation.

Open3D-ML extends the library with deep learning models for 3D semantic segmentation, which is increasingly important for scene understanding in robotics. Models like RandLA-Net (efficient random sampling for large-scale point clouds), KPConv (kernel point convolution for deformable 3D features), and PointTransformer (attention-based 3D understanding) are available as pre-trained Open3D-ML pipelines that ingest Open3D point cloud objects directly. Training on custom robotics datasets requires providing point clouds with per-point label arrays, which Open3D stores as custom attributes on the PointCloud object.

Claru delivers 3D data in Open3D-compatible formats (PCD binary compressed, PLY binary, or NumPy arrays) with calibration files for direct loading into Open3D processing pipelines. Point clouds include per-point RGB colors, surface normals, and optional semantic labels as custom PCD fields. For RGBD sequence datasets, we provide Open3D-compatible trajectory.json files with camera poses, enabling direct use with the TSDF reconstruction pipeline.

For manipulation datasets, Claru provides pre-segmented object point clouds with per-object PLY meshes suitable for ICP-based pose refinement and collision checking. For 3D semantic segmentation training, we deliver point clouds with per-point labels in Open3D-ML compatible format, with class taxonomies defined for your target environment (warehouse shelving, kitchen surfaces, factory floor). Large-scale point cloud deliveries (100M+ points) are pre-partitioned into spatial tiles with overlap regions to enable distributed processing and training.