Recording

The core SDK includes a built-in data recorder which leverages FFmpeg for video encoding. The data created using the recorder can used for

  • Troubleshooting (together with Spectacular AI support)

  • Post-processing and exporting for NeRF & 3DGS training

  • Other research and development purposes

The easiest way to use the recorder is through the pre-built recording tools that are included with the SDK packages, e.g., sai-record-realsense or sai-cli record oak.

It can also be enabled in any software integrating the SDK by setting the recordingFolder in the device-specific Configuration class (or the VIO builder). Refer to the wrapper pages for more details and examples.

In all cases, remember to first install FFmpeg.

Data format

The recording format created by the SDK and Spectacular Rec is documented here. It is based on encoded videos and JSONL, making it a convenient and effective choice for high-resolution, high-frequency multi-camera, multi-sensor data. In particular, we can efficiently and economically store

  • Multi-camera RGB or monochrome data

  • Depth data (via PNGs or FFV1)

  • IMU, synchronized with the camera frames

  • Other sensors, e.g., barometer or GPS

Recording settings

It is possible to control FFmpeg encoding parameters using the --ffmpeg_codec command line option (for recording tools) or the ffmpegVideoCodec parameter in vio_config.yaml. For example, the following option disables all encoding and writes raw video, which minmizes the CPU consumption overhead caused by enabling rrecording (on the other hand, this requires a lot of disk space and decent write speed)

ffmpegVideoCodec: rawvideo

The parameter can also be used to enable platform-specific codecs. The following enables hardware-accelerated video recording on the Nvidia Jetson Xavier platform (but not on Orin):

ffmpegVideoCodec: "h264_nvenc -b:v 16M"