Calibration

The Spectacular AI SDK includes a tool to easily calibrate camera setups. It computes camera intrinsics for several supported distortion models, and also stereo and camera-IMU extrinsics. Note that calibration is not generally required for any out-of-the-box supported devices listed on the wrappers page, but is relevant for custom hardware integrations.

Monocular camera-IMU calibration

  1. Setup the 6x6 Aprilgrid pattern on a computer monitor as large as possible. Measure the target.

  2. Create a Spectacular AI SDK recording of the target:

  • See Recording for how to record. If using the SDK API directly, input any valid placeholder calibration.json.

  • Refer to the tutorial video to make sure you include all required movements.

  • The calibration target must be stationary and the device moved around it.

  1. Install FFmpeg.

  2. Install and run the Spectacular AI calibration tool on the recording directory (here path/to/recording):

pip install "spectacularAI[full]"
sai-cli calibrate path/to/recording --tag_size 0.03209876 --output_html report.html --output calibration.json

where --tag_size gives in meters the value you got from measuring the target and dividing by 8.1.

  1. Open the generated report.html file in a web browser. The report shows whether the calibration was successful, and if not, indicates likely cause of failure.

  2. Input the generated calibration.json in the Spectacular AI SDK APIs.

Stereo camera-IMU calibration

Stereo camera calibration follows the same steps as monocular calibration, with the additional requirement of having enough video where both cameras see the calibration target simultaneously to accurately estimate stereo extrinsics.

  • When recording, first perform the calibration motions individually for each camera. Then, move the camera up & down and back & forth while keeping the target visible to both cameras. All motions should be part of the same recording.

The report.html includes an extra section for stereo calibration, showing the coverage of the stereo overlap.

Below is an example of footage with good stereo overlap, the target is fully visible in both cameras. This is typically not an issue unless the cameras have a large vergence angle and a small overlap area:

Camera-only calibration

Spectacular AI SDK requires IMU-camera calibration, so this mode is not sufficient to use most Spectacular AI features. However, it can be helpful if the camera calibration parameters are required for other purposes.

You can also calibrate a single camera without an IMU from any video file:

  1. Setup the 6x6 Aprilgrid pattern on a computer monitor as large as possible.

  2. Record a video of the target by moving the device around the stationary target. Refer to the tutorial video to make sure you include all required movements.

  3. Install FFmpeg.

  4. Install and run the Spectacular AI calibration tool on the video file (here video.mp4):

pip install "spectacularAI[full]"
sai-cli calibrate video.mp4

Example output:

Enabling visual-only calibration mode.

--- Camera calibration phase
Corner detection progress 100%
Failed detections 385/1429.
Removed 385/1429 invalid poses.

Estimating initial extrinsics.

Initialized optimization.
Iteration 30/70. Mean reprojection error: 0.17661550
Skipping IMU-camera extrinsics calibration.

Calibration result:
{
    "cameras": [
        {
            "distortionCoefficients": [ 1.7928108031038417, 0.4750649683947041, 1.0520005629849567e-05, 7.304509697616892e-06, 0.009304462386450309, 2.148028322920538, 1.0030979033407466, 0.07366554523759501 ],
            "focalLengthX": 596.3194856012566,
            "focalLengthY": 596.8101085057813,
            "imageHeight": 800,
            "imageWidth": 1280,
            "imuToCamera": [
                [ 1.0, 0.0, 0.0, 0.0 ],
                [ 0.0, 1.0, 0.0, 0.0 ],
                [ 0.0, 0.0, 1.0, 0.0 ],
                [ 0.0, 0.0, 0.0, 1.0 ]
            ],
            "model": "brown-conrady",
            "principalPointX": 610.3153299232023,
            "principalPointY": 416.25375348502
        }
    ]
}

If you wish to verify quality of the generated calibration, see the full calibration instructions above on how to use the --output_html option.

Additional information

To get full list of available options, run sai-cli calibrate --help.

There also exists a more detailed Calibration Manual that documents the camera models and conventions used by the SDK.