Spectacular AI SDK documentation

Spectacular AI SDK fuses data from cameras and IMU sensors (accelerometer and gyroscope) and outputs an accurate 6-degree-of-freedom pose of a device. This is called Visual-Inertial SLAM (VISLAM) and it can be used in, among other cases, tracking (autonomous) robots and vehicles, as well as Augmented, Mixed and Virtual Reality.

The SDK also includes a Mapping API that can be used to access the full SLAM map for both real-time and offline 3D reconstruction use cases.

SDK architecture

The main parts of the SDK are documented on their own pages:

  • Core SDK: device-indepencent APIs shared between all devices supported by the SDK.

  • Wrappers: for devices supported out-of-the-box.

The SDK also includes a set of tools implemented on top of the APIs, such as the sai-cli process command line tool for NeRFs and Gaussian Splatting.

System requirements

Operating system. Windows or Linux. For Linux, Ubuntu 18+ is recommended, but other distribution (e.g., Yocto) can be supported as well. If the binaries do not work out-of-the-box, contact us for commercial support.

Architecture Binaries for x86-64 are available on the SDK releases. For Linux 64-bit ARM binaries are also available with a commercial license contact us for more info.

CPU. The SDK is designed to work in real-time on Cortex-A57 type or better embedded processors. This includes Jetson Nano and Raspberry Pi 4 (A72). Slower processors, such as Cortex-A53 work under certain conditions or with reduced accuracy and functionality. At least two A5x or better cores are required.

RAM. 6-DoF pose tracking works with less than 50MB of RAM. However, certain optional features or configuration changes related to the Mapping API increase can the RAM consumption.

Software dependencies. The Core SDK itself has zero non-trivial external software dependencies. However, Wrappers, SDK examples and tools built on the SDK have additional dependencies, which are not all designed to work on embedded platforms (especially the Python-based 3D visualizations). See the relevant documentation sub-pages for more info.

Development dependencies

For development purposes, we recommend a relatively recent Linux or Windows laptop computer.

Windows

The following setup is recommended to run the SDK examples

For C++ development, you additionally need

Linux

You need at least Git, and Python and Pip

sudo apt update
sudo apt install python-is-python3 python3-pip git

For C++ development, you also need a relatively recent compiler (GCC or Clang) and CMake, e.g.,

sudo apt install cmake clang build-essential

FFmpeg

To use the recoding, replay API or tools depending on these APIs, you need to install FFmpeg. Linux

apt install ffmpeg

On Windows, see here. FFmpeg must be in your PATH so that ffmpeg works on the command line.