Azure Kinect

Spectacular AI C++ plugin for Microsoft Azure Kinect DK.

Note: The Azure Kinect has been discontinued, but nearly equivalent device, ORBBEC Femto is now available and uses the same depth sensing technology, which ORBBEC has licensed from Microsoft.

ORBBEC cameras are supported by their Spectacular AI SDK wrapper. See here.

Quick start

First, download and unpack spectacularAI_k4aPlugin_cpp_non-commercial package (only available for SDK version 1.28)

  1. If you have not used the Azure Kinect device before, you will need to setup udev rules, run the following script (or follow the instructions in Azure Kinect SDK):

  2. Attach your Azure Kinect device to a USB3 port

  3. Run the JSONL example (smoke test):

    cd bin
    # Now you should see rapidly flowing JSONL text
    # press Ctrl+C to exit
  4. Run the Python example (in the bin directory):

    pip install matplotlib # install dependencies

Recording data

Azure Kinect data can be recorded using the sai-record-k4a tool included in the SDK package (in the bin directory). First connect Azure Kinect over USB3, then run (add .exe on Windows):

./sai-record-k4a # Record with visualization, requires pip install spectacularAI[full]
./sai-record-k4a --recording_only --no_preview # More lightweight
./sai-record-k4a --output /path/to/output/folder # Set recording directory
./sai-record-k4a -h # See all options

The source code of recording tool is also available at which allows customizing the recording tool, if necessary.

Installation as a library

First verify that you have installed Azure Kinect SDK, e.g. check that open k4aviewer.

The following commands should be run in the folder where you have unpacked the SDK release package spectacularAI_k4aPlugin_cpp_non-commercial.

  1. First install the development dependencies.

  2. Select where you want the library installed, e.g.,:

    export MY_INSTALL_PREFIX=~/.local
    # or: export MY_INSTALL_PREFIX=`pwd`/installed
  3. Run make PREFIX=$MY_INSTALL_PREFIX install (or sudo make install)

  4. Build the vio_jsonl example using CMake:

    make PREFIX=$MY_INSTALL_PREFIX example
  5. Run the vio_jsonl binary as a smoke test.


The following instructions show how to compile and run the Spectacular AI K4A SDK examples.

  1. (Optional) Install the SDK as a library (see above).

  2. Clone the SDK examples repository:

    git clone --recursive

Run the following commands in the sdk-examples/cpp/k4a directory.

Build the targets using CMake:

mkdir target
cd target
cmake -DspectacularAI_k4aPlugin_DIR=<path/to/spectacularAI_k4aPlugin/install/lib/cmake/spectacularAI/> ..

The -DspectacularAI_k4aPlugin_DIR option is not needed is you have used sudo make install for the SDK.

API Reference

The classes specific to Azure Kinect are documented below. For the main output classes, refer to the C++ documentation in Core SDK.

Defined in #include <spectacularAI/k4a/configuration.hpp>

namespace spectacularAI
namespace k4aPlugin


k4a_device_configuration_t getK4AConfiguration(const std::string &colorResolution = "720p", int depthMode = K4A_DEPTH_MODE_WFOV_2X2BINNED, int frameRate = 30, bool useStereo = true)
std::string getYAMLConfiguration(const Configuration &config)
struct Configuration

Plugin and Spectacular AI VIO SDK configuration variables.

Public Members

bool useStereo = true

Set true to use RGB-D mode, false for mono (RGB camera only).

bool useSlam = true

Enable SLAM, required for ICP

bool fastVio = false

Enable more lightweight VIO parameters

bool alignedDepth = false

Align depth images to color images using Azure Kinect SDK.

Note that this affects ICP performance since the depth image has larger field of view than the color camera.

std::string recordingFolder = ""
bool recordingOnly = false
bool postProcessFinalMap = false

If true, final mapping API output is refined using post processing.

std::string aprilTagPath = ""

Path to .json file with AprilTag information. AprilTag detection is enabled when not empty. For the file format see: Note: sets useSlam=true

k4a_device_configuration_t k4aConfig = getK4AConfiguration()

Configuration for the k4a device.

bool ignoreOutputs = false

If false, user should read vio outputs from the Session. If true, the SDK discards vio outputs. Useful when VIO outputs are not needed: e.g. when using mapping API or recording.

std::map<std::string, std::string> internalParameters

Internal SDK parameter overrides (key-value pairs). Not safe for unsanitized user input.

Defined in #include <spectacularAI/k4a/pipeline.hpp>

namespace spectacularAI
namespace k4aPlugin
class Pipeline

The Pipeline object stores information about the device and also sets the default configuration, e.g., enables and configures necessary Azure Kinect streams and sensors. The actual VIO reading is started with startSession.

Public Functions


Create a pipeline with the default configuration.

Pipeline(const Configuration &config)

Create a pipeline with custom configuration

Pipeline(const Configuration &config, std::function<void(mapping::MapperOutputPtr)> onMapperOutput)

Create a pipeline with custom configuration with Mapping API enabled

std::unique_ptr<Session> startSession()

Start a new VIO session on the background. Internally starts the k4a device and creates threads for camera and imu.

k4a_device_t getDeviceHandle()

Get an handle to the Azure Kinect device, can be used to adjust device settings. Note that adjusting some settings, e.g. device FPS can affect VIO performance.


Private Members

std::unique_ptr<impl> pimpl

Defined in #include <spectacularAI/k4a/session.hpp>

namespace spectacularAI
namespace k4aPlugin
struct Session

Public Functions

virtual bool hasOutput() const = 0

Check if new output is available.

virtual std::shared_ptr<const VioOutput> getOutput() = 0

Get output from the queue, if available. If not returns {}.

virtual std::shared_ptr<const VioOutput> waitForOutput() = 0

Wait until new output is available and then return it.

virtual void ignoreOutputs(bool ignore) = 0

Set whether new outputs are added to the queue. Useful when the user is not interested in VIO outputs: e.g. when using mapping API or recording.


ignore – If true, new outputs are not added to the queue. Otherwise, the user has make sure the outputs are pulled from the queue.

virtual void addTrigger(double t, int tag) = 0

Add an external trigger input. Causes additional output corresponding to a certain timestamp to be generated.

  • t – timestamp, monotonic float seconds

  • tag – additonal tag to indentify this particular trigger event. The default outputs corresponding to input camera frames have a tag 0.

virtual ~Session()