Plumerai Video Intelligence C API¶

This document describes the C API for the Plumerai Video Intelligence library for videos.

The C API wraps the C++ API and is as similar as possible. Since classes are not supported in C, a CPlumeraiVideoIntelligence object is introduced which needs to be passed to every function as first argument. This object can be created using plumerai_video_intelligence_create, after which each video frame should be processed by calling the process_frame function. The various intelligence features such as object detection and familiar face identification are documented on sub-pages found in the navigation menu.

Please refer to the minimal example for example code to get started.

The API is re-entrant, i.e. you can initialize several video intelligence objects in different threads and use them independently. However, using the same instance from different threads at the same time is not supported.

Video Intelligence¶

create¶

CPlumeraiVideoIntelligence
plumerai_video_intelligence_create(int height, int width);

Initializes a new Video Intelligence object.

This needs to be called only once at the start of the application.

Arguments:

height: The height of the input image in pixels.
width: The width of the input image in pixels.

destroy¶

void plumerai_video_intelligence_destroy(
    CPlumeraiVideoIntelligence pvi);

Destroy the CPlumeraiVideoIntelligence object.

This needs to be called once at the very end to clean up.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

process_frame¶

PlumeraiErrorCode
plumerai_video_intelligence_process_frame_packed_rgb888(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, float delta_t);

PlumeraiErrorCode
plumerai_video_intelligence_process_frame_packed_rgba8888(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, float delta_t);

PlumeraiErrorCode
plumerai_video_intelligence_process_frame_packed_bgra8888(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, float delta_t);

PlumeraiErrorCode
plumerai_video_intelligence_process_frame_packed_yuyv(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, float delta_t);

PlumeraiErrorCode
plumerai_video_intelligence_process_frame_planar_yuv420(
    CPlumeraiVideoIntelligence pvi, const uint8_t* plane0,
    const uint8_t* plane1, const uint8_t* plane2, float delta_t);

PlumeraiErrorCode
plumerai_video_intelligence_process_frame_planar_nv12(
    CPlumeraiVideoIntelligence pvi, const uint8_t* plane0,
    const uint8_t* plane1, float delta_t);

Process a single frame from a video sequence.

Make sure the image is right side up. When it is upside down it can still work but accuracy is significantly degraded.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object
image_data: A pointer to the image data (planeX for planar image formats).
delta_t: The time in seconds it took between this and the previous video frame (1/fps). If set to 0, then the system clock will be used to compute this value.

Returns:

An error code of type PlumeraiErrorCode. See that enum for more details.

single_image¶

PlumeraiErrorCode
plumerai_video_intelligence_single_image_packed_rgb888(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, int height,
    int width);

PlumeraiErrorCode
plumerai_video_intelligence_single_image_packed_rgba8888(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, int height,
    int width);

PlumeraiErrorCode
plumerai_video_intelligence_single_image_packed_bgra8888(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, int height,
    int width);

PlumeraiErrorCode
plumerai_video_intelligence_single_image_packed_yuyv(
    CPlumeraiVideoIntelligence pvi, const uint8_t* image_data, int height,
    int width);

PlumeraiErrorCode
plumerai_video_intelligence_single_image_planar_yuv420(
    CPlumeraiVideoIntelligence pvi, const uint8_t* plane0,
    const uint8_t* plane1, const uint8_t* plane2, int height, int width);

PlumeraiErrorCode
plumerai_video_intelligence_single_image_planar_nv12(
    CPlumeraiVideoIntelligence pvi, const uint8_t* plane0,
    const uint8_t* plane1, int height, int width);

Process a single image not part of a video sequence.

This should not be used for video data. It can be used for face enrollments from a set of images. The object detection box id values obtained after calling plumerai_video_intelligence_single_image are not related to those generated through plumerai_video_intelligence_process_frame or through other calls to plumerai_video_intelligence_single_image.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object
image_data: A pointer to the image data (planeX for planar image formats).
height: The height of the input image in pixels. If height = 0 the height set in the constructor will be used.
width: The width of the input image in pixels. If width = 0 the width set in the constructor will be used.

Returns:

An error code of type PlumeraiErrorCode. See that enum for more details.

set_night_mode¶

PlumeraiErrorCode plumerai_video_intelligence_set_night_mode(
    CPlumeraiVideoIntelligence pvi, bool night_mode);

Configure the video intelligence algorithm for either day mode color videos (default) or night mode IR videos.

This configures the video intelligence algorithm for optimal performance on day mode color data (default) or on night mode IR data.

After switching from day to night mode or back, the motion detection algorithm will need a couple of video frames to stabilize, so the motion-grid will not be immediately available.

This function does not have to be called before every frame, only when switching from RGB to IR video data or back.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
night_mode: Set to true for night mode or false for day mode.

Returns:

Returns PlumeraiErrorCode::SUCCESS on success.

debug_next_frame¶

PlumeraiErrorCode plumerai_video_intelligence_debug_next_frame(
    CPlumeraiVideoIntelligence pvi, const uint8_t** debug_data_buffer,
    size_t* debug_data_size);

Enable debug mode for the next frame.

The next time plumerai_video_intelligence_process_frame is called, this will dump the input image as well as internal data and final results to binary blob, meant to be stored to a file. This file can then be shared with Plumerai support for further analysis. These files contain uncompressed image data and can become several megabytes large.

The user will receive a pointer to the data, but this will only be available after the next call to plumerai_video_intelligence_process_frame, see the example code. The data is only valid for one frame, and will be invalidated after the second call to plumerai_video_intelligence_process_frame, or another call to plumerai_video_intelligence_debug_next_frame.

This function is only available on platforms that support dynamic memory allocation.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
debug_data_buffer: An output parameter that receives a pointer to the debug data, after the next call to plumerai_video_intelligence_process_frame.
debug_data_size: An output parameter that receives the size of the debug data, after the next call to plumerai_video_intelligence_process_frame.

Returns:

Returns PlumeraiErrorCode::SUCCESS if all went well. Returns PlumeraiErrorCode::NOT_AVAILABLE on platforms where this functionality is not available. It can return PlumeraiErrorCode::INTERNAL_ERROR if this method is called twice without calling plumerai_video_intelligence_process_frame.

code_version¶

const char* plumerai_video_intelligence_code_version(
    CPlumeraiVideoIntelligence pvi);

Returns the version of the video intelligence code as a date and time.

For other version numbers, see also plumerai_object_detection_detector_version for the object detector, and plumerai_face_identification_embedding_version for the face embedder.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

The version of the code as YYYY.MM.DD.HH.MM date and time string.

Upgrade guide¶

From version 1.x to 2.0¶

The name 'People Detection' was changed to 'Video Intelligence' to reflect support for other detection classes and features such as advanced motion detection.
- The library filename changed from libplumerai{peopledetection,faceidentification} to libplumeraivideointelligence. This should be updated in the relevant build scripts.
- The main header file is now plumerai/video_intelligence_c.h instead of plumerai/people_detection_c.h, plumerai/face_identification_c.h or plumerai/face_identification_automatic_c.h.
Most function have been renamed. They now follow the scheme plumerai_<feature>_<function-name> where <feature> is one of detection_zones, face_enrollment_automatic, face_enrollment_manual, face_identification, motion_detection, object_detection or video_intelligence.
The function plumerai_video_intelligence_process_frame no longer returns bounding boxes as output. Instead, the boxes are now accessible through plumerai_object_detection_get_detections after the frame has been processed.

Code that looked like this before:

#include "plumerai/people_detection_c.h"

CPeopleDetection ppd = PeopleDetectionInit(height, width);

BoxPrediction predictions[10];
int num_results = 0;
PlumeraiErrorCode error_code = PeopleDetectionProcessFrame(
    ppd, image, predictions, 10, &num_results, delta_t);

Should be updated to this:

#include "plumerai/video_intelligence_c.h" // (1)

// Initialize the video intelligence algorithm (2)
CPlumeraiVideoIntelligence pvi =
    plumerai_video_intelligence_create(height, width);

// Process a video frame (3)
PlumeraiErrorCode error_code =
    plumerai_video_intelligence_process_frame_packed_rgb888(pvi, image, delta_t);

// Get bounding box results (4)
const BoxPrediction *predictions = NULL;
size_t num_results = 0;
error_code = plumerai_object_detection_get_detections(pvi, &predictions,
                                                      &num_results);

Replace plumerai/people_detection_c.h, plumerai/face_identification_c.h or plumerai/face_identification_automatic_c.h by plumerai/video_intelligence_c.h.
Replace CPeopleDetection by CPlumeraiVideoIntelligence and PeopleDetectionInit by plumerai_video_intelligence_create.
Replace PeopleDetectionProcessFrame by plumerai_video_intelligence_process_frame_<format>. Remove the predictions and num_results output arguments.
Get bounding box results through the new plumerai_object_detection_get_detections function.

From version 1.14 to 1.15¶

The confidence_threshold argument has been removed from the single_image function. The type PlumeraiErrorCode was introduced. Most functions now return this error code. The process-frame and single-image functions now return the number of results through an output argument instead of the return value.

If your code looked like this before:

int num_results =
    PeopleDetectionProcessFrame(ppd, image, predictions, 10, delta_t);

Then it should be updated as follows:

int num_results = 0;
PlumeraiErrorCode error_code = PeopleDetectionProcessFrame(
    ppd, image, predictions, 10, &num_results, delta_t);
if (error_code != PLUMERAI_SUCCESS) {
  printf("Error: %s\n", PeopleDetectionErrorCodeString(error_code));
}