Skip to content

Plumerai Video Intelligence Python API

This document describes the Python API for the Plumerai Video Intelligence library for videos.

The Python files are self-documented. The main entrypoint is the VideoIntelligence class which provides access to all functionality of the Plumerai software. It has a process_frame function that needs to be executed on each input frame. The various intelligence features such as object detection and familiar face identification are available through different interfaces, listed here.

Please refer to the minimal examples for example code to get started.

VideoIntelligence

VideoIntelligence

VideoIntelligence(height: int, width: int)

Initializes a new Video Intelligence object.

This needs to be called only once at the start of the application.

Arguments:

  • height: The height of the input image in pixels.
  • width: The width of the input image in pixels.

Returns:

  • Nothing.

process_frame

VideoIntelligence.process_frame(image, delta_t: float = 0.0) -> ErrorCode

Process a single frame from a video sequence with RGB input.

The image must have the height and width that was specified when the VideoIntelligence object was created.

Make sure the image is right side up. When it is upside down it can still work but accuracy is significantly degraded.

Arguments:

  • image: A tensor of shape (video_height, video_width, 3) with RGB image data. It can be a Numpy array or TensorFlow, PyTorch or Jax tensor.
  • delta_t: The time in seconds it took between this and the previous video frame (1/fps). If left to the default of 0, then the system clock will be used to compute this value.

Returns:

single_image

VideoIntelligence.single_image(image) -> ErrorCode

Process a single image not part of a video sequence.

This should not be used for video data. It can be used for face enrollments from a set of images. The object detection box id values obtained after calling single_image are not related to those generated through process_frame or through other calls to single_image.

Arguments:

  • image: A tensor of shape (*, *, 3) with RGB image data. It can be a Numpy array or TensorFlow, PyTorch or Jax tensor.

Returns:

store_state

VideoIntelligence.store_state() -> tuple[ErrorCode, bytes]

Store the current state of the algorithm to a byte array.

This function can be used when processing a video in chunks, doing different chunks at different times or on different machines. The state can be restored by calling restore_state with the data returned by store_state. When the library is built with support for familiar face identification, the state includes the face library. Constraints:

  • The delta_t parameter of process_frame can not be left to zero after restoring a previous state.
  • If familiar face identification is enabled, the state can only be stored and restored when not enrolling.

Arguments:

  • None.

Returns:

  • A tuple with an error code of type ErrorCode and a bytes object to store the serialized state in

restore_state

VideoIntelligence.restore_state(state: bytes) -> ErrorCode

Restore the state of the algorithm from a byte array.

See store_state for more information. The user must ensure that the height and width of the current object match the height and width of the state that is being restored.

Arguments:

  • state: A vector containing the serialized state.

Returns:

  • Returns SUCCESS on success. Returns STATE_CORRUPT or STATE_SETTINGS_MISMATCH on error.

debug_next_frame

VideoIntelligence.debug_next_frame() -> DebugData

Enable debug mode for the next frame.

The next time process_frame is called, this will dump the input image as well as internal data and final results to binary blob, meant to be stored to a file. This file can then be shared with Plumerai support for further analysis. These files contain uncompressed image data and can become several megabytes large.

The user will receive a DebugData instance, but the data inside it will only be available after the next call to process_frame. The data is only valid for one frame, and will be invalidated after the second call to process_frame, or another call to debug_next_frame.

Arguments:

  • None.

Returns:

  • Returns a DebugData object, which has a pointer to the data, which can be accessed through its data attribute.

code_version

VideoIntelligence.code_version(cls) -> str

Returns the version of the video intelligence code as a date and time.

For other version numbers, see also ObjectDetection::detector_version for the object detector, and FaceIdentification::embedding_version for the face embedder.

Arguments:

  • None.

Returns:

  • The version of the code as YYYY.MM.DD.HH.MM date and time string.

object_detection

@property
VideoIntelligence.object_detection -> ObjectDetection

Get the interface for the ObjectDetection video intelligence features.

motion_detection

@property
VideoIntelligence.motion_detection -> MotionDetection

Get the interface for the MotionDetection video intelligence features.

detection_zones

@property
VideoIntelligence.detection_zones -> DetectionZones

Get the interface for the DetectionZones video intelligence features.

face_identification

@property
VideoIntelligence.face_identification -> FaceIdentification

Get the interface for the FaceIdentification video intelligence features.

face_enrollment_automatic

@property
VideoIntelligence.face_enrollment_automatic -> FaceEnrollmentAutomatic

Get the interface for the FaceEnrollmentAutomatic video intelligence features.

face_enrollment_manual

@property
VideoIntelligence.face_enrollment_manual -> FaceEnrollmentManual

Get the interface for the FaceEnrollmentManual video intelligence features.