Plumerai Object and Motion Detection API¶

This document describes the API for the object detection functionality.

The motion detection functionality is not yet available in the Python API.

BoxPrediction¶

class DetectionClass(Enum):
    CLASS_UNKNOWN = 0
    CLASS_PERSON = 1
    CLASS_HEAD = 2
    CLASS_FACE = 3
    CLASS_VEHICLE = 4,
    CLASS_ANIMAL = 5,
    CLASS_PACKAGE = 6,

class BoxPrediction(NamedTuple):
    y_min: float  # top coordinate between 0 and 1 in height dimension
    x_min: float  # left coordinate between 0 and 1 in width dimension
    y_max: float  # bottom coordinate between 0 and 1 in height dimension
    x_max: float  # right coordinate between 0 and 1 in width dimension
    confidence: float  # between 0 and 1, higher means more confident
    id: int = 0  # the tracked identifier of this box
    class_id: int = DetectionClass.CLASS_UNKNOWN  # the class of the object

A structure representing a single resulting bounding box. Coordinates are between 0 and 1, the origin is at the top-left.

Confidence values lie between 0 and 1. Note that the algorithm comes with a built-in threshold (e.g. 0.6 - this differs per model and per class): boxes with confidences lower than that value won't be produced at all by the Plumerai software.

ObjectDetection¶

get_detections¶

ObjectDetection.get_detections() -> tuple[ErrorCode, tuple[BoxPrediction, ...]]

Obtain the object detections from the most recently processed frame.

Note that the algorithm comes with a built-in threshold (e.g. 0.6 - this differs per model): boxes with confidences lower than that value won't be produced at all by this function.

Arguments:

• None.

Returns:

• Nothing.

has_box_moved¶

ObjectDetection.has_box_moved(
    box: BoxPrediction, timeout_seconds: float = 20.0
) -> tuple[ErrorCode, bool]

Check if a box has moved significantly since its initial detection.

This function should only be used for boxes from the most recent video frame. This function only accepts boxes of the Person, Vehicle, Animal and Package classes.

The resulting boolean output parameter will be set to:

• false if the box has not moved since its initial detection
• true if the box has moved

If a box has moved but then stopped moving for at least 20 seconds, the has_moved value will be reset to false. This timeout can be changed by specifying the optional timeout_seconds parameter.

This function should not be called directly after restoring from a previous state.

This function should not be called with boxes obtained directly after VideoIntelligence.single_image calls.

Arguments:

• box: A box from the most recent video frame.
• timeout_seconds: The number of seconds after which the has_moved flag will be reset. The default value is 20s. It is not allowed to set this value larger than the default.

Returns:

• A tuple with a SUCCESS or INVALID_BOX value, and a boolean that will be set to true if the box has moved.

reset_tracker¶

ObjectDetection.reset_tracker() -> None

Reset the internal tracker state.

This function is only available if the library was built with tracking support.

This resets the internal tracker state and resets all tracker ids and face identifications if applicable. It is recommended to call this whenever two consecutive frames are too different from each other, such as when switching to a different camera input or when the camera abruptly moved.

Arguments:

• None.

Returns:

• Nothing.

detector_version¶

ObjectDetection.detector_version(cls) -> int

Returns a version number of the object detector neural network.

Arguments:

• None.

Returns:

• The version number of the object detector.

MotionDetection¶

get_grid_height¶

MotionDetection.get_grid_height() -> int

Retrieves the height of the motion detection grid.

For more information, see the docs under get_grid.

Arguments:

• None.

Returns:

• The height of the motion detection grid.

get_grid_width¶

MotionDetection.get_grid_width() -> int

Retrieves the width of the motion detection grid.

For more information, see the docs under get_grid.

Arguments:

• None.

Returns:

• The width of the motion detection grid.

set_grid_size¶

MotionDetection.set_grid_size(height: int, width: int) -> ErrorCode

Set the size of the motion detection grid.

There is a default grid size which scales with the input resolution, so it is not necessary to call this function. If a custom grid size is desired, this function can be called at the start of the application, before processing any frames.

Calling this function re-initializes the motion-detection algorithm so this should only be called at the start of the application.

For more information, see the docs under get_grid.

Arguments:

• height: The height of the motion detection grid.
• width: The width of the motion detection grid.

Returns:

• An error code of type ErrorCode. It will return SUCCESS if all went well, or INVALID_GRID_SIZE if the supplied grid size is invalid.

get_grid¶

MotionDetection.get_grid() -> tuple[ErrorCode, list[float]]

Retrieves the amount of motion found in each grid cell of the frame.

As a by-product of object detection, motion detection is performed. For specific use-cases, it might be useful to access this raw motion detection information as well. This function provides access to it, but it won't be available in the first few frames.

The result of this function is a 2D grid of type float, with dimensions that can be retrieved with get_grid_height and get_grid_width. The values in each grid cell are floats between 0.0 and 1.0, and denote how much motion was detected in that grid cell. A higher value indicates more motion. The height is the outer dimension and the width is the inner dimension.

Arguments:

• None.

Returns:

• A tuple with an error code of type ErrorCode and a list with the result motion grid values. It will return SUCCESS if all went fine, or MOTION_GRID_NOT_YET_READY if this function is called too soon after initialization of the VideoIntelligence object. It needs to process at least a few frames before the motion grid is valid.

DetectionZones¶

add_zone¶

DetectionZones.add_zone(
    coordinates: list[tuple[float, float]], classes: list[DetectionClass]
) -> tuple[ErrorCode, int]

Specify a detection zone polygon for a group of classes.

The zone can be used to verify whether a bounding-box is inside or outside of it using the is_box_in_zone function. It can also be used internally by the Plumerai library to improve detection quality. A single detection zone can be used for one or more classes.

Arguments:

• coordinates: A list of (x, y) coordinate pairs specifying the polygon of the detection zone in normalized coordinates, between 0 and 1. The code assumes that the first coordinate is also the final coordinate of the polygon: it should not be given by the user. The polygon must be simple (not complex): it can't have holes or self-intersections. It is allowed to be both concave or convex.
• classes: A list of classes for which this detection zone is valid.

Returns:

• A tuple of an error code and the created detection zone's unique ID, for use in is_box_in_zone. The error code will be SUCCESS if all went fine, or otherwise INVALID_ZONE_CLASS or INVALID_ZONE_GEOMETRY.

remove_zone¶

DetectionZones.remove_zone(zone_id: int) -> ErrorCode

Remove an existing detection zone.

Arguments:

• zone_id: The detection zone ID as returned by add_zone.

Returns:

• An error code of type ErrorCode. It will return SUCCESS if all went fine, or otherwise INVALID_ZONE_ID.

is_box_in_zone¶

DetectionZones.is_box_in_zone(
    zone_id: int, box: BoxPrediction
) -> tuple[ErrorCode, bool]

Determines whether a box prediction is within a detection zone.

Arguments:

• zone_id: The detection zone ID as returned by 'add_zone'.
• box: A bounding-box prediction.

Returns:

• A tuple of an error code and the resulting boolean that indicates whether the given box is completely or partially inside the given detection zone in the current frame. The error code will be SUCCESS if all went fine, or otherwise INVALID_ZONE_ID, INVALID_ZONE_CLASS or INVALID_ZONE_GEOMETRY.