Skip to content

Plumerai Familiar Face Identification Python API

This document describes the Python API for the Plumerai Familiar Face Identification with automatic enrollment software for videos.

The API

The Python API consists of a single class with a constructor that needs to be ran once, and a process_frame function that needs to be executed on each input frame.

FaceIdentificationAutomatic

init

FaceIdentificationAutomatic.__init__(height: int, width: int)

Initializes a new people detection object.

This needs to be called only once at the start of the application.

Arguments:

  • height:: The height of the input image in pixels.
  • width:: The width of the input image in pixels.

Returns:

  • Nothing.

process_frame

FaceIdentificationAutomatic.process_frame(
    image, delta_t: float = 0.0
) -> tuple[ErrorCodeFamiliarFaceIDAutomatic, tuple[BoxPrediction, ...]]

Process a single frame from a video sequence with RGB input.

The image must have the height and width that was specified when the FaceIdentificationAutomatic object was created.

Arguments:

  • image: A tensor of shape (video_height, video_width, 3) with RGB image data. It can be a Numpy, TensorFlow, PyTorch or Jax tensor.
  • delta_t: The time in seconds it took between this and the previous video frame (1/fps). If left to the default of 0, then the system clock will be used to compute this value.

Returns:

get_face_ids

FaceIdentificationAutomatic.get_face_ids() -> List[int]

Return the face ids of all automatically enrolled faces.

Arguments:

  • None.

Returns:

  • A list of integers, each representing a face id.

get_face_id

FaceIdentificationAutomatic.get_face_id(person_box: BoxPrediction) -> int

Retrieve the face ID that belongs to a person box.

Retrieves the face ID given a person box returned by process_frame. This function has three possible return values: - FACE_ID_UNIDENTIFIED when the provided box is not of CLASS_PERSON or when a face is not yet identified, e.g. the face is not fully visible. - FACE_ID_NOT_IN_LIBRARY when the face is visible but not found in the library, e.g. when a stranger is in view. - Any non-negative integer face-ID provided when a familiar face is identified. This function should not be called directly after restoring from a previous state.

Arguments:

  • person_box: A box returned by process_frame with class_id equal to DetectionClass::CLASS_PERSON.

Returns:

  • A non-negative integer face-ID when a face is identified, FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY when it is not identified successfully, or FACE_ID_UNIDENTIFIED when the provided box is not a valid person box.

BoxPrediction

A structure representing a single resulting bounding box. Coordinates are between 0 and 1, the origin is at the top-left.

class BoxPrediction(NamedTuple):
    y_min: float  # top coordinate between 0 and 1 in height dimension
    x_min: float  # left coordinate between 0 and 1 in width dimension
    y_max: float  # bottom coordinate between 0 and 1 in height dimension
    x_max: float  # right coordinate between 0 and 1 in width dimension
    confidence: float  # between 0 and 1, higher means more confident
    id: int = 0  # the tracked identifier of this box
    class_id: int = DetectionClass.CLASS_UNKNOWN  # the class of the object

DetectionClass

class DetectionClass(Enum):
    CLASS_UNKNOWN = 0
    CLASS_PERSON = 1
    CLASS_HEAD = 2
    CLASS_FACE = 3

ErrorCodeFamiliarFaceIDAutomatic

Error codes returned which can be returned by the API.

class ErrorCodeFamiliarFaceIDAutomatic(Enum):
    # All went well
    SUCCESS = 0

    # Should not occur, contact Plumerai if this happens
    INTERNAL_ERROR = -1

    # The `delta_t` parameter should be >= 0
    INVALID_DELTA_T = -2

    # The state could not be restored
    STATE_CORRUPT = -4

    # Returned by `remove_face_embedding` when the provided face ID is invalid.
    UNKNOWN_FACE_ID = -2001

    # Returned if the face library is full.
    FACE_LIBRARY_FULL = -2002

Constants

FACE_ID_UNIDENTIFIED = -1
FACE_ID_NOT_IN_LIBRARY = -2

If the face-identification model is not confident about a face, it will output FACE_ID_UNIDENTIFIED. If the model is confident that a face is not in the face library, it will output FACE_ID_NOT_IN_LIBRARY.

Example usage

Below is an example of using the Python API shown above.

import numpy as np
import plumerai_people_detection as ppd_api

# Settings, to be changed as needed
width = 1600  # camera image width in pixels
height = 1200  # camera image height in pixels

# Initialize the `FaceIdentificationAutomatic` object
ffid = ppd_api.FaceIdentificationAutomatic(height, width)

# Loop over frames in a video stream (example: 20 frames)
for t in range(20):
    # Some example input here, normally this is where camera data is acquired
    image = np.zeros((height, width, 3), dtype=np.uint8)

    # The time between two video frames in seconds. In this example we assume
    # a constant frame rate of 30 fps, but variable rates are supported.
    delta_t = 1. / 30.

    # Process the frame
    error_code, predictions = ffid.process_frame(image, delta_t)
    if error_code != ppd_api.ErrorCodeFamiliarFaceIDAutomatic.SUCCESS:
      raise RuntimeError(f"Error in 'process_frame': {error_code}")

    # Display the results
    for p in predictions:
        if p.class_id != ppd_api.DetectionClass.CLASS_PERSON:
          continue
        # After a few frames of showing a clear face in view, the face-ID should
        # become non-negative as the face is automatically enrolled.
        face_id = ffid.get_face_id(p)
        print(
            f"Box #{p.id} with face ID {face_id} with confidence {p.confidence} "
            f"@(x,y)->({p.x_min:.2f},{p.y_min:.2f})-({p.x_max:.2f},{p.y_max:.2f})"
        )
    if len(predictions) == 0:
        print("No bounding boxes found in this frame")