Plumerai Familiar Face Identification API¶

This document describes the API for the Familiar Face Identification functionality.

Note that the software is built with either the Automatic Enrollment option or the Manual Enrollment option enabled. This page provides documentation for both options, but at runtime only one of the two will be available.

FaceIdentification¶

get_face_id¶

FaceIdentification.get_face_id(person_box: BoxPrediction) -> int

Retrieve the face ID that belongs to a person box.

Retrieves the face ID given a person box from the most recent video frame. This function has three possible return values:

FACE_ID_UNIDENTIFIED when the provided box is not of CLASS_PERSON or when a face is not yet identified, e.g. the face is not fully visible.
FACE_ID_NOT_IN_LIBRARY when the face is visible but not found in the library, e.g. when a stranger is in view.
Any integer face-ID provided previously to FaceEnrollmentManual.add_embedding when a familiar face is identified.

Arguments:

person_box: A box from the most recent video frame with class_id equal to DetectionClass::CLASS_PERSON.

Returns:

Returns an integer face-ID as provided when calling add_face_embedding when a face is identified, FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY when it is not identified successfully, or FACE_ID_UNIDENTIFIED when the provided box is not a valid person box.

get_person_box_from_face_box¶

FaceIdentification.get_person_box_from_face_box(
    face_box: BoxPrediction
) -> tuple[ErrorCode, BoxPrediction]

Retrieve a person box corresponding to a face box.

This function should not be called with results of VideoIntelligence.single_image calls.

Arguments:

face_box: The target face to find a match for. This has to be a box from the most recent video frame.

Returns:

Return a tuple of an error code and the person box, only valid if the error code was SUCCESS. Otherwise, the error code can be NO_BOX_MATCH if it was not possible to uniquely match a face box to a person box or returns INVALID_BOX in case of invalid arguments.

get_face_box_from_person_box¶

FaceIdentification.get_face_box_from_person_box(
    person_box: BoxPrediction
) -> tuple[ErrorCode, BoxPrediction]

Retrieve a face box corresponding to a person box.

This function should not be called with results of VideoIntelligence.single_image calls.

Arguments:

person_box: The target person to find a match for. This has to be a box from the most recent video frame.

Returns:

Return a tuple of an error code and the face box, only valid if the error code was SUCCESS. Otherwise, the error code can be NO_BOX_MATCH if it was not possible to uniquely match a face box to a person box or returns INVALID_BOX in case of invalid arguments.

embedding_version¶

FaceIdentification.embedding_version(cls) -> int

Returns a version number for the face embedding data used by the face identification logic.

This version can be used to check if the face embedding data generated with one version of the Plumerai software are compatible with those generated by another version of the Plumerai software.

Arguments:

None.

Returns:

The version number of the face embedding data.

FaceEnrollmentAutomatic¶

configure_face_snapshots¶

FaceEnrollmentAutomatic.configure_face_snapshots(
    enabled: bool, height: int, width: int
) -> ErrorCode

Configure the face snapshots for automatic enrollment.

By default, face snapshots are disabled.

Calling this function will invalidate any stored face snapshots of faces that are already enrolled. Trying to retrieve those face snapshots will return uninitialized data.

Arguments:

enabled: Whether to enable or disable the face snapshots.
height: The height of the face snapshots.
width: The width of the face snapshots.

Returns:

An error code of type ErrorCode.

get_face_snapshot¶

FaceEnrollmentAutomatic.get_face_snapshot(face_id: int) -> tuple[ErrorCode, bytes]

Returns a snapshot of an automatically enrolled face.

Note that FACE_ID_UNIDENTIFIED can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

None.

Returns:

An error code of type ErrorCode and the resulting snapshot bytes.

get_face_ids¶

FaceEnrollmentAutomatic.get_face_ids() -> tuple[ErrorCode, list[int]]

Return the face ids of all automatically enrolled faces.

This list of face ids is not related to the faces currently in view. The face ids returned by this function are all faces that have ever been enrolled into the face library. To retrieve information about the faces currently in view, use FaceIdentification.get_face_id.

Arguments:

None.

Returns:

An error code of type `ErrorCode and a list of integers, each representing a face id.

tag_face_id¶

FaceEnrollmentAutomatic.tag_face_id(face_id: int, important: bool = True) -> ErrorCode

Tag a face id as 'important' to prevent it from being overwritten.

When the face library is full and a new face is encountered, the algorithm will overwrite the oldest face in the library that is not tagged as important. A typical use-case of this is to tag a face as important when the user has named them in a front-end application.

When all entries in the face library are tagged as important and a new face is encountered, then no new faces will be enrolled.

By default, an automatically enrolled face is not tagged as important. However, library entries created by restore_embedding_data are automatically tagged as important.

Note that UNKNOWN_FACE_ID can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

face_id: A non-negative integer face-id.
important: Whether to tag the face as important or to remove a previously applied tag.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID when the specified face id could not be found.

remove_embedding¶

FaceEnrollmentAutomatic.remove_embedding(face_id: int) -> ErrorCode

Remove a face embedding from the face library for any VideoIntelligence::process_frame calls that follow.

Note that UNKNOWN_FACE_ID can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

face_id: A non-negative integer face-id.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID on failure.

remove_all_embeddings¶

FaceEnrollmentAutomatic.remove_all_embeddings() -> ErrorCode

Remove all face embeddings from the face library.

Arguments:

None.

Returns:

Returns SUCCESS on success.

max_face_library_size¶

FaceEnrollmentAutomatic.max_face_library_size(cls) -> int

Returns the maximum size of the face library.

When the library is full and new faces are detected, they will overwrite older faces, except for faces that are tagged as important using tag_face_id. When all faces are tagged as important, new faces will not be added to the library beyond this maximum size.

Arguments:

None.

Returns:

The maximum size of the face library.

get_embedding_data¶

FaceEnrollmentAutomatic.get_embedding_data(
    face_id: int, include_face_snapshot: bool
) -> tuple[ErrorCode, bytes]

Get the face embedding data for a face-ID, for persistent storage.

This can for example be used to store the face library when a device shuts down and restore it later when it reboots.

This embedding data can change during any process_frame call.

Note that FACE_ID_UNIDENTIFIED can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

face_id: A non-negative integer face-id.
include_face_snapshot: Whether to include the face snapshot in the embedding data. This snapshot is stored as raw RGB data.

Returns:

An error code of type ErrorCode and the resulting embedding data.

restore_embedding_data¶

FaceEnrollmentAutomatic.restore_embedding_data(face_id: int, embedding_data: bytes) -> ErrorCode

Restore the face embedding data for a face-ID, from persistent storage.

When the specified face_id does not yet exist then it will be created. When the specified face_id already exists, the internal embedding data will be overwritten with new embedding data. If the new embedding data is corrupt, this will result in removal of the existing face embedding data.

If the embedding data does not include a face snapshot, then the face snapshot data for this face id will contain uninitialized data. If the embedding data includes a face snapshot, then configure_face_snapshots must have been called first, with enabled set to true and the same height and width as the stored data. If the height and width do not match, then STATE_SETTINGS_MISMATCH is returned.

Arguments:

face_id: A non-negative integer face-id.
embedding_data: The embedding data to restore.

Returns:

Returns SUCCESS on success, or FACE_LIBRARY_FULL when the face library is full, or STATE_CORRUPT when the embedding data was corrupted. If the height and width of an included face snapshot do not match, then STATE_SETTINGS_MISMATCH is returned.

merge_embeddings¶

FaceEnrollmentAutomatic.merge_embeddings(face_id_dst: int, face_id_src: int) -> ErrorCode

Merge two entries of the face library.

If a single person is enrolled twice in the face library, this function can be used to merge their enrollments back to a single one. After calling this function face_id_src is no longer valid and face_id_dst will be updated with the additional information from face_id_src.

Arguments:

face_id_dst: A non-negative integer face-id.
face_id_src: A non-negative integer face-id, must be different from face_id_dst.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID when either of the face ids could not be found, or INTERNAL_ERROR in case of an unexpected error.

FaceEnrollmentManual¶

embedding_size¶

FaceEnrollmentManual.embedding_size(cls) -> int

Returns the size of the face embeddings obtained from finish_enrollment and required by add_embedding.

Arguments:

None.

Returns:

The size of the face embeddings.

add_embedding¶

FaceEnrollmentManual.add_embedding(embedding: bytes, face_id: int) -> ErrorCode

Add a face embedding to the face library for any process_frame calls that follow.

A second embedding for the same face id will overwrite the previous one. When calling FaceIdentification.get_face_id on a detected person box, it will use the face_id provided here as return value.

On some platforms, a version that uses std::vector is available.

Arguments:

embedding: A face embedding as obtained from finish_enrollment.
face_id: An integer face-id, is not allowed to be equal to FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY.

Returns:

Returns SUCCESS on success, INVALID_FACE_ID or INVALID_EMBEDDING on failure.

remove_embedding¶

FaceEnrollmentManual.remove_embedding(face_id: int) -> ErrorCode

Remove a face embedding from the face library for any process_frame calls that follow.

Arguments:

face_id: An integer face-ID, previously used in add_embedding.

Returns:

Returns the error code SUCCESS on success, INVALID_FACE_ID on failure.

start_enrollment¶

FaceEnrollmentManual.start_enrollment(
    region_of_interest: tuple[float, float, float, float] = (0, 0, 1, 1),
) -> ErrorCode

This starts the enrollment procedure for a new face.

During subsequent calls to process_frame or single_image, face embeddings will be computed. During enrollment there should be a single face in the image, or in the optional region specified by region_of_interest, ideally clearly and completely visible. The enrollment procedure can be finalized by calling finish_enrollment.

Arguments:

region_of_interest: This can be set to a region of interest in the image as y_min, x_min, y_max, x_max in relative coordinates. Only faces that have overlap with this region will be used for enrollment. The default of 0, 0, 1, 1 includes the entire image.

Returns:

Returns the error code "ENROLLMENT_IN_PROGRESS" if the enrollment procedure was started successfully, otherwise returns "ALREADY_ENROLLING".

finish_enrollment¶

FaceEnrollmentManual.finish_enrollment() -> tuple[ErrorCode, bytes]

Finalize the face enrollment procedure started by start_enrollment and obtain the face embedding.

The application is responsible for storing this embedding on persistent storage and passing it to add_embedding every time an instance of this class is created. The face enrollment procedure does not automatically call add_embedding.

Arguments:

None.

Returns:

A tuple with an error code of type ErrorCode and the resulting face embedding bytes

get_face_ids¶

FaceEnrollmentManual.get_face_ids() -> tuple[ErrorCode, list[int]]

Return the face ids of all enrolled faces.

This list of face ids is not related to the faces currently in view. The face ids returned by this function are all faces that have ever been enrolled into the face library. To retrieve information about the faces currently in view, use FaceIdentification.get_face_id.

Arguments:

None.

Returns:

Returns a tuple with an error code (SUCCESS on success) and a list of enrolled face ids.

remove_all_embeddings¶

FaceEnrollmentManual.remove_all_embeddings() -> ErrorCode

Remove all face embeddings from the face library.

Arguments:

None.

Returns:

Returns SUCCESS on success.

max_face_library_size¶

FaceEnrollmentManual.max_face_library_size(cls) -> int

Returns the maximum size of the face library.

When the library is full, add_embedding will return FACE_LIBRARY_FULL and won't be able to add new embeddings, until FaceEnrollmentManual.remove_embedding is called.

Arguments:

None.

Returns:

The maximum size of the face library.

Constants¶

plumerai_video_intelligence.FACE_ID_UNIDENTIFIED
plumerai_video_intelligence.FACE_ID_NOT_IN_LIBRARY

If the face-identification model is not confident about a face, it will output FACE_ID_UNIDENTIFIED. If the model is confident that a face is not in the face library, it will output FACE_ID_NOT_IN_LIBRARY.