Plumerai Familiar Face Identification API¶

This document describes the API for the Familiar Face Identification functionality.

Note that the software is built with either the Automatic Enrollment option or the Manual Enrollment option enabled. This page provides documentation for both options, but at runtime only one of the two will be available.

Face Identification¶

get_face_id¶

int plumerai_face_identification_get_face_id(
    CPlumeraiVideoIntelligence pvi, const BoxPrediction *person_box);

Retrieve the face ID that belongs to a person box.

Retrieves the face ID given a person box from the most recent video frame. This function has three possible return values:

FACE_ID_UNIDENTIFIED when the provided box is not of CLASS_PERSON or when a face is not yet identified, e.g. the face is not fully visible.
FACE_ID_NOT_IN_LIBRARY when the face is visible but not found in the library, e.g. when a stranger is in view.
Any integer face-ID provided previously to plumerai_face_enrollment_manual_add_embedding when a familiar face is identified. This function should not be called directly after restoring from a previous state through plumerai_video_intelligence_restore_state. This function should not be called with results of plumerai_video_intelligence_single_image calls.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
person_box: A box from the most recent video frame with class_id equal to DetectionClass::CLASS_PERSON.

Returns:

An integer face-ID as provided when calling plumerai_face_enrollment_manual_add_embedding when a face is identified, FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY when it is not identified successfully, or FACE_ID_UNIDENTIFIED when the provided box is not a valid person box.

get_person_box_from_face_box¶

PlumeraiErrorCode
plumerai_face_identification_get_person_box_from_face_box(
    CPlumeraiVideoIntelligence pvi, const BoxPrediction *face_box,
    BoxPrediction *result_person_box);

Retrieve a person box corresponding to a face box.

This function should not be called with results of plumerai_video_intelligence_single_image calls.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_box: The target face to find a match for. This has to be a box from the most recent video frame.
result_person_box: The resulting box, only valid if the return value is true (see below). The caller needs to allocate memory for this box prior to calling this function.

Returns:

Returns SUCCESS in case a match was found. Returns NO_BOX_MATCH if it was not possible to uniquely match a face box to a person box. Returns INVALID_BOX in case of invalid arguments.

get_face_box_from_person_box¶

PlumeraiErrorCode
plumerai_face_identification_get_face_box_from_person_box(
    CPlumeraiVideoIntelligence pvi, const BoxPrediction *person_box,
    BoxPrediction *result_face_box);

Retrieve a face box corresponding to a person box.

This function should not be called with results of plumerai_video_intelligence_single_image calls.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
person_box: The target person to find a match for. This has to be a box from the most recent video frame.
result_face_box: The resulting box, only valid if the return value is true (see below). The caller needs to allocate memory for this box prior to calling this function.

Returns:

Returns SUCCESS in case a match was found. Returns NO_BOX_MATCH if it was not possible to uniquely match a face box to a person box. Returns INVALID_BOX in case of invalid arguments.

embedding_version¶

uint32_t
plumerai_face_identification_embedding_version(CPlumeraiVideoIntelligence pvi);

Returns a version number for the face embedding data used by the face identification logic.

This version can be used to check if the face embedding data generated with one version of the Plumerai software are compatible with those generated by another version of the Plumerai software.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

The version number of the face embedding data.

Face Enrollment Automatic¶

configure_face_snapshots¶

PlumeraiErrorCode
plumerai_face_enrollment_automatic_configure_face_snapshots(
    CPlumeraiVideoIntelligence pvi, const CFaceSnapshotSettings *settings);

Configure the face snapshots for automatic enrollment.

By default, face snapshots are disabled.

Calling this function will invalidate any stored face snapshots of faces that are already enrolled. Trying to retrieve those face snapshots will return uninitialized data.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
settings: See FaceSnapshotSettings for more details.

Returns:

Returns SUCCESS on success, or INVALID_ARGUMENT when the provided settings are not valid.

get_face_snapshot¶

PlumeraiErrorCode
plumerai_face_enrollment_automatic_get_face_snapshot(
    CPlumeraiVideoIntelligence pvi, const int face_id,
    const uint8_t **face_snapshot_data);

Return a snapshot of an automatically enrolled face.

Note that UNKNOWN_FACE_ID can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_id: A non-negative integer face-id.
face_snapshot_data: The pointer to the stored snapshot data is returned via this output argument.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID when the specified face id could not be found, or FACE_SNAPSHOTS_DISABLED when face snapshots were not configured using plumerai_face_enrollment_automatic_configure_face_snapshots.

get_face_ids¶

PlumeraiErrorCode plumerai_face_enrollment_automatic_get_face_ids(
    CPlumeraiVideoIntelligence pvi, int *face_ids, size_t *num_face_ids);

Return the face ids of all automatically enrolled faces.

This list of face ids is not related to the faces currently in view. The face ids returned by this function are all faces that have ever been enrolled into the face library. To retrieve information about the faces currently in view, use plumerai_face_identification_get_face_id.

The caller must provide a buffer of size at least max_library_size integers.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_ids: A pointer to a user-allocated list that receives the enrolled face ids. It must hold at least max_library_size integers.
num_face_ids: An output parameter that receives the number of returned values.

Returns:

Returns SUCCESS on success.

tag_face_id¶

PlumeraiErrorCode plumerai_face_enrollment_automatic_tag_face_id(
    CPlumeraiVideoIntelligence pvi, const int face_id, bool important);

Tag a face id as 'important' to prevent it from being overwritten.

When the face library is full and a new face is encountered, the algorithm will overwrite the oldest face in the library that is not tagged as important. A typical use-case of this is to tag a face as important when the user has named them in a front-end application.

When all entries in the face library are tagged as important and a new face is encountered, then no new faces will be enrolled.

By default, an automatically enrolled face is not tagged as important. However, library entries created by plumerai_face_enrollment_automatic_restore_embedding_data are automatically tagged as important.

Note that UNKNOWN_FACE_ID can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_id: A non-negative integer face-id.
important: Whether to tag the face as important or to remove a previously applied tag.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID when the specified face id could not be found.

remove_embedding¶

PlumeraiErrorCode plumerai_face_enrollment_automatic_remove_embedding(
    CPlumeraiVideoIntelligence pvi, const int face_id);

Remove a face embedding from the face library for any plumerai_video_intelligence_process_frame calls that follow.

Note that UNKNOWN_FACE_ID can be returned when a face id that was previously valid is overwritten with a new entry.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_id: A non-negative integer face-id.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID on failure.

remove_all_embeddings¶

PlumeraiErrorCode
plumerai_face_enrollment_automatic_remove_all_embeddings(
    CPlumeraiVideoIntelligence pvi);

Remove all face embeddings from the face library.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

Returns SUCCESS on success.

merge_embeddings¶

PlumeraiErrorCode plumerai_face_enrollment_automatic_merge_embeddings(
    CPlumeraiVideoIntelligence pvi, const int face_id_dst,
    const int face_id_src);

Merge two entries of the face library.

If a single person is enrolled twice in the face library, this function can be used to merge their enrollments back to a single one. After calling this function face_id_src is no longer valid and face_id_dst will be updated with the additional information from face_id_src.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_id_dst: A non-negative integer face-id.
face_id_src: A non-negative integer face-id, must be different from face_id_dst.

Returns:

Returns SUCCESS on success, UNKNOWN_FACE_ID when either of the face ids could not be found, or INTERNAL_ERROR in case of an unexpected error.

max_face_library_size¶

size_t plumerai_face_enrollment_automatic_max_face_library_size(
    CPlumeraiVideoIntelligence pvi);

Returns the maximum size of the face library.

When the library is full and new faces are detected, they will overwrite older faces, except for faces that are tagged as important using plumerai_face_enrollment_automatic_tag_face_id. When all faces are tagged as important, new faces will not be added to the library beyond this maximum size.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

The maximum size of the face library.

Face Enrollment Manual¶

embedding_size¶

size_t
plumerai_face_enrollment_manual_embedding_size(CPlumeraiVideoIntelligence pvi);

Returns the size of the face embeddings obtained from plumerai_face_enrollment_manual_finish_enrollment and required by plumerai_face_enrollment_manual_add_embedding.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

The size of the face embeddings.

max_face_library_size¶

size_t plumerai_face_enrollment_manual_max_face_library_size(
    CPlumeraiVideoIntelligence pvi);

Returns the maximum size of the face library.

When the library is full, plumerai_face_enrollment_manual_add_embedding will return FACE_LIBRARY_FULL and won't be able to add new embeddings, until plumerai_face_enrollment_manual_remove_embedding is called.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

The maximum size of the face library.

start_enrollment¶

PlumeraiErrorCode plumerai_face_enrollment_manual_start_enrollment(
    CPlumeraiVideoIntelligence pvi);

This starts the enrollment procedure for a new face.

During subsequent calls to plumerai_video_intelligence_process_frame or plumerai_video_intelligence_single_image, face embeddings will be computed. During enrollment there should be a single face in the image, or in the optional Region specified by region_of_interest (not available in the C API), ideally clearly and completely visible. The enrollment procedure can be finalized by calling plumerai_face_enrollment_manual_finish_enrollment.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

Returns ENROLLMENT_IN_PROGRESS if the enrollment procedure was started successfully, otherwise returns ALREADY_ENROLLING.

finish_enrollment¶

PlumeraiErrorCode plumerai_face_enrollment_manual_finish_enrollment(
    CPlumeraiVideoIntelligence pvi, int8_t *face_embedding);

Finalize the face enrollment procedure started by plumerai_face_enrollment_manual_start_enrollment and obtain the face embedding.

The caller must provide a buffer of size plumerai_face_enrollment_manual_embedding_size.

The application is responsible for storing this embedding on persistent storage and passing it to plumerai_face_enrollment_manual_add_embedding every time an instance of this class is created. The face enrollment procedure does not automatically call plumerai_face_enrollment_manual_add_embedding.

For an example, see the example under plumerai_face_enrollment_manual_start_enrollment above.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_embedding: The resulting face embedding if the result is SUCCESS or LOW_QUALITY_ENROLLMENT.

Returns:

Returns SUCCESS when enrollment was successful. Otherwise, returns one of NOT_ENROLLING, ENROLLMENT_FAILED, LOW_QUALITY_ENROLLMENT. See the enum for more information.

add_embedding¶

PlumeraiErrorCode plumerai_face_enrollment_manual_add_embedding(
    CPlumeraiVideoIntelligence pvi, const int8_t *face_embedding,
    size_t face_embedding_size, int face_id);

Add a face embedding to the face library for any plumerai_video_intelligence_process_frame calls that follow.

A second embedding for the same face id will overwrite the previous one. When calling plumerai_face_identification_get_face_id on a detected person box, it will use the face_id provided here as return value.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_embedding: A face embedding as obtained from plumerai_face_enrollment_manual_finish_enrollment.
face_embedding_size: Only used in the case when face_embedding is a raw pointer: the size of the face embedding data.
face_id: An integer face-id, is not allowed to be equal to FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY.

Returns:

Returns SUCCESS on success, INVALID_FACE_ID, FACE_LIBRARY_FULL, or INVALID_EMBEDDING on failure.

get_face_ids¶

PlumeraiErrorCode plumerai_face_enrollment_manual_get_face_ids(
    CPlumeraiVideoIntelligence pvi, int *face_ids, size_t *num_face_ids);

Return the face ids of all enrolled faces.

This list of face ids is not related to the faces currently in view. The face ids returned by this function are all faces that have ever been enrolled into the face library. To retrieve information about the faces currently in view, use plumerai_face_identification_get_face_id.

The caller must provide a buffer of size at least max_library_size integers.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_ids: A pointer to a user-allocated list that receives the enrolled face ids. It must hold at least max_library_size integers.
num_face_ids: An output parameter that receives the number of returned values.

Returns:

Returns SUCCESS on success.

remove_embedding¶

PlumeraiErrorCode plumerai_face_enrollment_manual_remove_embedding(
    CPlumeraiVideoIntelligence pvi, int face_id);

Remove a face embedding from the face library for any plumerai_video_intelligence_process_frame calls that follow.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.
face_id: An integer face-id, previously used in plumerai_face_enrollment_manual_add_embedding.

Returns:

Returns SUCCESS on success, INVALID_FACE_ID on failure.

remove_all_embeddings¶

PlumeraiErrorCode
plumerai_face_enrollment_manual_remove_all_embeddings(
    CPlumeraiVideoIntelligence pvi);

Remove all face embeddings from the face library.

Arguments:

pvi: An initialized CPlumeraiVideoIntelligence object.

Returns:

Returns SUCCESS on success.

Constants¶

static const int FACE_ID_UNIDENTIFIED = -1;
static const int FACE_ID_NOT_IN_LIBRARY = -2;

If the face-identification model is not confident about a face, it will output FACE_ID_UNIDENTIFIED. If the model is confident that a face is not in the face library, it will output FACE_ID_NOT_IN_LIBRARY.