Skip to content

Plumerai Familiar Face Identification C++ API

This document describes the C++ API for the Plumerai Familiar Face Identification software for videos on Arm Cortex-A and x86. For an example, see here.

The C++ API consists of a class plumerai::FaceIdentification which is a subclass of plumerai::PeopleDetection. Processing video input is described in the documentation of the base class. The FaceIdentification class adds functionality for controlling the face library, which is documented here.

The API is re-entrant, i.e. you can instantiate several FaceIdentification objects in different threads and use them independently. However, using the same instance from different threads at the same time is not supported.

FaceIdentification

FaceIdentification::FaceIdentification(int height, int width)

Initializes a new FaceIdentification object.

This needs to be called only once at the start of the application.

Arguments

  • height int: The height of the input image in pixels.
  • width int: The width of the input image in pixels.

Returns

Nothing.

face_embedding_size

static std::size_t FaceIdentification::face_embedding_size();

Returns the size of the face embedding vectors returned by finish_face_enrollment and required by add_face_embedding.

Returns

std::size_t: The size of the face embedding vectors.

face_embedding_version_checksum

static std::uint32_t FaceIdentification::face_embedding_version_checksum();

Returns a version checksum for the face embedding vectors returned by finish_face_enrollment and required by add_face_embedding.

This checksum is used to check if the face embedding vectors generated with one library build are compatible with another library.

Returns

std::uint32_t: The version checksum of the face embedding vectors.

start_face_enrollment

ErrorCodeFamiliarFaceID FaceIdentification::start_face_enrollment(
    const std::vector<std::int8_t> &previous_enrollment =
        std::vector<std::int8_t>(),
    const Region &region_of_interest = Region());

This starts the enrollment procedure for a new face.

During subsequent calls to process_frame or single_image, face embeddings will be computed as well as metrics that quantify the quality of the embeddings. After every frame get_cumulative_enrollment_score can be called to check if enough frames have been processed for a high quality embedding. During enrollment there should be a single face in the image, or in the optional Region specified by region_of_interest, ideally clearly and completely visible. The enrollment procedure can be finalized by calling finish_face_enrollment.

Arguments

  • previous_enrollment std::vector<std::int8_t>: This can be set to the result of a previous enrollment procedure to supplement it with new data. It can be left empty if no previous enrollment data is available. The default is empty.
  • region_of_interest Region: This can be set to a region of interest in the image. Only faces that have overlap with this region will be used for enrollment. The default region is the entire image.

Returns

ErrorCodeFamiliarFaceID: Returns ENROLLMENT_IN_PROGRESS if the enrollment procedure was started successfully, otherwise returns ALREADY_ENROLLING or INVALID_PREVIOUS_ENROLLMENT.

get_cumulative_enrollment_score

float FaceIdentification::get_cumulative_enrollment_score();

Return the cumulative enrollment score.

It is important to realize that the score is the total score until now, not only of the last frame. Properties of the score:

  • The value lies between 0.0 and 1.0.
  • The higher, the better.
  • The score is some way to measure diversity of enrolled images. And thus, the first successfully enrolled frame will always have a score 0.0.
  • If the next frame also results in a score of 0.0, then it was either identical (from the neural network perspective) or there was an error.
  • The score can also go down as new images are added.
  • It is possible to end enrollment with a score of 0, however it is not recommended. Still, familiar face identification might still work.

Returns

float: The cumulative enrollment score.

finish_face_enrollment

std::vector<std::int8_t> FaceIdentification::finish_face_enrollment();

Finalize the face enrollment procedure started by start_face_enrollment and obtain the face embedding.

The application is responsible for storing this embedding on persistent storage and passing it to add_face_embedding every time an instance of this class is created. The face enrollment procedure does not automatically call add_face_embedding.

Returns

std::vector<std::int8_t>: The resulting face embedding vector, or an empty vector in case of failure.

add_face_embedding

bool FaceIdentification::add_face_embedding(
    const std::vector<std::int8_t> &face_embedding,
    const int face_id);

Add a face embedding to the face library for any process_frame calls that follow.

A second embedding for the same face id will overwrite the previous one. When process_frame detects a person that matches a face, it will use the face_id provided here in the returned BoxPrediction struct.

Arguments

Returns

bool: Returns true on success.

remove_face_embedding

bool FaceIdentification::remove_face_embedding(const int face_id);

Remove a face embedding to the face library for any process_frame calls that follow.

Arguments

Returns

bool: Returns true on success.

get_face_id

int FaceIdentification::get_face_id(const BoxPrediction &person_box) const;

Retrieve the face ID that belongs to a person box.

Retrieves the face ID given a person box returned by process_frame. This function has three possible return values:

  • FACE_ID_UNIDENTIFIED when the provided box is not of CLASS_PERSON or when a face is not yet identified, e.g. the face is not fully visible.
  • FACE_ID_NOT_IN_LIBRARY when the face is visible but not found in the library, e.g. when a stranger is in view.
  • Any integer face-ID provided previously to add_face_embedding when a familiar face is identified.

Arguments

  • person_box BoxPrediction: A box returned by process_frame with class_id equal to DetectionClass::CLASS_PERSON.

Returns

int: An integer face-ID as provided when calling add_face_embedding when a face is identified, FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY when it is not identified successfully, or FACE_ID_UNIDENTIFIED when the provided box is not a valid person box.

get_person_box_from_face_box

See get_face_box_from_person_box below.

get_face_box_from_person_box

bool FaceIdentification::get_person_box_from_face_box(
    const BoxPrediction &face_box,
    BoxPrediction *result_person_box) const;
bool FaceIdentification::get_face_box_from_person_box(
    const BoxPrediction &person_box,
    BoxPrediction *result_face_box) const;

Retrieve a person box corresponding to a face box or vice-versa.

Arguments

  • {face,person}_box const BoxPrediction: The target face or person to find a match for. This has to be a box from the most recent call to process_frame.
  • result_{person,face}_box BoxPrediction*: The resulting box, only valid if the return value is true (see below). The caller needs to allocate memory for this box prior to calling this function.

Returns

bool: False in case of invalid arguments or when it was not possible to uniquely match a face box to a person box. Returns true in case a match was found.

Region

struct Region {
  float y_min = 0.0f;  // top coordinate between 0 and 1 in height dimension
  float x_min = 0.0f;  // left coordinate between 0 and 1 in width dimension
  float y_max = 1.0f;  // bottom coordinate between 0 and 1 in height dimension
  float x_max = 1.0f;  // right coordinate between 0 and 1 in width dimension
};

A structure representing a region of the input image. Coordinates are between 0 and 1, the origin is at the top-left.

ErrorCodeFamiliarFaceID

typedef enum {
  // Everything OK, enrollment is in progress.
  ENROLLMENT_IN_PROGRESS = -1000,

  // This library is not built with support for face identification.
  FACE_IDENTIFICATION_DISABLED = -1001,

  // Called `plumerai_start_face_enrollment` but enrollment is already started.
  ALREADY_ENROLLING = -1002,

  // Called `plumerai_start_face_enrollment` but the last enrollment is invalid.
  INVALID_PREVIOUS_ENROLLMENT = -1003,

  // Last frame was ignored because multiple faces were detected within the
  // specified region of interest. Try again with one face visible.
  MULTIPLE_FACES_IN_VIEW = -1004,

  // Last frame was ignored because no faces were detected within the specified
  // region of interest. Try again with one face visible.
  NO_FACE_IN_VIEW = -1005,

  // Last frame was ignored because the face detection was too close to the
  // edge. Too close to the edge means that the border of the face bounding-box
  // is within 2% of the edge of the image. In theory, it is OK if a face is
  // close to edge, however, it can also mean that a face is only partially
  // visible. Thus, this check is added to prevent partial faces from being
  // enrolled.
  FACE_CLOSE_TO_EDGE = -1006,
} ErrorCodeFamiliarFaceID;
Possible error codes for the enrollment procedure. Furthermore, the regular ErrorCode values might be returned.

Constants

static const int FACE_ID_UNIDENTIFIED = -1;
static const int FACE_ID_NOT_IN_LIBRARY = -2;

If the face-identification model is not confident about a face, it will output FACE_ID_UNIDENTIFIED. If the model is confident that a face is not in the face library, it will output FACE_ID_NOT_IN_LIBRARY.