Plumerai Familiar Face Identification C++ API¶
This document describes the C++ API for the Plumerai Familiar Face Identification software for videos on Arm Cortex-A and x86. For an example, see here.
The C++ API consists of a class plumerai::FaceIdentification
which is a subclass of plumerai::PeopleDetection
. Processing video input is described in the documentation of the base class. The FaceIdentification
class adds functionality for controlling the face library, which is documented here.
The API is re-entrant, i.e. you can instantiate several FaceIdentification
objects in different threads and use them independently. However, using the same instance from different threads at the same time is not supported.
FaceIdentification¶
Initializes a new FaceIdentification
object.
This needs to be called only once at the start of the application.
Arguments
- height
int
: The height of the input image in pixels. - width
int
: The width of the input image in pixels.
Returns
Nothing.
face_embedding_size¶
Returns the size of the face embedding vectors returned by finish_face_enrollment
and required by add_face_embedding
.
Returns
std::size_t
: The size of the face embedding vectors.
face_embedding_version_checksum¶
Returns a version checksum for the face embedding vectors returned by finish_face_enrollment
and required by add_face_embedding
.
This checksum is used to check if the face embedding vectors generated with one library build are compatible with another library.
Returns
std::uint32_t
: The version checksum of the face embedding vectors.
start_face_enrollment¶
ErrorCodeFamiliarFaceID FaceIdentification::start_face_enrollment(
const std::vector<std::int8_t> &previous_enrollment =
std::vector<std::int8_t>(),
const Region ®ion_of_interest = Region());
This starts the enrollment procedure for a new face.
During subsequent calls to process_frame
or single_image
, face embeddings will be computed as well as metrics that quantify the quality of the embeddings. After every frame get_cumulative_enrollment_score
can be called to check if enough frames have been processed for a high quality embedding. During enrollment there should be a single face in the image, or in the optional Region
specified by region_of_interest
, ideally clearly and completely visible. The enrollment procedure can be finalized by calling finish_face_enrollment
.
Arguments
- previous_enrollment
std::vector<std::int8_t>
: This can be set to the result of a previous enrollment procedure to supplement it with new data. It can be left empty if no previous enrollment data is available. The default is empty. - region_of_interest
Region
: This can be set to a region of interest in the image. Only faces that have overlap with this region will be used for enrollment. The default region is the entire image.
Returns
ErrorCodeFamiliarFaceID
: Returns ENROLLMENT_IN_PROGRESS
if the enrollment procedure was started successfully, otherwise returns ALREADY_ENROLLING
or INVALID_PREVIOUS_ENROLLMENT
.
get_cumulative_enrollment_score¶
Return the cumulative enrollment score.
It is important to realize that the score is the total score until now, not only of the last frame. Properties of the score:
- The value lies between 0.0 and 1.0.
- The higher, the better.
- The score is some way to measure diversity of enrolled images. And thus, the first successfully enrolled frame will always have a score 0.0.
- If the next frame also results in a score of 0.0, then it was either identical (from the neural network perspective) or there was an error.
- The score can also go down as new images are added.
- It is possible to end enrollment with a score of 0, however it is not recommended. Still, familiar face identification might still work.
Returns
float
: The cumulative enrollment score.
finish_face_enrollment¶
Finalize the face enrollment procedure started by start_face_enrollment
and obtain the face embedding.
The application is responsible for storing this embedding on persistent storage and passing it to add_face_embedding
every time an instance of this class is created. The face enrollment procedure does not automatically call add_face_embedding
.
Returns
std::vector<std::int8_t>
: The resulting face embedding vector, or an empty vector in case of failure.
add_face_embedding¶
bool FaceIdentification::add_face_embedding(
const std::vector<std::int8_t> &face_embedding,
const int face_id);
Add a face embedding to the face library for any process_frame
calls that follow.
A second embedding for the same face id will overwrite the previous one. When process_frame
detects a person that matches a face, it will use the face_id
provided here in the returned BoxPrediction
struct.
Arguments
- face_embedding
const std::vector<std::int8_t> &
: A face embedding as returned byfinish_face_enrollment
. - face_id
int
: An integer face-id, is not allowed to be equal toFACE_ID_UNIDENTIFIED
orFACE_ID_NOT_IN_LIBRARY
.
Returns
bool
: Returns true
on success.
remove_face_embedding¶
Remove a face embedding to the face library for any process_frame
calls that follow.
Arguments
- face_id
int
: An integer face-id, previously used inadd_face_embedding
Returns
bool
: Returns true
on success.
get_face_id¶
Retrieve the face ID that belongs to a person box.
Retrieves the face ID given a person box returned by process_frame
. This function has three possible return values:
FACE_ID_UNIDENTIFIED
when the provided box is not ofCLASS_PERSON
or when a face is not yet identified, e.g. the face is not fully visible.FACE_ID_NOT_IN_LIBRARY
when the face is visible but not found in the library, e.g. when a stranger is in view.- Any integer face-ID provided previously to
add_face_embedding
when a familiar face is identified.
Arguments
- person_box
BoxPrediction
: A box returned byprocess_frame
withclass_id
equal toDetectionClass::CLASS_PERSON
.
Returns
int
: An integer face-ID as provided when calling add_face_embedding
when a face is identified, FACE_ID_UNIDENTIFIED
or FACE_ID_NOT_IN_LIBRARY
when it is not identified successfully, or FACE_ID_UNIDENTIFIED
when the provided box is not a valid person box.
get_person_box_from_face_box¶
See get_face_box_from_person_box
below.
get_face_box_from_person_box¶
bool FaceIdentification::get_person_box_from_face_box(
const BoxPrediction &face_box,
BoxPrediction *result_person_box) const;
bool FaceIdentification::get_face_box_from_person_box(
const BoxPrediction &person_box,
BoxPrediction *result_face_box) const;
Retrieve a person box corresponding to a face box or vice-versa.
Arguments
- {face,person}_box
const BoxPrediction
: The target face or person to find a match for. This has to be a box from the most recent call toprocess_frame
. - result_{person,face}_box
BoxPrediction*
: The resulting box, only valid if the return value is true (see below). The caller needs to allocate memory for this box prior to calling this function.
Returns
bool
: False in case of invalid arguments or when it was not possible to uniquely match a face box to a person box. Returns true in case a match was found.
Region¶
struct Region {
float y_min = 0.0f; // top coordinate between 0 and 1 in height dimension
float x_min = 0.0f; // left coordinate between 0 and 1 in width dimension
float y_max = 1.0f; // bottom coordinate between 0 and 1 in height dimension
float x_max = 1.0f; // right coordinate between 0 and 1 in width dimension
};
A structure representing a region of the input image. Coordinates are between 0 and 1, the origin is at the top-left.
ErrorCodeFamiliarFaceID¶
typedef enum {
// Everything OK, enrollment is in progress.
ENROLLMENT_IN_PROGRESS = -1000,
// This library is not built with support for face identification.
FACE_IDENTIFICATION_DISABLED = -1001,
// Called `plumerai_start_face_enrollment` but enrollment is already started.
ALREADY_ENROLLING = -1002,
// Called `plumerai_start_face_enrollment` but the last enrollment is invalid.
INVALID_PREVIOUS_ENROLLMENT = -1003,
// Last frame was ignored because multiple faces were detected within the
// specified region of interest. Try again with one face visible.
MULTIPLE_FACES_IN_VIEW = -1004,
// Last frame was ignored because no faces were detected within the specified
// region of interest. Try again with one face visible.
NO_FACE_IN_VIEW = -1005,
// Last frame was ignored because the face detection was too close to the
// edge. Too close to the edge means that the border of the face bounding-box
// is within 2% of the edge of the image. In theory, it is OK if a face is
// close to edge, however, it can also mean that a face is only partially
// visible. Thus, this check is added to prevent partial faces from being
// enrolled.
FACE_CLOSE_TO_EDGE = -1006,
} ErrorCodeFamiliarFaceID;
ErrorCode
values might be returned. Constants¶
If the face-identification model is not confident about a face, it will output FACE_ID_UNIDENTIFIED
. If the model is confident that a face is not in the face library, it will output FACE_ID_NOT_IN_LIBRARY
.