Skip to content

Plumerai Familiar Face Identification C++ API

This document describes the C++ API for the Plumerai Familiar Face Identification software with automatic enrollment for videos on Arm Cortex-A and x86.

The C++ API consists of a class plumerai::FaceIdentificationAutomatic which is a subclass of plumerai::PeopleDetection. Processing video input is described in the documentation of the base class. The FaceIdentificationAutomatic class adds functionality for controlling the face library, which is documented here. The API is similar to the standard Plumerai People Detection API, so please refer to the People Detection C++ API example for example code to get started.

The API is re-entrant, i.e. you can instantiate several FaceIdentificationAutomatic objects in different threads and use them independently. However, using the same instance from different threads at the same time is not supported.

API

FaceIdentificationAutomatic

FaceIdentificationAutomatic

FaceIdentificationAutomatic(int height, int width);

Initializes a new face identification object.

This needs to be called only once at the start of the application.

Arguments:

  • height: The height of the input image in pixels.
  • width: The width of the input image in pixels.

Returns:

  • Nothing.

get_face_ids

std::vector<int> get_face_ids() const;

Return the face ids of all automatically enrolled faces.

Arguments:

  • None.

Returns:

  • A vector of integers, each representing a face id.

remove_face_embedding

ErrorCodeType remove_face_embedding(const int face_id);

Remove a face embedding from the face library for any process_frame calls that follow.

Arguments:

  • face_id: A non-negative integer face-id.

Returns:

  • Returns SUCCESS on success, UNKNOWN_FACE_ID on failure.

get_face_id

int get_face_id(const BoxPrediction &person_box) const;

Retrieve the face-ID that belongs to a person box.

Retrieves the face-ID given a person box returned by process_frame. This function has three possible return values:

  • FACE_ID_UNIDENTIFIED when the provided box is not of CLASS_PERSON or when a face is not yet identified, e.g. the face is not fully visible.
  • FACE_ID_NOT_IN_LIBRARY when the face is visible but not found in the library, e.g. when a stranger is in view.
  • Any non-negative integer face-ID provided when a familiar face is identified.

This function should not be called directly after restoring from a previous state.

Arguments:

  • person_box: A box returned by process_frame with class_id equal to DetectionClass::CLASS_PERSON.

Returns:

  • A non-negative integer face-ID when a face is identified, FACE_ID_UNIDENTIFIED or FACE_ID_NOT_IN_LIBRARY when it is not identified successfully, or FACE_ID_UNIDENTIFIED when the provided box is not a valid person box.

ErrorCodeFamiliarFaceIDAutomatic

typedef enum {
  // Returned by `remove_face_embedding` when the provided face-ID is invalid.
  UNKNOWN_FACE_ID = -2001,

  // Returned by `load_face_embedding_data` if the face library is full.
  FACE_LIBRARY_FULL = -2002,
} ErrorCodeFamiliarFaceIDAutomatic;

Possible error codes for the FFID API. Furthermore, the regular ErrorCode values might be returned.

Constants

static const int FACE_ID_UNIDENTIFIED = -1;
static const int FACE_ID_NOT_IN_LIBRARY = -2;

If the face-identification model is not confident about a face, it will output FACE_ID_UNIDENTIFIED. If the model is confident that a face is not in the face library, it will output FACE_ID_NOT_IN_LIBRARY.

Example usage

Below is an example of using the C++ API shown above.

#include <cstdint>
#include <vector>

#include "plumerai/face_identification_automatic.h"

int main() {
  // Settings, to be changed as needed
  constexpr int width = 1600;   // camera image width in pixels
  constexpr int height = 1200;  // camera image height in pixels
  constexpr auto image_format = plumerai::ImageFormat::PACKED_RGB888;

  // Initialize the `FaceIdentificationAutomatic` object
  auto ffid = plumerai::FaceIdentificationAutomatic(height, width);

  // Loop over frames in a video stream (example: 20 frames)
  for (int t = 0; t < 20; ++t) {
    // Some example input here, normally this is where camera data is acquired
    auto image = std::vector<std::uint8_t>(height * width * 3);  // 3 for RGB

    // The time between two video frames in seconds
    // In this example we assume a constant frame rate of 30 fps, but variable
    // rates are supported.
    const float delta_t = 1.f / 30.f;

    // Process the frame
    std::vector<BoxPrediction> predictions(0);
    const auto error_code =
        ffid.process_frame<image_format>(image.data(), predictions, delta_t);
    if (error_code != plumerai::ErrorCode::SUCCESS) {
      printf("Error: %s\n", plumerai::error_code_string(error_code));
      return 1;
    }

    // Report the number of faces in the library so far. At first the library
    // will be empty, but as soon as a face is well visible for a while, it
    // will be added to the library with a new unique face-ID. The library
    // will grow over time, unless `remove_face_embedding` is called.
    const auto face_ids = ffid.get_face_ids();
    printf("Total of %zu people in the familiar face-ID library\n",
           face_ids.size());

    // Display the results to stdout
    for (auto &p : predictions) {
      if (p.confidence < 0.8f) { continue; }  // only high-confidence boxes
      if (p.class_id != CLASS_PERSON) { continue; } // only people
      const auto face_id = ffid.get_face_id(p);  // one from 'face_ids'
      printf(
          "Box #%d with face-ID %d with confidence %.2f "
          "(x,y)->(%.2f,%.2f) till (%.2f,%.2f)\n",
          p.id, face_id, p.confidence, p.x_min, p.y_min, p.x_max, p.y_max);
    }
    if (predictions.size() == 0) {
      printf("No bounding boxes found in this frame\n");
    }
  }
  return 0;
}