Crate plumerai_video_intelligence

Expand description

§Plumerai Video Intelligence

The entry point for this crate is through the VideoIntelligence struct. Specific functionality can be accessed through these structs, accessed by methods of the same name:

§Minimal Examples

§Video Processing

This example shows how to initialize the video intelligence class and process a video frame.

// Settings, to be changed as needed
let input_resolution = plumerai::Size { width: 1600, height: 1200 };

// Initialize the video intelligence algorithm
let mut pvi = plumerai::VideoIntelligence::new(input_resolution);

// Set whether the video stream is night mode (IR) or not.
pvi.set_night_mode(false);

// Loop over frames in a video stream (example: 10 frames)
for _t in 0..10 {
    // Some example input here, normally this is where camera data is acquired
    let image_data = vec![0; 1600 * 1200 * 3]; // RGB888
    let image = plumerai::ImagePointer::PackedRGB888(&image_data);

    // The time between two video frames in seconds
    // In this example we assume a constant frame rate of 30 fps, but variable
    // rates are supported.
    // None means that the system clock will be used.
    let delta_t = Some(1. / 30.);

    // Process the frame
    pvi.process_frame(&image, delta_t)?;

    let predictions = pvi.object_detection().get_detections();

    // Display the results to stdout
    for p in predictions {
        println!("prediction: {:}", p);
    }
}

§Automatic Face Enrollment:

This example extends the code above and shows how to use the automatic face enrollment functionality.

// Settings, to be changed as needed
let input_resolution = plumerai::Size { width: 1600, height: 1200 };

// Initialize the video intelligence algorithm
let mut pvi = plumerai::VideoIntelligence::new(input_resolution);
let mut face_enrollment_automatic = pvi.face_enrollment_automatic()?;
let mut face_identification = pvi.face_identification()?;

// Set whether the video stream is night mode (IR) or not.
pvi.set_night_mode(false);

// Loop over frames in a video stream (example: 10 frames)
for _t in 0..10 {
    // Some example input here, normally this is where camera data is acquired
    let image_data = vec![0; 1600 * 1200 * 3]; // RGB888
    let image = plumerai::ImagePointer::PackedRGB888(&image_data);

    // The time between two video frames in seconds
    // In this example we assume a constant frame rate of 30 fps, but variable
    // rates are supported.
    // None means that the system clock will be used.
    let delta_t = Some(1. / 30.);

    // Process the frame
    pvi.process_frame(&image, delta_t)?;

    // Report the number of faces in the library so far. At first the library
    // will be empty, but as soon as a face is well visible for a while, it
    // will be added to the library with a new unique face ID. The library
    // will grow over time, unless `remove_face_embedding` is called.
    let face_ids = face_enrollment_automatic.get_face_ids();
    println!("Total of {:} people in the familiar face-ID library", face_ids.len());

    let predictions = pvi.object_detection().get_detections();

    // Display the results to stdout
    for p in predictions {
        // After a few frames of showing a clear face in view, the face ID should
        // become non-negative as the face is automatically enrolled.
        if p.class_id() == plumerai::DetectionClass::Face {
            let face_id = face_identification.get_face_id(&p)?;
            println!("Person box with face id {:}: {:}", face_id, p);
        } else {
            println!("prediction: {:}", p);
        }
    }
}

§Manual Face Enrollment:

This example shows how to use the manual face enrollment functionality. It consists of two main loops:

An example enrollment loop, which runs for a fixed number of frames and computes a face embedding vector to enroll one person in the face library.
An example video processing loop, similar to the first example.

// Settings, to be changed as needed
let input_resolution = plumerai::Size { width: 1600, height: 1200 };

// Initialize the video intelligence algorithm
let mut pvi = plumerai::VideoIntelligence::new(input_resolution);
let mut face_enrollment_manual = pvi.face_enrollment_manual()?;
let mut face_identification = pvi.face_identification()?;

// Set whether the video stream is night mode (IR) or not.
pvi.set_night_mode(false);

// Set an area of the frame to be used for face enrollment.
let enrollment_region = plumerai::RelativeCoords::all();

// Start the manual enrollment
face_enrollment_manual.start_enrollment(&enrollment_region)?;

// Enroll for 10 frames (just an example, more is better)
for _t in 0..10 {
    // Some example input here, normally this is where camera data is acquired
    let image_data = vec![0; 1600 * 1200 * 3]; // RGB888
    let image = plumerai::ImagePointer::PackedRGB888(&image_data);

    // Process the frame. If the enrollment frames come from a video source,
    // then use `process_frame` instead.
    // For single images, the size can be different to the input resolution.
    let image_size = plumerai::ImageSize::VideoSize;
    pvi.single_image(&image, image_size)?;
}

// Finish enrollment
let embedding = face_enrollment_manual.finish_enrollment()?;

// Add the embedding to the library with face ID `1`
let face_id = plumerai::FaceID::Identified(1);
face_enrollment_manual.add_embedding(embedding, &face_id)?;

// Loop over frames in a video stream (example: 10 frames)
for _t in 0..10 {
    // Some example input here, normally this is where camera data is acquired
    let image_data = vec![0; 1600 * 1200 * 3]; // RGB888
    let image = plumerai::ImagePointer::PackedRGB888(&image_data);

    // The time between two video frames in seconds
    // In this example we assume a constant frame rate of 30 fps, but variable
    // rates are supported.
    // None means that the system clock will be used.
    let delta_t = Some(1. / 30.);

    // Process the frame
    pvi.process_frame(&image, delta_t)?;

    let predictions = pvi.object_detection().get_detections();

    // Display the results to stdout
    for p in predictions {
        // After a few frames of showing a clear face in view, the face ID should
        // become non-negative as the face is automatically enrolled.
        if p.class_id() == plumerai::DetectionClass::Person {
            let face_id = face_identification.get_face_id(&p)?;
            println!("Person box with face id {:}: {:}", face_id, p);
        } else {
            println!("prediction: {:}", p);
        }
    }
}

Structs§

BoxPrediction: A struct representing a single resulting bounding box.
DebugBytes: Debug data to be sent to Plumerai support for analysis.
DetectionZones: Detection zone functionality.
EnrollmentData: Enrollment data for an automatic face ID.
FaceEmbedding: Face embedding data.
FaceEnrollmentAutomatic: Automatic face enrollment functionality.
FaceEnrollmentManual: Manual face enrollment functionality.
FaceIdentification: General face identification functionality.
FaceSnapshot: A snapshot of a face.
Grid: The motion grid.
MotionDetection: Motion detection functionality.
ObjectDetection: Object Detection functionality.
RelativeCoords: Coordinates relative to the full image.
Size: A struct representing the size of an image.
State: The state of the VideoIntelligence algorithm.
TrackID: An ID assigned by the tracker.
VideoIntelligence: Contains all Plumerai Video Intelligence functionality.
ZoneID: A unique identifier for a detection zone.

Enums§

DebugImageOptions: Options for debug_mode images.
DetectionClass: Enum representing the classes of objects that can be detected.
Error: Error type for the Plumerai video intelligence library. See here for an explanation of each type of error.
FaceID: An identifier for a face.
FaceIDTag: A tag indicating importance of a face ID.
FaceSnapshotSettings: Settings for the face snapshot feature.
ImagePointer: A reference to an image, supporting various formats.
ImageSize
SnapshotOptions: Options for including or excluding face snapshots in the enrollment data.

Crate plumerai_video_intelligenceCopy item path