Plumerai Multi-Camera Re-Identification¶
Plumerai Multi-Camera Re-Identification (ReID) recognizes the same person across multiple cameras and over time, even after they leave a camera's view and return later. It runs at the edge or in the cloud.
ReID identifies people by their overall appearance, such as clothing and posture, rather than their face. This keeps a consistent identity even when a face is not visible, which is the common case in real footage.
Reliably matching two sightings to the same person is a building block for higher-level features such as cross-camera tracking, notification grouping, and behavior analysis.
In smart home cameras, ReID supports use cases such as:
- Reducing notification overload. Without ReID, a camera sends a fresh alert every time it re-detects the same person, so a single visitor can generate dozens of notifications. ReID recognizes these as the same person and collapses them into one event per visit. This works with just a single camera, and also extends across multiple cameras, merging the sightings from all of them into one event.
- Stitching videos. Grouped sightings can be stitched into a single timeline, letting the user follow a person's path across cameras in one continuous clip rather than scrubbing through separate recordings.
- Enriching notifications. Knowing that two sightings are the same person adds context that makes alerts more meaningful. For example, a person seen by an outdoor camera and then by an indoor camera can be recognized as the same individual, infering that this person entered the house.
In the example below, a gardener mowing the lawn triggers 34 push notifications in under an hour without ReID. With ReID, those sightings collapse into one event.
In enterprise settings, the same capability supports use cases such as:
- Cross-camera tracking. Follow individuals across a camera network, supporting tasks such as loitering detection, where the system needs to know that the person lingering near a side entrance is the same one seen in the parking lot fifteen minutes earlier.
- Footfall analysis and people counting. Because it distinguishes unique individuals from repeat sightings, ReID avoids double-counting the same person as they move between cameras or re-enter a scene.
- Forensic search. Click on any person in the footage to instantly find every other clip they appear in, across all cameras. Instead of scrubbing through hours of recordings, an operator gets a complete trail of that individual in seconds.
Technical documentation¶
See the following for technical details:
For information about tracking, robustness, camera support, privacy, and lighting, distance, mounting, and location requirements, see the object detection page.
How Re-Identification works¶
ReID is a stronger form of tracking. It pairs an improved short-term tracker with appearance-based matching, so a person keeps a consistent identity through gaps that break ordinary frame-to-frame tracking. A person can be identified even if they:
- Leave the camera's field of view and return later.
- Are temporarily occluded (for example, behind another person or an object).
- Move through areas the camera cannot see.
- Reappear after minutes or hours (depending on configuration and scene conditions).
- Move between different cameras, maintaining a consistent identity as they go (for example, from a driveway camera to a doorbell camera).
Under the hood, ReID maps image crops of each person through an embedding network into a high-dimensional space, where crops of the same person cluster together and crops of different people sit far apart. Because a single embedding from a single frame is too noisy to act on, ReID aggregates observations over time into a compact description of each person that is robust to any single bad crop. New observations are matched against these descriptions to decide whether a person has been seen before. The descriptions are short-lived working data, kept only long enough to link sightings within a limited window and discarded afterward. ReID does not build a permanent record of individuals.
For a deeper look at how the pipeline is built and why appearance-based ReID is harder than face identification, see a technical article on our blog.
Re-Identification features¶
Appearance-based identification¶
Face identification works well when there is a clean face shot, but often there isn't: people are too far away, turned sideways, walking away, or partly occluded. ReID recognizes people from full-body appearance instead, using non-facial cues such as clothing, posture, and hairstyle, so it keeps working when faces are unavailable. ReID works on its own, without Familiar Face Identification (FFID) or any facial features. It can optionally be combined with FFID to improve matching when faces are visible.
Works automatically, no setup required¶
ReID starts working on its own, with no manual setup, tagging, or scanning step. When it sees a person, it decides on the fly whether this is someone it has recently seen or a new person, and links their sightings together. These identity records are temporary. They exist only to connect sightings over a limited window and can be discarded afterward, so ReID does not require or build a lasting database of people.
Robust identity representations¶
Not every frame is worth keeping. ReID estimates the quality of each crop and selects high-quality, diverse observations to build each person's aggregated representation, rather than averaging together many near-identical or degraded frames. This keeps aggregated identities accurate even when individual crops include partial people, multiple people, or motion blur.
Multi-camera matching¶
ReID synchronizes identity information across cameras, so a person's identity stays consistent as they move through a camera network. This works for both consumer multi-camera setups and larger enterprise deployments.
Synchronization can be implemented in different ways depending on the deployment:
- Camera to camera. Cameras exchange identity information directly with each other.
- On-premises. A local hub or gateway coordinates synchronization between cameras.
- Cloud. Identity information is synchronized through a cloud backend.
Privacy¶
ReID does not rely on privacy-sensitive biometric data such as facial features. Because it identifies people from ordinary appearance cues instead, it can be deployed in settings where face recognition is restricted or undesirable. The identity information it uses is temporary and appearance-based, kept only long enough to link sightings and then discarded, rather than stored as a lasting record of who a person is.
Efficient and flexible deployment¶
ReID is small and fast enough to run on-device, and exposes a single API for on-device, on-premises, cloud, and hybrid deployments. Compute and memory usage stay low enough for cost-effective edge hardware and cloud infrastructure. The models are trained on large amounts of video data using Plumerai's Tiny AI and reinforcement learning techniques.
Scope and limitations¶
ReID matches people across gaps in observation, but is not designed to reliably handle substantial changes in appearance or context. Because appearance is a weaker identity signal than a face, the following are out of scope:
- Re-identifying a person after they have changed clothing.
- Distinguishing people with near-identical clothing, such as school or company uniforms.
- Re-identifying a person across long time gaps, such as the next day.