Skip to content

Plumerai People Detection

At Plumerai we've built the world’s most accurate people detection for the edge. We deliver a complete software solution for camera-based people detection. Our models are tiny and fast and detect people with a high accuracy in any situation.

We have split the technical documentation for our people detection products into four sections:

Additionally, we provide example usage of the People Detection library in the form of a simple OpenCV webcam-to-display pipeline. Documentation and source code is available in a GitHub repository.

People detection features and examples

Tracking

The library allows for tracking each person by providing a unique ID for each detected bounding box. This enables the application to understand the movement of a person from frame to frame. This information can then be used to detect when a person crosses a line or enters a specific area, or determine the direction they are walking into.

Tracking: each bounding box has a different color, representing the unique IDs.

Robustness

The library accurately detects people in a wide range of situations:

  • Pose coverage: walking, standing, sitting, standing, and crouching.
  • Occlusion: where the person is blocked by an object, for example such as a desk or couch.
  • Diversity: people with different skin tones, ethnicities, genders, and ages.
  • Clothing and headdress: wearing a wide range of clothing and head coverings, including hats, masks, or a hijab.

A person crouching A person sitting

Example detections of people in different poses.

A partially occluded person A partially occluded person

Example detections of occluded, partially visible people.

Camera

The people detection library supports a wide variety of cameras:

  • Lens: supports a variety of lenses, including wide angle lenses with a 180 degrees field of view.
  • Resolution: from low resolution like QVGA (320x240) to beyond 4K (3840x2160).
  • Color: supports both color (RGB) or black and white images.
  • Aspect ratio: from landscape to portrait, for example 16:9, 1:1, or 9:16.

A color RGB image A black and white image

Lighting

The people detection library correctly detects people under a wide variety of lighting conditions:

  • Low light: even with noisy or blurry images the detector is accurate.
  • Night vision: cameras that use NIR LEDs to light up the scene in the dark are supported.

Night vision image Night vision image Night vision image Low light situation

Detections in varying lighting conditions and with IR LED lighting.

Distance

Detects people up close or at a distance:

  • Close: detects people that are up close and only partially in view.
  • Distant: detects people at more than 20 meters or 65 feet.

Person far away Person far away

Examples of detections at a distance.

Mounting

A variety of mounting heights and angles are supported:

  • Standard: mounted at a height of 1-2 meters or 3-6 feet.
  • High: mounted at a height of 2-5 meters or 6-15 feet.
  • Low: mounted lower than 1 meter or 3 feet.
  • Angles: from level to 45 degrees downward.

High angle image High angle image Low angle image Standard angle image

Cameras mounted in different positions and at different angles.

Locations

Whatever the location, the people detection performs well:

  • Indoor: variety of settings such as homes, offices, stores, hospitals, and malls.
  • Outdoor: variety of locations such as office and home surroundings such as streets, yards, parking lots, sidewalks.

Fixed or moving camera

Both fixed and moving cameras are supported:

  • Fixed: cameras mounted on the wall or embedded into a device such as a display or laptop.
  • Moving: cameras mounted on a robot or vehicle, or pan-tilt-zoom cameras.

Dataset

The software has been trained with over 32 million images and videos to ensure robust performance in a large variety of situations:

  • Curated: to ensure that all the important use-cases are included, and in the right balance.
  • Unbiased: we use proprietary data analysis methods to identify specific issues stemming from sampling biases in public datasets.
  • Varied: our dataset consists of a very large variety of images from all kinds of people, settings and conditions.
  • Labeled: we have an extensive methodology to label our datasets to ensure labels are accurate and precise.

Validation

  • Validation: The people detection models have been extensively validated on a diverse range of people, poses, settings, and situations.
  • Unit tests: Data unit tests guarantee that the model works reliably for specific subsets of our dataset. For example, for people standing far away or for people who are only showing their back.

Privacy

Our people detection software ensures that personal data is protected and that privacy is maintained:

  • Edge compute: our people detection model is small and lightweight, enabling it to run on the device rather than in the cloud where the image data can more easily be accessed or misused.
  • No image storage: the original image data is not stored on the device, eliminating the possibility of misuse.
  • No image transmission: since all computations are done at the edge, the images do not need to be transmitted over the internet, eliminating the possibility of interception during transmission.
  • Distributed: Since the image processing happens on the devices at the edge, instead of in a single location on a central server in the cloud, there’s protection against potential misuse or unauthorized access.

Platform support

The people detection runs on the following processor platforms:

  • Arm Cortex-A with/without Neon
    • Cortex-A5, A7, A9, A53, A72, etc
    • Snapdragon, NXP i.MX, TI Sitara
    • Ambarella, SigmaStar, Realtek
    • Raspberry Pi
  • Arm Cortex-M with/without Helium
    • Arm Cortex-M33, M4, M7, M55, M85
  • x86 SSE2, SSE4, AVX, AVX2, AVX512
    • Intel Celeron, i3, i5, i7, Xeon, etc
    • AMD Opteron, Ryzen, Epyc, etc
  • ESP32-S3, RISC-V, ARC EM, ARM7, ARM9
  • Various NPUs and accelerators