Plumerai People Detection¶
At Plumerai we've built the world’s most accurate people detection for the edge. We deliver a complete software solution for camera-based people detection. Our models are tiny and fast and detect people with a high accuracy in any situation.
We have split the technical documentation for our people detection products into four sections:
Additionally, we provide example usage of the People Detection library in the form of a simple OpenCV webcam-to-display pipeline. Documentation and source code is available in a GitHub repository.
People detection features and examples¶
The library allows for tracking each person by providing a unique ID for each detected bounding box. This enables the application to understand the movement of a person from frame to frame. This information can then be used to detect when a person crosses a line or enters a specific area, or determine the direction they are walking into.
The library accurately detects people in a wide range of situations:
- Pose coverage: walking, standing, sitting, standing, and crouching.
- Occlusion: where the person is blocked by an object, for example such as a desk or couch.
- Diversity: people with different skin tones, ethnicities, genders, and ages.
- Clothing and headdress: wearing a wide range of clothing and head coverings, including hats, masks, or a hijab.
The people detection library supports a wide variety of cameras:
- Lens: supports a variety of lenses, including wide angle lenses with a 180 degrees field of view.
- Resolution: from low resolution like QVGA (320x240) to beyond 4K (3840x2160).
- Color: supports both color (RGB) or black and white images.
- Aspect ratio: from landscape to portrait, for example 16:9, 1:1, or 9:16.
The people detection library correctly detects people under a wide variety of lighting conditions:
- Low light: even with noisy or blurry images the detector is accurate.
- Night vision: cameras that use NIR LEDs to light up the scene in the dark are supported.
Detects people up close or at a distance:
- Close: detects people that are up close and only partially in view.
- Distant: detects people at more than 20 meters or 65 feet.
A variety of mounting heights and angles are supported:
- Standard: mounted at a height of 1-2 meters or 3-6 feet.
- High: mounted at a height of 2-5 meters or 6-15 feet.
- Low: mounted lower than 1 meter or 3 feet.
- Angles: from level to 45 degrees downward.
Whatever the location, the people detection performs well:
- Indoor: variety of settings such as homes, offices, stores, hospitals, and malls.
- Outdoor: variety of locations such as office and home surroundings such as streets, yards, parking lots, sidewalks.
Fixed or moving camera¶
Both fixed and moving cameras are supported:
- Fixed: cameras mounted on the wall or embedded into a device such as a display or laptop.
- Moving: cameras mounted on a robot or vehicle, or pan-tilt-zoom cameras.
The software has been trained with over 32 million images and videos to ensure robust performance in a large variety of situations:
- Curated: to ensure that all the important use-cases are included, and in the right balance.
- Unbiased: we use proprietary data analysis methods to identify specific issues stemming from sampling biases in public datasets.
- Varied: our dataset consists of a very large variety of images from all kinds of people, settings and conditions.
- Labeled: we have an extensive methodology to label our datasets to ensure labels are accurate and precise.
- Validation: The people detection models have been extensively validated on a diverse range of people, poses, settings, and situations.
- Unit tests: Data unit tests guarantee that the model works reliably for specific subsets of our dataset. For example, for people standing far away or for people who are only showing their back.
Our people detection software ensures that personal data is protected and that privacy is maintained:
- Edge compute: our people detection model is small and lightweight, enabling it to run on the device rather than in the cloud where the image data can more easily be accessed or misused.
- No image storage: the original image data is not stored on the device, eliminating the possibility of misuse.
- No image transmission: since all computations are done at the edge, the images do not need to be transmitted over the internet, eliminating the possibility of interception during transmission.
- Distributed: Since the image processing happens on the devices at the edge, instead of in a single location on a central server in the cloud, there’s protection against potential misuse or unauthorized access.
The people detection runs on the following processor platforms:
- Arm Cortex-A with/without Neon
- Cortex-A5, A7, A9, A53, A72, etc
- Snapdragon, NXP i.MX, TI Sitara
- Ambarella, SigmaStar, Realtek
- Raspberry Pi
- Arm Cortex-M with/without Helium
- Arm Cortex-M33, M4, M7, M55, M85
- x86 SSE2, SSE4, AVX, AVX2, AVX512
- Intel Celeron, i3, i5, i7, Xeon, etc
- AMD Opteron, Ryzen, Epyc, etc
- ESP32-S3, RISC-V, ARC EM, ARM7, ARM9
- Various NPUs and accelerators