Skip to content

Plumerai People Detection library for microcontrollers

This document describes how to use the Plumerai People Detection library on e.g. Arm Cortex-M and Espressif ESP32-S3 microcontrollers. For documentation about the API and an example, please look here. Go here for more information about the Plumerai People Detection solution and to try it out yourself. For specific instructions on how to build an example on the ESP32-S3 with ESP-IDF/WHO, see here for more information.

Note: currently our microcontroller library has a different API than our other platform targets. The microcontroller-specific API will be replaced by the generic C/C++ API in the future.

Overview

The Plumerai People Detection software expects a video feed and computes bounding boxes with a confidence value for each detected person. For the best results we recommend mounting the camera to a fixed position. The software works with a pre-determined input resolution (e.g. 320x240 or 640x480) in a 3-channel RGB format either as RGB888 (3 bytes per pixel) or RGB565 (2 bytes per pixel). If your camera produces images in a different format, Plumerai can provide a library with an optimized resize/convert function specific for your use-case.

Supported microcontrollers

The Plumerai People Detection library works on many different microcontrollers. It has specifically been optimized and tested for:

  • Arm Cortex-M0/M0+
  • Arm Cortex-M4 with DSP instructions
  • Arm Cortex-M7 with DSP instructions
  • Arm Cortex-M33 with DSP instructions
  • Arm Cortex-M55 with Helium extensions
  • Arm Cortex-M85 with Helium extensions
  • Espressif ESP32-S3 with Tensilica Xtensa LX7
  • Synopsys ARC EM4
  • RISC-V

Contents of the package

The library package consists of the following files inside a plumerai_people_detection_micro.zip archive:

plumerai_people_detection_micro/include/plumerai/people_detection_micro.h
plumerai_people_detection_micro/include/plumerai/box_prediction.h
plumerai_people_detection_micro/include/plumerai/model_defines.h
plumerai_people_detection_micro/lib/{target}/libplumerai_people_detection_micro.a

where {target} can be arm-cortex-m7-fpv5-d16-hard or arm-cortex-m4-fpv4-sp-d16-hard for example, indicating (in order) the target micro-controller family (e.g. arm-cortex-m4), the FPU type (e.g. fpv4-sp-d16), and the float-abi (e.g. hard).

Building

To build, make sure the header files can be found on the compiler include path, and link with the libplumerai_people_detection_micro.a suited for your target device. An example (device-agnostic) application can be found at the bottom of the API page. You’ll have to add your own device-specific start-up code, camera read-out code, and other boilerplate code to actually run it on device.