Skip to content

Plumerai People Detection demo on ESP32-S3

This is a demo of Plumerai People Detection for the Espressif ESP32-S3-EYE board with the OV2640 camera. See also our blog for more information about the demo. For information about Plumerai People Detection in general, visit our website.

Instructions on how to integrate the Plumerai People Detection library into your own application on the ESP32-S3 are also available.

Requirements

The three requirements are the ESP32-S3-EYE itself, a host system, and Espressif's esptool:

  1. The ESP32-S3-EYE board. It might work with multiple version, but Plumerai tested it with the version that has ESP32-S3-EYE_SUB_V1.1 just above the display and ESP32-S3-EYE-MB_V2.2 on the back. The board should be attached to a computer through a USB cable that supports data transfer.

  2. A Linux, macOS, or Windows system with Python 2.7 or 3.4 or newer with Python's pip package manage installed. On macOS, it is required to first install a USB-to-UART-driver, otherwise the ESP32-S3-EYE will not show up under /dev/*.

  3. The esptool Python package from Espressif. Simply run pip install esptool or see the full installation instructions for more details. Alternatively, if you already have ESP-IDF installed because you are an ESP32-S3 developer, you can also find a copy of esptool in esp-idf/components/esptool_py/esptool/esptool.py.

How to flash

The ESP32-S3 demo is provided by Plumerai in the form of a .zip file. The contents of this file can be extracted to a folder of your choice. Make sure the following .bin files all end up in a single folder:

  1. bootloader.bin
  2. partition-table.bin
  3. plumerai_people_detection.bin

Open a terminal window, navigate to the folder containing the files above, and run the following command:

esptool.py --chip esp32s3 -p <USB_DEVICE_NAME> -b 460800 --before=default_reset --after=hard_reset --no-stub write_flash --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 bootloader.bin 0x10000 plumerai_people_detection.bin 0x8000 partition-table.bin

On some installations the above esptool.py will be simply named esptool, so if the above does not work try that instead. Here <USB_DEVICE_NAME> is the name of the USB device, which differs per device and operating system:

  • On Windows this typically is COM1, COM2, COM3, etc. depending on which other USB devices are connected. You can use the Windows Device Manager and check under Ports (COM & LTP) to find out which COM device the ESP32-S3-EYE actually is.
  • On Linux this is a tty device, typically /dev/ttyACM0 or /dev/ttyUSB0, depending on your system and which other devices are connected. To find the actual device name, see below.
  • On macOS this is a tty or cu device, typically with either a name starting with /dev/cu.usbmodem or a long name with hexadecimal characters under /dev/tty*. Make sure to install the USB-to-UART-driver first. To find the actual device name, see below.

For Linux and macOS, you can find the actual name by following these steps in a terminal window:

  1. Unplug the USB device.
  2. ls /dev/* > before.txt.
  3. Plug in the ESP32-S3-EYE USB device.
  4. ls /dev/* > after.txt.
  5. diff before.txt after.txt.

In case of issues with flashing, try this reset.

How to run the demo

The demo will automatically start within a second after the ESP32-S3-EYE has received power. Power supply does not have to be from a computer but can also be from e.g. a USB battery, power-bank, or USB wall socket. The demo works best if the camera is right-side-up and not rotated. Furthermore, for the best demo experience, the device needs to be placed in a fixed position and preferably not held by hand.

Frequently asked questions

Q. Flashing the device does not work

A. If flashing the device results in an error, or you get a message such as A fatal error occurred: Could not open /dev/ttyACM0, the port doesn't exist (while the port is correct), please see Espressif's troubleshooting page. Another common cause on a Linux system could be that your user account needs to be part of the correct group. Use ls -l /dev/ttyACM0 or stat -c '%G' /dev/ttyACM0 (or another port of course) to find out the group name, usually this is dialout. Use groups or id to verify if you are in this group, and if not, run sudo usermod -a -G dialout $USER, reboot, and try again.

Q. How does the demo show detected people?

A. Once the Plumerai People Detection software has found a person, it will draw a coloured bounding-box around the person. It will continue to do so until it no longer detects the person.

Q. The screen is dark, the colours are wrong, or the camera image displayed on screen is of bad quality.

A. In this case your ESP32-S3-EYE probably has a broken screen or camera, or their connections are loose. Double-check the connections or try with a different device.

Q. What can I do about poor detection quality?

A. Make sure the following are all satisfied:

  1. The device is not rotated or upside down, i.e. the Plumerai text should be on top of the screen. For best quality detections, the device should be positioned perfectly horizontally.
  2. In darkness or low-light situations the OV2640 camera quality is too bad to detect people. For situations where the camera needs to operate in the dark, we advise adding NIR LEDs to the system to light up the scene a bit or to use a camera that captures better quality images in low light.

Q. How fast does the demo run?

A. The Plumerai People Detection software uses a single core Xtensa LX7 at 240 MHz and runs at more than 3 frames per second, using less than 170KB of RAM and only 1.5MB of flash. The full demo runs around 30ms slower, since it also has to perform camera capture and display the images on the LCD screen. The demo runs on top of FreeRTOS but the Plumerai People Detection software can also run without an operating system.

Q. I like the demo, I want to use the Plumerai People Detection in my product. What are the next steps?

A. For more information about the Plumerai People Detection software and how to contact us, please visit our website.