Plumerai People Detection demo on ESP32-S3¶
This is a demo of Plumerai People Detection for the Espressif ESP32-S3-EYE board with the OV2640 camera. See also our blog for more information about the demo. For information about Plumerai People Detection in general, visit our website.
Instructions on how to integrate the Plumerai People Detection library into your own application on the ESP32-S3 are also available.
The three requirements are the ESP32-S3-EYE itself, a host system, and Espressif's
The ESP32-S3-EYE board. It might work with multiple version, but Plumerai tested it with the version that has
ESP32-S3-EYE_SUB_V1.1just above the display and
ESP32-S3-EYE-MB_V2.2on the back. The board should be attached to a computer through a USB cable that supports data transfer.
A Linux, macOS, or Windows system with Python 2.7 or 3.4 or newer with Python's
pippackage manage installed. On macOS, it is required to first install a USB-to-UART-driver, otherwise the ESP32-S3-EYE will not show up under
esptoolPython package from Espressif. Simply run
pip install esptoolor see the full installation instructions for more details. Alternatively, if you already have ESP-IDF installed because you are an ESP32-S3 developer, you can also find a copy of
How to flash¶
The ESP32-S3 demo is provided by Plumerai in the form of a
.zip file. The contents of this file can be extracted to a folder of your choice. Make sure the following
.bin files all end up in a single folder:
Open a terminal window, navigate to the folder containing the files above, and run the following command:
On some installations the above
esptool.py will be simply named
esptool, so if the above does not work try that instead. Here
<USB_DEVICE_NAME> is the name of the USB device, which differs per device and operating system:
- On Windows this typically is
COM3, etc. depending on which other USB devices are connected. You can use the Windows
Device Managerand check under
Ports (COM & LTP)to find out which
COMdevice the ESP32-S3-EYE actually is.
- On Linux this is a
/dev/ttyUSB0, depending on your system and which other devices are connected. To find the actual device name, see below.
- On macOS this is a
cudevice, typically with either a name starting with
/dev/cu.usbmodemor a long name with hexadecimal characters under
/dev/tty*. Make sure to install the USB-to-UART-driver first. To find the actual device name, see below.
For Linux and macOS, you can find the actual name by following these steps in a terminal window:
- Unplug the USB device.
ls /dev/* > before.txt.
- Plug in the ESP32-S3-EYE USB device.
ls /dev/* > after.txt.
diff before.txt after.txt.
In case of issues with flashing, try this reset.
How to run the demo¶
The demo will automatically start within a second after the ESP32-S3-EYE has received power. Power supply does not have to be from a computer but can also be from e.g. a USB battery, power-bank, or USB wall socket. The demo works best if the camera is right-side-up and not rotated. Furthermore, for the best demo experience, the device needs to be placed in a fixed position and preferably not held by hand.
Frequently asked questions¶
Q. Flashing the device does not work¶
A. If flashing the device results in an error, or you get a message such as
A fatal error occurred: Could not open /dev/ttyACM0, the port doesn't exist (while the port is correct), please see Espressif's troubleshooting page. Another common cause on a Linux system could be that your user account needs to be part of the correct group. Use
ls -l /dev/ttyACM0 or
stat -c '%G' /dev/ttyACM0 (or another port of course) to find out the group name, usually this is
id to verify if you are in this group, and if not, run
sudo usermod -a -G dialout $USER, reboot, and try again.
Q. How does the demo show detected people?¶
A. Once the Plumerai People Detection software has found a person, it will draw a coloured bounding-box around the person. It will continue to do so until it no longer detects the person.
Q. The screen is dark, the colours are wrong, or the camera image displayed on screen is of bad quality.¶
A. In this case your ESP32-S3-EYE probably has a broken screen or camera, or their connections are loose. Double-check the connections or try with a different device.
Q. What can I do about poor detection quality?¶
A. Make sure the following are all satisfied:
- The device is not rotated or upside down, i.e. the
Plumeraitext should be on top of the screen. For best quality detections, the device should be positioned perfectly horizontally.
- In darkness or low-light situations the OV2640 camera quality is too bad to detect people. For situations where the camera needs to operate in the dark, we advise adding NIR LEDs to the system to light up the scene a bit or to use a camera that captures better quality images in low light.
Q. How fast does the demo run?¶
A. The Plumerai People Detection software uses a single core Xtensa LX7 at 240 MHz and runs at more than 3 frames per second, using less than 170KB of RAM and only 1.5MB of flash. The full demo runs around 30ms slower, since it also has to perform camera capture and display the images on the LCD screen. The demo runs on top of FreeRTOS but the Plumerai People Detection software can also run without an operating system.
Q. I like the demo, I want to use the Plumerai People Detection in my product. What are the next steps?¶
A. For more information about the Plumerai People Detection software and how to contact us, please visit our website.