Skip to content

Plumerai Video Intelligence demo on Realtek Ameba Pro2

This demo is created for the AMB82 MINI board with the Realtek Ameba Pro2 (RTL8735B) chip. This demo showcases the Plumerai People Detection and Plumerai Familiar Face Identification products running on the Ameba Pro2.

The demo has the following features:

  1. It draws tracked bounding boxes around detected people and faces on the RTSP video stream.
  2. It prints the same tracked bounding box information in the UART terminal.
  3. It allows on-device face enrollment to create a face library (see below).
  4. It displays identified faces from the library on the RTSP stream and in the UART terminal.

The demo runs at roughly 10 frames per second on 1920x1080 video input. The Plumerai neural networks run very efficiently on the Ameba Pro2 NPU and fit within only 6MB of flash.

This Ameba Pro2 demo can be requested by contacting us and specifying 'Ameba Pro2 demo' in the subject line.

Instructions on how to integrate the Plumerai Video Intelligence library with Plumerai People Detection and Plumerai Familiar Face Identification into your own application on the Ameba Pro2 can be requested by contacting us and specifying 'Ameba Pro2 library' in the subject line.

Device preparation

  1. Connect the supplied camera to the AMB82 MINI board following the official Realtek instructions.
  2. Tape the camera to the board such that the camera doesn't move.
  3. Position the board upright and on a steady position: do not move the board while testing; the Plumerai software assumes a static camera.
  4. Clean the camera lens if necessary.

How to flash

See the official Realtek documentation for more information. This documentation is available from AmebaIoT.com and requires you to login.

  1. Connect the board to USB using the port with the RED LED, on the right side where it says CH34C.
  2. Bring the device into download mode by pressing the reset button (right side) while holding the 'load code' button (left side). Ssee also the official Realtek manual.
  3. Download the official Realtek Ameba Pro2 image flash tools.
  4. Flash the board using the official instructions. On Linux this could look like ./uartfwburn.linux -p /dev/ttyUSB0 -f /path/to/flash_files/flash_ntz.nn.bin -b 3000000 -n pro2.
  5. Use the reset button (right side) to start the device.

How to run the demo

Once the demo is flashed on the board (see above), follow these steps to run it:

  1. Connect the board to a computer using the USB port with the RED LED, on the right side where it says CH34C.
  2. Connect to the board using UART, e.g. with PuTTY, screen, minicom, etc. See the official Realtek instructions.
  3. In the UART terminal, connect to WiFi using the ATW0=<SSID_NAME>, ATW1=<PASSWORD>, and ATWC commands as normal. See the official Realtek documentation.
  4. Connect to the RTSP stream as normally (use ATW? to find out the IP address), e.g. using VLC at rtsp://<SOME_IP>/video0, which should now show the output of the Plumerai software. See the Realtek documentation.
  5. The UART terminal will also print information about detected boxes.

How to use Familiar Face Identification

For Familiar Face Identification to work, a person first has to enroll themselves. We use manual face enrollment in this demonstration, where the user explicitly rolls their head in front of the camera to enroll. This can be done by typing the ENROLL=<SOME_NAME> command in the UART terminal. A red progress bar along with instructions will now be shown on the RTSP stream:

Enrollment in progress

Example of enrollment in progress (extra annotations in pink).

Make sure only one face is visible in the camera's view. Move your face up and down and left-to-right until the progress bar is filled. Once enrollment is finished the demo will automatically switch back to normal mode. Once a familiar face is identified, it will be shown in the RTSP stream (highlighted in red with their name) and also printed in the UART terminal:

Successful face identification

Successful face identification (extra annotations in pink).

Up to 20 people can be enrolled at the same time. A single person should be enrolled only once.

In the top right corner of the RTSP stream, the current face library is printed. To clear the entire library, use the FACECLEAR command in the UART terminal. The face library is persistent across reboots of the device.

Frequently asked questions

Q. How does the demo show detected people?

A. Once the Plumerai Video Intelligence software has found a person or a face, it will draw a grey bounding-box around the person. It will continue to do so until it no longer detects it.

Q. What can I do about poor detection quality?

A. Make sure the following are all satisfied:

  1. The device is not rotated or upside down. For best quality detections, the device should be positioned perfectly horizontally.
  2. In darkness or low-light situations the camera quality might be too bad to detect people since the camera module does not support night vision. Try during daytime or add additional lighting to the scene.

Q. How fast does the demo run?

A. The Plumerai Video Intelligence software runs at approximately 10 frames per second using the Ameba Pro2 NPU.

Q. I like the demo, I want to use the Plumerai software in my product. What are the next steps?

A. For more information about the Plumerai Video Intelligence software and how to contact us, please visit our website.