Plumerai Video Intelligence library¶
This document describes how to use the Plumerai Video Intelligence software library. Visit one of the following pages for the API reference and and a basic example:
For more examples, see the examples page that illustrates how easy it is to use the API to build applications.
Visit the Plumerai website for more information about our smart home solutions, People Detection, Familiar Face Identification, and to try it out yourself live in the browser.
Overview¶
The Plumerai Video Intelligence software expects a video feed and computes bounding boxes for each object that is detected. In case the Plumerai software is built with Familiar Face Identification (FFID) enabled, faces are matched with a face library and the resulting person-boxes will get a face-ID when recognized.
The software assumes that the camera does not move. For the best results we recommend mounting the camera to a fixed position. The software works for any input resolution, but a higher resolution is always preferred and leads to higher accuracy. Although there is a small latency overhead the extra accuracy is usually worth it. The software works for any aspect ratio, but for best accuracy we recommend an aspect ratio between 16:9 and 9:16.
Contents of the package¶
The library package consists of the following files:
plumeraivideointelligence/include/plumerai/*.h
plumeraivideointelligence/lib/{target}/libplumeraivideointelligence.{a,so,dylib}
plumeraivideointelligence/python/plumerai_video_intelligence-{pytarget}.whl
plumeraivideointelligence/tests # optional in case of FFID with manual enrollment, for testing enrollment
plumeraivideointelligence/README.md
plumeraivideointelligence/VERSION
Multiple {target} options are available, including platforms with accelerators. For Arm Cortex-A Linux targets we also include files libgcc.a, libstdc++.a and libstdc++.so.6.0.24, which could be needed on older systems, see below.
The library package can be unzipped to a folder of your choice, which is referred to as $PLUMERAI_DIR below. For example:
unzip plumerai_video_intelligence_library.zip -d /opt/
export PLUMERAI_DIR=/opt/plumeraivideointelligence/
To install the Python API, see below.
Building and linking¶
Follow these steps to build and link your application with the Plumerai library (example applications can be found here for C++ and here for C):
- Add
$PLUMERAI_DIR/includeto the include path, e.g.-I$PLUMERAI_DIR/include. - Link to the Plumerai library, e.g.
$PLUMERAI_DIR/lib/{target}/libplumeraivideointelligence.a. - On Linux targets, link to the
m, pthread, dllibraries, e.g.-lm -lpthread -ldl.
Here's how an example C++ application can be built and linked:
g++ -std=c++11 example.cc $PLUMERAI_DIR/lib/arm-linux-gnueabihf/libplumeraivideointelligence.a -I$PLUMERAI_DIR/include -lpthread -ldl -o example
And similarly for C:
gcc -std=c99 example.c $PLUMERAI_DIR/lib/arm-linux-gnueabihf/libplumeraivideointelligence.a -I$PLUMERAI_DIR/include -lstdc++ -lm -lpthread -ldl -o example
There are different versions of the library for different targets:
-
There are
x86_64-linux-gnuandx86_64-macosversions which can be used with a normal 64-bit Intel or AMD CPU on Linux or x86-64 macOS respectively (see below for arm64 macOS). -
There is a
arm64-macosversion for more recent Apple computers with macOS and Arm CPUs. -
There is a
aarch64-linux-gnuversion for use on Cortex-A (or similar) Arm devices with a 64-bit CPU and OS. See below for instructions for systems with old versions of the C++ standard library. -
There are
arm-linux-gnueabi(compiled with-mfloat-abi=softfp) andarm-linux-gnueabihf(compiled with-mfloat-abi=hard) versions for the AArch32 target, for use on Arm devices with a 32-bit CPU and OS. See below for instructions for systems with old versions of the C++ standard library. -
There is a
arm-none-eabibare-metal version for Cortex-M microcontrollers. -
There is a version for Espressif ESP32-S3 microcontrollers.
-
There are versions for platforms with specialized neural network accelerators.
Linux dependency: glibc¶
On Linux targets, the library has been linked with GLIBC 2.19 which is the minimum required version to run our software. This is the version of C library that comes with GCC 4.9.4 by default.
Linux dependency: libstdc++ and libgcc¶
The library requires libstdc++.so.6.0.24 (or newer) and libgcc which come with GCC 7.2 (or newer). For systems with an older version of libstdc++ or libgcc, we provided copies of libstdc++ and libgcc in our release. We recommend linking to them statically, by adding their path to the linking command, for example:
LIB_DIR=$PLUMERAI_DIR/lib/arm-linux-gnueabihf
gcc -std=c99 example.c $LIB_DIR/libplumeraivideointelligence.a $LIB_DIR/libstdc++.a $LIB_DIR/libgcc.a -I$PLUMERAI_DIR/include -lm -lpthread -ldl -o example
It is possible to link with the libstdc++ library dynamically instead of statically, although this increases the total binary size. In this case, make sure the rpath property of the final executable or the LD_LIBRARY_PATH environment variable is set correctly so that the loader finds the newer libstdc++.so before the default system library. Note that libgcc does not allow for dynamic linking.
In case dynamic linking with libplumeraivideointelligence.so is preferred, then libstdc++.so must also be linked dynamically.
Installing Python wrapper¶
The Python wrapper for Linux and macOS can be installed for Python 3.9 or newer using:
We provide pre-built wheels for Arm and x86_64 for Python 3.9, 3.10, 3.11, and 3.12. If you have different requirements, please contact Plumerai.
An API description along with a Python example can be found here.
Expected latency¶
Below is a table with expected latency numbers for some example Arm CPUs and input resolutions1:
| OS | CPU | Threads | Latency @ 800x600x3 | Latency @ 1600x1200x3 |
|---|---|---|---|---|
| 32-bit | Cortex-A53 @ 1.2GHz2 | 1 | 42 ms | 44 ms |
| 64-bit | Cortex-A53 @ 1.2GHz2 | 1 | 42 ms | 44 ms |
| 64-bit | Cortex-A72 @ 1.5GHz3 | 1 | 20 ms | 21 ms |