Skip to content

Plumerai Advanced Motion Detection

This page explains what Plumerai Advanced Motion Detection is and how it can be used.

Plumerai Advanced Motion Detection is optimized for low-cost cameras running at the edge, eliminating the need to transfer data to the cloud or another device It is developed around the following three principles:

  • It only detects relevant motion for security and smart-home applications, such as moving people, vehicles, animals, and hazards such as a falling tree or a fire. It ignores natural phenomena such as wind, rain, snow, or lighting changes. See below for more information.
  • It runs efficiently in real-time on constrained devices, using as little CPU and memory as possible. See below for more information.
  • It can start at any moment: unlike many other algorithms, it does not require an initial frame without motion or a learning phase. This is essential for cameras that go to sleep based on a secondary sensor, such as a PIR sensor.
Figure 1: Advanced Motion Detection applied to a simple scene.
The red squares indicate where the algorithm detected motion.

Robustness

Plumerai Advanced Motion Detection is designed to be robust against natural phenomena and lighting changes. Below we illustrate this robustness with a few examples of different categories.

In all videos below we show the Plumerai algorithm on the left and a simple motion detection algorithm on the right. The simple algorithm first applies blurring to remove camera noise, then computes a moving average based background model, averages over a small area, and finally applies a threshold to determine whether there is motion. Of course, a different algorithm might perform better or worse in these situations: we mainly show it as a reference to highlight where other algorithms might fail.

Robustness against rain

The Plumerai algorithm ignores both the falling rain and the raindrops hitting the puddle. Motion is only reported for the moving car and motorbike.

Figure 2: Robustness against rain

Robustness against snow

The Plumerai algorithm does not report any motion in the heavy snowfall, despite the snowflakes being illuminated by the IR light of the camera.

Figure 3: Robustness against snow

Robustness against lighting changes

In this example Plumerai reports motion only for the moving person, despite the changing lighting conditions. In this case, a cloud cover suddenly blocks the sunlight and the illumination changes in parts of the view. The algorithm recognizes that the lighting changes, but the texture does not. Lighting changes that are not caused by sun, such as turning on a light, are also handled in a similar way.

Figure 4: Robustness against lighting changes, e.g. cloud cover

Robustness against IR-illuminated dust

Both indoors and outdoors, a camera’s IR light can illuminate tiny airborne dust particles that are otherwise invisible to the human eye. The Plumerai algorithm specifically suppresses this type of motion.

Figure 5: Robustness against IR-illuminated dust at night

Robustness against motion caused by wind

The Plumerai algorithm does not report any motion for the moving trees and plants, despite the leaves moving in the wind. It achieves this by recognizing the dynamic and repetitive nature of their movement.

Figure 6: Robustness against motion caused by wind

Efficiency

Before we discuss the efficiency of the Plumerai Advanced Motion Detection algorithm, we first have some good news for users of the Plumerai Video Intelligence API: the algorithm is already running and can be used without any additional memory or compute cost! In other words, if you are already running e.g. Plumerai People Detection or Plumerai Familiar Face Identification, you can use the motion detection results without any compute or memory cost.

If you are not planning to run any other Plumerai algorithms, the Plumerai Advanced Motion Detection algorithm is still very efficient:

  • In terms of ROM, the algorithm is very efficient and only requires a few kilobytes of memory for code: there are no neural networks or other ML weight arrays.

  • In terms of RAM, the requirement depends on the input resolution and a few other factors. For a typical configuration and a few common input resolutions, the RAM usage is as follows:

    Input resolution RAM usage
    640 x 480 243 KB
    800 x 600 370 KB
    1280 x 720 693 KB
    1600 x 1200 1425 KB
    1920 x 1080 1537 KB

    Contact Plumerai support in case this memory usage is prohibitive for your system, we can tune several internal parameters.

  • The Plumerai Advanced Motion Detection algorithm also consumes a very low amount of CPU cycles and can run practically on any system, ranging from powerful x86 or Aarch64 machines all the way down to low-power and low-cost microcontrollers such as the Arm Cortex-M3 or an Espressif ESP32-S3. Even on an Arm Cortex-M3, it can run at 10 FPS while consuming less than 10% of the CPU cycles.

Using the API

Plumerai Advanced Motion Detection is part of the Plumerai Video Intelligence API and can be used from C++, C or Python. The videos on this page show examples of the output of the algorithm (the red rectangles), but the actual API provides more control:

  • You can configure the grid size, and thus control the granularity at which you want to detect motion. In the example videos on this page a grid size of 32x24 was used.
  • The output at each grid cell isn't a simple yes/no answer, but a floating-point value indicating the amount of motion. This can be used to control the sensitivity of the algorithm. In the example videos on this page, any non-zero motion was visualized as a red rectangle.