Skip to content

Running the GStreamer demo

Selecting a camera

Some systems might have multiple cameras attached. The demo therefore requires the user to select a camera to use. You can use the gst-device-monitor-1.0 Video command to see which devices you have attached and their location (see under device.path on Linux).

For the different systems, this is the typical set-up:

  • On Linux with V4L the first camera is typically at /dev/video0, and subsequent cameras at /dev/videoN where N is 1, 2 or higher. If e.g. /dev/video1 does not work, try /dev/video2, since sometimes one index is skipped. Both the compressed JPEG (default) and raw YUY2 camera input formats are supported.
  • On Linux with libcamera and the PiCam, use picam as input_source argument and set --camera_input_format to YUY2. The demo currently does not support camera selection. It assumes there is only one camera attached.
  • On macOS the GStreamer demo simply requires an integer index for the camera ID. So for the first camera this will be simply 0, and subsequent camera are 1, 2, etc. Note that on macOS there are sometimes virtual cameras set-up as well, so the first actual camera might not have ID 0. On macOS, only the YUY2 camera input format is supported.

Choosing an input resolution and camera input format

Once the camera is chosen, the input resolution (width and height) need to be chosen. Again, the gst-device-monitor-1.0 Video command can provide this information. For example, the command might output: 

Device found:
 name  : Integrated_Webcam_HD: Integrate
 class : Video/Source
caps  : image/jpeg, width=1280, height=720, framerate=30/1
        image/jpeg, width=960, height=540, framerate=30/1
        image/jpeg, width=848, height=480, framerate=30/1
        image/jpeg, width=640, height=480, framerate=30/1
        image/jpeg, width=640, height=360, framerate=30/1
        video/x-raw, format=YUY2, width=640, height=480, framerate=30/1
        video/x-raw, format=YUY2, width=640, height=360, framerate=30/1
        video/x-raw, format=YUY2, width=424, height=240, framerate=30/1
        video/x-raw, format=YUY2, width=320, height=240, framerate=30/1
        video/x-raw, format=YUY2, width=320, height=180, framerate=30/1
        video/x-raw, format=YUY2, width=160, height=120, framerate=30/1
This means that two camera input formats are supported (compressed JPEG and raw YUY2), both supporting 5 input resolutions, of which 1280x720 in JPEG-mode is the highest.

The higher the input resolution, the better the results can become. However, note that this might slow down the entire demo application, because the camera capture and video-displaying might take up more resources. The frame rate displayed on screen is purely for the Plumerai People Detection algorithm itself, and does not count camera capture or displaying of the results.

Installing the demo application

Installation of the Plumerai People Detection demo can be done by following these steps:

  1. Unpack the plumerai_demo and the libplumerai_gstreamer_plugin.* files for your platform into a folder of your choice. Make sure both files are in the same folder.
  2. The binary needs execution permissions, so first (and only once) you'll have to run chmod +x ./plumerai_demo from the above folder inside the terminal.

Running the demo application

Running the demo can be done from the command line.

After following the steps above, you can run it with the following arguments: 

./plumerai_demo input_source video_width video_height {OPTIONS}

    Plumerai People Detection demo.

OPTIONS:

      -h, --help                        Display this help menu
      input_source                      This can be one of three options:
                                        camera, file, or RTSP stream. In case of
                                        a camera/webcam device, use e.g.
                                        `/dev/video0` on Linux or simply `0` on
                                        MacOS. In case of a pre-recorded video
                                        file from disk, use e.g.
                                        `~/Videos/video_file.mp4`. In case of an
                                        RTSP stream, use an URL starting with
                                        rtsp://. In case of a Raspberry Pi PiCam
                                        with libcamera, use `picam`. When using
                                        the PiCam, do also set
                                        'camera_input_format' to 'YUY2' instead
                                        of 'JPEG'.
      video_width                       The width of the camera input or
                                        (rescaled) video input, e.g. 1280.
      video_height                      The height of the camera input or
                                        (rescaled) video input, e.g. 720.
      --camera_input_format=[camera_input_format]
                                        Optional. Only in case an input camera
                                        is used, this option can be changed to
                                        set the camera input format. By default
                                        this is JPEG on Linux with V4L, raw YUY2
                                        for the PiCam, and raw YUY2 (=UYVY) on
                                        macOS. You can check with
                                        'gst-device-monitor-1.0 Video' what your
                                        device supports. Allowed values: 'JPEG'
                                        and 'YUY2'.
      --max_framerate=[max_framerate]   Optional, defaults to -1 (== no
                                        maximum). When in camera-input mode,
                                        will run at most at this framerate in
                                        frames-per-second (fps).
      --threshold=[threshold]           Optional, defaults to 0.7. Will not
                                        display any low-confidence
                                        bounding-boxes below this confidence
                                        value.
      --output_file=[output_file]       Optional. If set, do not display results
                                        but write to a video file. Requires the
                                        .mkv extension to be used.
      --input_framerate=[input_framerate]
                                        Sets the framerate of the input camera
                                        or input video file. In case of a video
                                        file this defaults to 30fps, otherwise
                                        it defaults to what GStreamer gets from
                                        the camera. In case of a camera
                                        framerate, it might be necessary to also
                                        specify the 'input_framerate_divider'
                                        option.
      --input_framerate_divider=[input_framerate_divider]
                                        Sets the framerate of the input camera
                                        or input video file. Might be necessary
                                        in case a camera device does not support
                                        e.g. 30/1 but does support the
                                        equivalent 10000000/333333. In that case
                                        set 'input_framerate' to 10000000 and
                                        'input_framerate_divider' to 333333.
                                        Defaults to 1.
      --image_output_path=[image_output_path]
                                        Optionally stores each unique person box
                                        detection as .ppm file on disk in this
                                        (existing) folder. Requires data in RGB
                                        format.

For example:

chmod +x plumerai_demo  # one time only

# Camera input in JPEG-mode, output to screen
./plumerai_demo /dev/video0 1280 720

# Camera input in YUY2-mode, output to screen
./plumerai_demo /dev/video0 640 480 --camera_input_format YUY2

# Video file input, output to screen
./plumerai_demo /path/to/input.mp4 1280 720

# Video file input, output to video file
./plumerai_demo /path/to/input.mp4 1280 720 --output_file output.mkv

chmod +x plumerai_demo  # one time only

# Camera input, output to screen
./plumerai_demo picam 800 600 --camera_input_format YUY2

# Video file input, output to screen
./plumerai_demo /path/to/input.mp4 800 600

# Video file input, output to video file
./plumerai_demo /path/to/input.mp4 800 600 --output_file output.mkv

# Give execution rights (one time only)
chmod +x plumerai_demo

# Add security exception to macOS (one time only)
xattr -d com.apple.quarantine plumerai_demo
xattr -d com.apple.quarantine libplumerai_gstreamer_plugin.dylib
xattr -d com.apple.quarantine libosxvideo.dylib

# Camera input, output to screen
./plumerai_demo 0 1280 720

# Video file input, output to screen
./plumerai_demo /path/to/input.mp4 1280 720

# Video file input, output to video file
./plumerai_demo /path/to/input.mp4 1280 720 --output_file output.mkv

Note about speed

Note that the demo application itself, i.e. the camera capture, video decoding and displaying, is not optimized for speed. Furthermore, the people detection algorithm in its current form is not optimized for speed on x86 systems, only for aarch64 targets such as Arm Cortex-A.