Skip to content

Plumerai Inference Engine C API for microcontrollers

This document describes the C API for the Plumerai Inference Engine for microcontrollers.

The API

PlumeraiInferenceInit

PlumeraiInference PlumeraiInferenceInit(unsigned char* tensor_arena_ptr,
                                        int tensor_arena_size);

Creates the inference engine object. The tensor arena has to be provided by the user and should be large enough to hold the model's activation tensors. Ideally the tensor arena is 16-byte aligned. The class does not take ownership of the tensor arena or optional profiler. The contents of the tensor arena should not be overwritten during the lifetime of the object, except by setting input tensor data through the corresponding functions.

PlumeraiInferenceAllocateTensors

TfLiteStatus PlumeraiInferenceAllocateTensors(PlumeraiInference* engine);

Allocates input, output and intermediate tensors in the tensor arena. This needs to be called before running inference with PlumeraiInferenceInvoke. Returns kTfLiteError when not enough space is available.

PlumeraiInferenceInvoke

TfLiteStatus PlumeraiInferenceInvoke(PlumeraiInference* engine);

Run inference assuming input data is already set. See the functions below for setting input data. Returns kTfliteOK if everything went right.

Input and output tensors

TfLiteTensor* PlumeraiInferenceInput(PlumeraiInference* engine, int input_id);
TfLiteTensor* PlumeraiInferenceOutput(PlumeraiInference* engine, int output_id);
int PlumeraiInferenceInputsSize(PlumeraiInference* engine);
int PlumeraiInferenceOutputsSize(PlumeraiInference* engine);

Get access to the input and output tensors. The TfLiteTensor object is the same as the one in Tensorflow (tensorflow/lite/c/common.h). Relevant functionality includes getting a pointer to the data, the datatype and the shape of the tensor:

TfLiteTensor* input_tensor = PlumeraiInferenceInput(&engine, 0);

TfLiteType input_data_type = input_tensor->type;

char* input_data = input_tensor->data.raw;

TfLiteIntArray* input_shape = input_tensor->dims;

PlumeraiInferenceArenaUsedBytes

int PlumeraiInferenceArenaUsedBytes(PlumeraiInference* engine);

For debugging only. This method gives the optimal arena size. It's only available after PlumeraiInferenceAllocateTensors has been called.

#include "plumerai_tf_compat.h"

int plumerai_printf(const char* format, ...);

// C interface to a C++ class using an opaque pointer
typedef struct {
  void* handle;
} PlumeraiInference;

// The lifetime of the tensor arena and optional profiler must be at least
// as long as that of the interpreter object, since the interpreter may need
// to access them at any time. The interpreter doesn't do any deallocation of
// any of the pointed-to objects, ownership remains with the caller.
PlumeraiInference PlumeraiInferenceInit(unsigned char* tensor_arena_ptr,
                                        int tensor_arena_size);
void PlumeraiInferenceCleanUp(PlumeraiInference* engine);

// Runs through the model and allocates all necessary input, output and
// intermediate tensors in the tensor arena.
TfLiteStatus PlumeraiInferenceAllocateTensors(PlumeraiInference* engine);

// Run inference, assumes input data is already set
TfLiteStatus PlumeraiInferenceInvoke(PlumeraiInference* engine);

TfLiteTensor* PlumeraiInferenceInput(PlumeraiInference* engine, int input_id);
TfLiteTensor* PlumeraiInferenceOutput(PlumeraiInference* engine, int output_id);
int PlumeraiInferenceInputsSize(PlumeraiInference* engine);
int PlumeraiInferenceOutputsSize(PlumeraiInference* engine);

// For debugging only.
// This method gives the optimal arena size. It's only available after
// `AllocateTensors` has been called.
int PlumeraiInferenceArenaUsedBytes(PlumeraiInference* engine);

Example

The Plumerai Inference Engine can be used as follows from a C program:

#include "plumerai_inference_c.h"

#include <stdio.h>

// Example tensor arena of 8KB, to be changed depending on the model
#define ARENA_SIZE (8 * 1024)
unsigned char tensor_arena[ARENA_SIZE];

// TODO: Implement this to define how debug printing is done
void DebugLog(const char *s) {
  // printf("%s", s);
}

int main(void) {
  PlumeraiInference engine = PlumeraiInferenceInit(tensor_arena, ARENA_SIZE);

  // Allocate memory from the tensor_arena for the model's tensors.
  TfLiteStatus allocate_status = PlumeraiInferenceAllocateTensors(&engine);
  if (allocate_status != kTfLiteOk) {
    plumerai_printf("AllocateTensors() failed\n");
    return 1;
  }

  // Obtain pointers to the model's input and output tensors.
  // TODO: Assumes the model has one input and one output, modify this if there
  // are more.
  TfLiteTensor* input = PlumeraiInferenceInput(&engine, 0);
  TfLiteTensor* output = PlumeraiInferenceOutput(&engine, 0);

  // Example: print the input shape
  plumerai_printf("Input shape:\n");
  for (int i = 0; i < input->dims->size; ++i) {
    plumerai_printf(" %d\n", input->dims->data[i]);
  }

  // Example: run inference in an infinite loop.
  while (true) {
    // Set input data example. TODO: Get data from sensor.
    char* input_data = input->data.raw;
    input_data[0] = 17;  // example, setting first element to '17'

    // Run inference on a single input.
    TfLiteStatus invoke_status = PlumeraiInferenceInvoke(&engine);
    if (invoke_status != kTfLiteOk) {
      plumerai_printf("Invoke failed\n");
      return 1;
    }

    // Read results and print first output to screen.
    char* output_data = output->data.raw;
    plumerai_printf("Result: %d\n", (int)output_data[0]);
  }

  return 0;
}