eIQ MCU Media Processing Pipeline API
MPP VERSION 3.5
1. MPP API
Functions
int mpp_api_init (mpp_api_params_t ∗params)
mpp_t mpp_create (mpp_params_t ∗params, int ∗ret)
int mpp_camera_add (mpp_t mpp, const char ∗name, mpp_camera_params_t ∗params)
int mpp_static_img_add (mpp_t mpp, mpp_img_params_t ∗params, void ∗addr)
int mpp_display_add (mpp_t mpp, const char ∗name, mpp_display_params_t ∗params)
int mpp_element_add (mpp_t mpp, mpp_element_id_t id, mpp_element_params_t ∗params, mpp_elem_handle_t ∗elem_h)
int mpp_split (mpp_t mpp, unsigned int num, mpp_params_t ∗params, mpp_t ∗out_list)
int mpp_background (mpp_t mpp, mpp_params_t ∗params, mpp_t ∗out_mpp)
int mpp_element_update (mpp_t mpp, mpp_elem_handle_t elem_h, mpp_element_params_t ∗params)
int mpp_start (mpp_t mpp, int last)
char ∗mpp_get_version (void)
1.1 Detailed Description
This section provides the detailed documentation for the MCU Media Processing Pipeline API.
1.1.1 Function Documentation
mpp_api_init()
int mpp_api_init ( mpp_api_params_t ∗ params )
Pipeline initialization. This function initializes the library and its data structures. It must be called before any other function of the API is called.
Parameters
in/out |
name |
description |
---|---|---|
in |
params |
API global parameters |
out |
ret |
return code (0 - success, non-zero - error) |
Returns
2. mpp_create()
mpp_t mpp_create ( mpp_params_t ∗ params, int ∗ ret )
Basic pipeline creation. This function returns a handle to the pipeline.
Parameters
in/out |
name |
description |
---|---|---|
in |
params |
pipeline parameters |
out |
ret |
return code (0 - success, non-zero - error) |
Returns
handle to the pipeline if success, NULL if there is an error.
mpp_camera_add()
int mpp_camera_add ( mpp_t mpp, const char ∗ name, mpp_camera_params_t ∗ params )
Camera addition.
This function adds a camera to the pipeline.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
name |
camera driver name |
in |
params |
parameters to be configured on the camera |
Returns
int mpp_static_img_add ( mpp_t mpp, mpp_img_params_t ∗ params, void ∗ addr, mpp_elem_handle_t ∗ elem_h )
Static image addition.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
params |
static image parameters |
in |
addr |
image buffer |
out |
elem_h |
element handle in pipeline |
Returns
Precondition
Image buffer allocation/free is the responsibility of the user.
mpp_display_add()
int mpp_display_add ( mpp_t mpp, const char ∗ name, mpp_display_params_t ∗ params )
Display addition.
This function adds a display to the pipeline.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
name |
display driver name |
in |
params |
parameters that are configured on the display |
Returns
int mpp_nullsink_add ( mpp_t mpp )
Null sink addition.
This function adds a null-type sink to the pipeline.
After this call pipeline is closed and no further elements can be added. Input frames are discarded.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
Returns
mpp_element_add()
int mpp_element_add ( mpp_t mpp, mpp_element_id_t id, mpp_element_params_t ∗ params, mpp_elem_handle_t ∗ elem_h )
Add processing element (single input, single output) This function adds an element to the pipeline. Available elements are:
2D image processing
ML inference engine
Labeled rectangle
Compositor
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
id |
element id |
in |
params |
element parameters |
out |
elem_h |
element handle in pipeline |
Returns
mpp_split()
int mpp_split ( mpp_t mpp, unsigned int num, mpp_params_t ∗ params, mpp_t ∗ out_list )
Pipeline multiplication.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
num |
number of output pipeline |
in |
params |
split mpp parameters |
out |
out_list |
list of output pipelines |
Returns
Precondition
out_list array must contain at least num elements.
mpp_background()
int mpp_background ( mpp_t mpp, mpp_params_t ∗ params, mpp_t ∗ out_mpp )
Put next elements processing in background.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
params |
new mpp parameters (exec_flag must be MPP_EXEC_PREEMPT) |
out |
out_mpp |
output pipeline |
Returns
mpp_element_update()
int mpp_element_update ( mpp_t mpp, mpp_elem_handle_t elem_h, mpp_element_params_t ∗ params )
Update element parameters.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
input pipeline |
in |
elem_h |
element handle in the pipeline. |
in |
params |
new element parameters |
Returns
int mpp_start ( mpp_t mpp, int last )
Start pipeline.
When called with last=0, this function prepares the branch of the pipeline specified with mpp. When called with last!=0, this function starts the data flow of the pipeline.
Data flow should start after all the branches of the pipeline have been prepared.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
pipeline branch handle to start/prepare |
in |
last |
if non-zero start pipeline processing. No further start call is possible thereafter. |
Returns
int mpp_stop ( mpp_t mpp )
Stop a branch of the pipeline.
This function stops the data processing and peripherals of a pipeline branch.
Parameters
in/out |
name |
description |
---|---|---|
in |
mpp |
pipeline branch to stop |
Returns
void mpp_stats_enable ( mpp_stats_grp_t grp )
Enable statistics collection.
This function enables statistics collection for a given group. Statistics collection is disabled by default after API initialization. Calling this function when stats are enabled has no effect.
Parameters
in/out |
name |
description |
---|---|---|
in |
grp |
statistics group |
Returns
void mpp_stats_disable ( mpp_stats_grp_t grp )
Disable statistics collection.
This function disables statistics collection for a given group. Calling this function when stats are disabled has no effect. This function is used to ensure stats are not updated while application tasks use the stats structures.
Parameters
in/out |
name |
description |
---|---|---|
in |
grp |
statistics group |
char ∗ mpp_get_version ( void )
Get MPP version.
Returns
pointer to the MPP version string
2. MPP Types
Data Structures
union mpp_stats_t
struct mpp_api_params_t
struct mpp_params_t
struct mpp_camera_params_t
struct mpp_img_params_t
struct mpp_display_params_t
struct mpp_tensor_dims_t
struct mpp_inference_cb_param_t
union mpp_color_t
struct mpp_labeled_rect_t
struct mpp_area_t
struct mpp_dims_t
struct mpp_position_t
struct mpp_inference_params_t
struct mpp_element_params_t
struct mpp_stats_t.api
struct mpp_stats_t.mpp
struct mpp_stats_t.elem
struct mpp_color_t.rgb
Macros
#define MPP_INFERENCE_MAX_OUTPUTS
#define MPP_INFERENCE_MAX_INPUTS
#define MPP_INVALID
#define MPP_EVENT_ALL
#define MAX_TENSOR_DIMS
Typedefs
typedef void ∗mpp_t
typedef uintptr_t mpp_elem_handle_t
typedef unsigned int mpp_evt_mask_t
typedef int(∗inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗)
Enumerations
enum mpp_evt_t {
MPP_EVENT_INFERENCE_OUTPUT_READY,
enum mpp_exec_flag_t {
enum mpp_stats_grp_t {
enum mpp_rotate_degree_t {
enum mpp_flip_mode_t {
enum mpp_convert_ops_t {
enum mpp_pixel_format_t {
enum mpp_element_id_t {
enum mpp_tensor_type_t {
enum mpp_tensor_order_t {
enum mpp_inference_type_t {
2.1 Detailed Description
This section provides the detailed documentation for the MCU Media Processing Pipeline types.
2.1.1 Data Structure Documentation
Data Fields
type |
name |
description |
---|---|---|
struct mpp_stats_t.api |
api |
Global execution performance counters. |
struct mpp_stats_t.mpp |
mpp |
Pipeline execution performance counters. |
struct mpp_stats_t.elem |
elem |
Element execution performance counters. |
Data Fields
type |
name |
description |
---|---|---|
stats |
API stats. |
|
unsigned int |
rc_cycle_min |
minimum cycle duration for RC tasks (ms), 0: sets default value |
unsigned int |
rc_cycle_inc |
time increment for RC tasks (ms), 0: sets default value |
int |
pipeline_task_max_prio |
pipeline tasks maximum priority. |
Pipeline creation parameters.
Data Fields
int(∗evt_callback_f )(mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data)
mpp_evt_mask_t mask
mpp_exec_flag_t exec_flag
void ∗cb_userdata
mpp_stats_t ∗stats
Camera parameters.
Data Fields
type |
name |
description |
---|---|---|
int |
height |
buffer height |
int |
width |
buffer width |
format |
pixel format |
|
int |
fps |
frames per second |
bool |
stripe |
stripe mode |
Static image parameters.
Data Fields
type |
name |
description |
---|---|---|
int |
height |
buffer height |
int |
width |
buffer width |
format |
pixel format |
|
bool |
stripe |
stripe mode |
int |
compressed_size |
size in bytes for compressed format |
Display parameters.
Data Fields
type |
name |
description |
---|---|---|
int |
height |
buffer resolution: setting to 0 will default to panel physical resolution |
int |
width |
buffer resolution: setting to 0 will default to panel physical resolution |
int |
pitch |
buffer resolution: setting to 0 will default to panel physical resolution |
int |
left |
active rect: setting to 0 will default to fullscreen |
int |
top |
active rect: setting to 0 will default to fullscreen |
int |
right |
active rect: setting to 0 will default to fullscreen |
int |
bottom |
active rect: setting to 0 will default to fullscreen |
rotate |
rotate degree |
|
format |
pixel format |
|
bool |
stripe |
stripe mode |
void ∗ |
handle |
pointer to an lvgl image widget |
Inference tensor dimensions.
Data Fields
type |
name |
description |
---|---|---|
uint32_t |
size |
|
uint32_t |
data[MAX_TENSOR_DIMS] |
tensor parameters
Data Fields
type |
name |
description |
---|---|---|
const uint8_t ∗ |
data |
data address |
dims |
tensor data dimensions |
|
type |
tensor data type |
Inference callback parameters.
Data Fields
type |
name |
description |
---|---|---|
void ∗ |
user_data |
callback will pass this pointer |
out_tensors[MPP_INFERENCE_MAX_OUTPUTS] |
output tensors parameters |
|
int |
inference_time_ms |
inference run time measurement - output to user |
inference_type |
type of the inference |
mpp color encoding
Data Fields
type |
name |
description |
---|---|---|
uint32_t |
raw |
Raw color. |
struct mpp_color_t.rgb |
rgb |
rgb color values RGB color |
mpp labeled rectangle element structure
Data Fields
type |
name |
description |
---|---|---|
uint8_t |
label[64] |
label to print |
uint16_t |
clear |
clear rectangle |
uint16_t |
line_width |
rectangle line thickness |
line_color |
rectangle line color |
|
int16_t |
top |
rectangle top position |
int16_t |
left |
rectangle left position |
int16_t |
bottom |
rectangle bottom position |
int16_t |
right |
rectangle right position |
uint16_t |
tag |
labeled rectangle tag |
uint16_t |
reserved |
pad for 32 bits alignment |
bool |
stripe |
stripe mode |
Image area coordinates.
Data Fields
type |
name |
description |
---|---|---|
int |
top |
|
int |
left |
|
int |
bottom |
|
int |
right |
Image dimensions.
Data Fields
type |
name |
description |
---|---|---|
unsigned int |
width |
|
unsigned int |
height |
Image position.
Data Fields
type |
name |
description |
---|---|---|
int |
||
int |
left |
Model parameters.
Data Fields
type |
name |
description |
---|---|---|
uint64_t |
constant_weight_MemSize |
model constant weights memory size |
uint64_t |
mutable_weight_MemSize |
Defines the amount of memory required both input & output data buffers. |
uint64_t |
activations_MemSize |
Size of scratch memory used for intermediate computations needed by the model. |
int |
num_inputs |
model’s number of inputs |
int |
num_outputs |
model’s number of outputs |
uint64_t |
inputs_offsets[MPP_INFERENCE_MAX_INPUTS] |
offset of each input |
uint64_t |
outputs_offsets[MPP_INFERENCE_MAX_OUTPUTS] |
offset ofeach output |
model_entry_point |
function called to perform the inference |
|
model_input_tensors_type |
type of input buffer |
Static image and Processing elements parameters.
Data Fields
type |
name |
description |
---|---|---|
__unnamed__ |
||
stats |
Data Fields
type |
name |
description |
---|---|---|
unsigned int |
rc_cycle |
run-to-completion (RC) cycle duration (ms) |
unsigned int |
rc_cycle_max |
run-to-completion work deadline (ms) |
unsigned int |
pr_slot |
available slot for preemptable (PR) work (ms) |
unsigned int |
pr_rounds |
number of RC cycles required to complete one PR cycle (ms) |
unsigned int |
app_slot |
remaining time for application (ms) |
Data Fields
type |
name |
description |
---|---|---|
mpp |
||
unsigned int |
mpp_exec_time |
pipeline execution time (ms) |
Data Fields
type |
name |
description |
---|---|---|
hnd |
||
unsigned int |
elem_exec_time |
element execution time (ms) |
rgb color values
Data Fields
type |
name |
description |
---|---|---|
uint8_t |
R |
Red byte. |
uint8_t |
G |
Green byte. |
uint8_t |
B |
Blue byte. |
uint8_t |
pad |
padding byte |
Data Fields
type |
name |
description |
---|---|---|
static_image |
Static Image element’s parameters. |
|
compose |
Compose element’s parameters - NOT IMPLEMENTED YET. |
|
struct |
labels |
Labeled Rectangle element’s parameters. |
convert |
Convert element’s parameters. |
|
struct |
resize |
Resize element’s parameters. |
color_conv |
Color convert element’s parameters. |
|
struct |
rotate |
Rotate element’s parameters. |
struct |
test |
Test element’s parameters. |
struct |
decode |
Decoder element’s parameters. |
ml_inference |
ML inference element’s parameters. |
Static Image element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
img_params |
static image parameters |
|
void ∗ |
img_buffer |
static image buffer address |
Compose element’s parameters - NOT IMPLEMENTED YET.
Data Fields
type |
name |
description |
---|---|---|
float |
||
float |
b |
Labeled Rectangle element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
uint32_t |
max_count |
maximum number of rectangles |
uint32_t |
detected_count |
detected rectangles |
rectangles |
array of rectangle data |
Convert element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
out_buf |
output buffer dimensions |
|
pixel_format |
new pixel format |
|
angle |
rotation angle |
|
flip |
flip mode |
|
crop |
input crop area |
|
out_window |
output window position |
|
scale |
scaling dimensions |
|
ops |
operation selector mask |
|
const char ∗ |
dev_name |
device name used for graphics |
bool |
stripe_in |
input stripe mode |
bool |
stripe_out |
output stripe mode |
Resize element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
unsigned int |
width |
|
unsigned int |
height |
Color convert element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
Rotate element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
angle |
Test element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
_Bool |
inp |
|
unsigned int |
width |
|
unsigned int |
height |
|
format |
Decoder element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
const char ∗ |
dev_name |
device name used for decoder |
unsigned int |
width |
|
unsigned int |
height |
|
out_format |
ML inference element’s parameters.
Data Fields
type |
name |
description |
---|---|---|
const void ∗ |
model_data |
pointer to model binary |
type |
inference type |
|
int |
model_size |
model binary size |
float |
model_input_mean |
model ‘mean’ of input values, used for normalization |
float |
model_input_std |
model ‘standard deviation’ of input values, used for normalization |
tensor_order |
model input tensor component order |
|
inference_params |
model specific parameters used by the inference |
2.1.2 Macro Definition Documentation
MPP_INFERENCE_MAX_OUTPUTS
#define MPP_INFERENCE_MAX_OUTPUTS
Maximum number of inference inputs and outputs. Maximum number of outputs supported by the pipeline
#define MPP_INFERENCE_MAX_INPUTS
Maximum number of inputs supported by the pipeline.
MPP_INVALID
#define MPP_INVALID Invalid pipeline handle.
MPP_EVENT_ALL
Bit mask to receive all Events.
MAX_TENSOR_DIMS
Maximum number of dimensions for tensors.
2.1.3 Typedef Documentation
mpp_t
typedef void∗ mpp_t Pipeline handle type.
mpp_elem_handle_t
typedef uintptr_t mpp_elem_handle_t Element handle type.
mpp_evt_mask_t
typedef unsigned int mpp_evt_mask_t Event mask for pipeline creation.
inference_entry_point_t
typedef int(∗ inference_entry_point_t) (uint8_t ∗, uint8_t ∗, uint8_t ∗) Bundle inference function type.
2.1.4 Enumeration Type Documentation
mpp_evt_t
enum mpp_evt_t
Pipeline generated events.
Enumerator
label |
description |
---|---|
invalid event |
|
inference out is ready |
|
INTERNAL: DO NOT USE. |
|
DO NOT USE. |
mpp_exec_flag_t
enum mpp_exec_flag_t
Execution parameters.
These parameters control the execution of the elements of an mpp.
The “mpps” created using the flag MPP_EXEC_RC are guaranteed to run up to the completion of all processing elements, while not being preempted by other “mpps”.
The “mpps” created using the flag MPP_EXEC_PREEMPT are preempted after a given time interval by “mpps” that will run-to-completion again.
The “mpps” created with the MPP_EXEC_INHERIT flag inherit the same execution flag as the parent(s) in case of split operation.
Note: It is not possible to request run-to-completion execution when spliting preemptable-execution “mpps”.
Enumerator
label |
description |
---|---|
inherit from parent(s) |
|
run-to-completion |
|
preemptable |
mpp_stats_grp_t
enum mpp_stats_grp_t
Enumerator
label |
description |
---|---|
API (global) stats. |
|
mpp_t stats |
|
element stats |
|
number of groups |
mpp_rotate_degree_t
enum mpp_rotate_degree_t
Rotation value.
Enumerator
label |
description |
---|---|
0 degree |
|
90 degrees |
|
180 degrees |
|
270 degrees |
mpp_flip_mode_t
enum mpp_flip_mode_t
Flip type.
Enumerator
label |
description |
---|---|
no flip |
|
horizontal flip |
|
vertical flip |
|
vertical and horizontal flip |
mpp_convert_ops_t
enum mpp_convert_ops_t
The convert operations selector flags.
Enumerator
label |
description |
---|---|
no frame conversion |
|
frame rotation and flip |
|
scaling from input_frame toward output window |
|
frame color conversion |
|
input frame crop |
|
output window |
mpp_pixel_format_t
enum mpp_pixel_format_t
Pixel format.
Enumerator
label |
description |
---|---|
ARGB 32 bits. |
|
BGRA 32 bits. |
|
RGBA 32 bits. |
|
BGRX 32 bits. |
|
RGBX 32 bits. |
|
RGB 24 bits. |
|
RGB 16 bits. |
|
BGR 24 bits. |
|
gray 3x8 bits |
|
gray 3x8 bits +8 unused bits |
|
gray 8 bits |
|
gray 16 bits |
|
YUVX interleaved 4:4:4. |
|
VYUY interleaved 4:2:2. |
|
UYVY interleaved 4:2:2. |
|
YUYV interleaved 4:2:2. |
|
depth 16 bits |
|
depth 8 bits |
|
YUV planar 4:2:0. |
|
JPEG. |
|
invalid pixel format |
mpp_element_id_t
enum mpp_element_id_t
Processing element ids.
Enumerator
label |
description |
---|---|
Invalid element. |
|
Image composition - NOT IMPLEMENTED YET. |
|
Labeled rectangle - bounding box. |
|
Test inplace element - NOT FOR USE. |
|
Inference engine. |
|
Image conversion: resolution, orientation, color format. |
|
Image decompression: JPEG, PNG. |
|
DO NOT USE. |
mpp_tensor_type_t
enum mpp_tensor_type_t
Inference tensor type.
Enumerator
label |
description |
---|---|
unsigned integer 8 bits |
|
signed integer 8 bits |
mpp_tensor_order_t
enum mpp_tensor_order_t
Inference input tensor order.
Enumerator
label |
description |
---|---|
order not set |
|
order: Batch, Height, Width, Channels |
|
order: Batch, Channels, Height, Width |
mpp_inference_type_t
enum mpp_inference_type_t
Inference type.
Enumerator
label |
description |
---|---|
TensorFlow-Lite. |
3. Return_codes
Macros
#define MPP_SUCCESS
#define MPP_ERROR
#define MPP_INVALID_ELEM
#define MPP_INVALID_PARAM
#define MPP_ERR_ALLOC_MUTEX
#define MPP_INVALID_MUTEX
#define MPP_MUTEX_TIMEOUT
#define MPP_MUTEX_ERROR
#define MPP_MALLOC_ERROR
3.1 Detailed Description
MPP APIs return status definitions.
3.1.1 Macro Definition Documentation
MPP_SUCCESS
Success return code.
MPP_ERROR
A generic error occured.
MPP_INVALID_ELEM
Invalid element provided.
MPP_INVALID_PARAM
Invalid parameter provided.
MPP_ERR_ALLOC_MUTEX
Error occured while allocating mutex.
MPP_INVALID_MUTEX
Invalid mutex provided.
MPP_MUTEX_TIMEOUT
Mutex timeout occured.
MPP_MUTEX_ERROR
Mutex error occured.
MPP_MALLOC_ERROR
Memory allocation error occured.