MCUXpresso SDK Documentation

Version: 25.12.00 25.12.00-pvw2 25.12.00-pvw1 25.09.00 25.09.00-pvw2 latest

Open on Repo Open an Issue

Source (.md/.rst) Page PDF (.pdf)

eIQ MPP Hardware Abstraction Layer API

MPP-HAL VERSION 3.6

Chapter 1

This is the documentation for the Hardware Abstraction Layer(HAL) API.

1.1 HAL overview

The hardware abstraction layer is used to abstract hardware and software components. With the usage of an HAL abstraction, the vision pipeline will be leveraging hardware accelerated components whenever possible.

1.1.1 MPP hal description

The HAL is presented with respect of the following points:

• A common header file “hal.h” includes all hardware top level functions.

• All hardware top level functions are using the prefix: “hal_ “.

• For each platform all hal_ functions defined in hal.h should be implemented at least with an empty function.

Here is an overview:

Figure 1.1 HAL overview

1.1.2 MPP HAL components

1. Source elements HAL

• Camera

• Static image

2. processing elements HAL

• Graphics driver

• Vision algorithms

• Labeled rectangle

3. Sink elements HAL

• Display

1.1.3 Supported devices

At present, the MPP HAL supports the following devices:

• Cameras:

  • OV5640

  • MT9M114

  • OV7670

  • Logitech C920 PRO HD WEBCAM

• Displays:

  • LVGL

  • RK055AHD091

  • RK055MHD091

  • RK043FN02H-CT

  • Mikroe TFT Proto 5(SSD1963 controller)

  • NXP’s LCD-PAR-S035 (ST7796S controller)

• Graphics:

  • PXP

  • CPU

  • GPU

1.1.4 Supported boards

Currently, the MPP HAL supports the following boards:

• (deprecated) evkmimxrt1170 is supported with the following devices:

  • Cameras: OV5640.

  • Displays: RK055AHD091 and RK055MHD091.

• (deprecated) evkbimxrt1050 is supported with the following devices:

  • Cameras: MT9M114.

  • Displays: RK043FN02H-CT.

• evkbmimxrt1170 is supported by porting the following devices:

  • Cameras: OV5640.

  • Displays: LVGL, RK055AHD091 and RK055MHD091.

• frdmmcxn947 is supported by porting the following devices:

  • Cameras: OV7670.

  • Displays: Mikroe TFT Proto 5” and NXP’s LCD-PAR-S035.

• mimxrt700evk is supported by porting the following devices:

  • Cameras: OV7670, Logitech C920 PRO HD WEBCAM

  • Displays: RK055AHD091 and RK055MHD091.

How to port new boards/devices: The MPP Hal provides the flexibility to the user to port new boards and devices(cameras and displays).

Supporting new boards: To support a new board a new file hal_{board_name} should be added under the ‘hal’ directory.

Supporting new devices: The hal components that can support new devices are:

• Cameras

• Display

• Graphics processing

A new device can simply be supported by:

• Providing the approriate hal_{device_module} implementation.

• Adding his name and setup entry point to the appropriate device list in the associated board hal_{board_name} file.

Enabling/Disabling Hal components and devices:

• The HAL components can be enabled/disabled from “mpp_config.h” using the compilation flags(HAL_ENABLE_{component_name}).

• The HAL devices can also be enabled/disabled from “mpp_config.h” using the compilation flags(HAL_ENABLE_{device_name}).

Chapter 2

2.1 HAL Types

Data Structures

• struct camera_dev_static_config_t

• struct camera_dev_private_capability_t

• struct camera_dev_t

• struct static_image_static_config_t

• struct static_image_t

• struct gfx_surface_t

• struct gfx_rotate_config_t

• struct gfx_dev_t

• struct hal_rect_t

• struct vdec_dev_t

• struct model_param_t

• struct valgo_dev_private_capability_t

• struct vision_frame_t

• struct vision_algo_dev_t

• struct display_dev_private_capability_t

• struct display_dev_t

• struct hw_buf_desc_t

• struct hal_img_decoder_setup_t

• struct hal_graphics_setup_t

• struct hal_display_setup_t

• struct hal_camera_setup_t

• struct checksum_data_t

Macros

• #define HAL_GFX_DEV_CPU_NAME

• #define GUI_PRINTF_BUF_SIZE

• #define GUI_PRINTF_BUF_SIZE

• #define HAL_VDEC_DEV_NAME

• #define MAX_INPUT_PORTS

• #define [MAX_INPUT_PORTS ]

• #define HAL_DEVICE_NAME_MAX_LENGTH

Typedefs

• camera_dev_callback_t)

• [vision_algo_private_data_t]

• mpp_callback_t)

• img_decoder_setup_func_t)

• graphic_setup_func_t)

• display_setup_func_t)

• camera_setup_func_t)

Enumerations

• enum hal_camera_status_t {

  kStatus_HAL_CameraSuccess,

  kStatus_HAL_CameraBusy,

  kStatus_HAL_CameraNoData,

  kStatus_HAL_CameraNonBlocking,

  kStatus_HAL_CameraError

}

• enum camera_event_t {

  kCameraEvent_SendFrame,

  kCameraEvent_CameraDeviceInit

}

• enum hal_image_status_t {

  MPP_kStatus_HAL_ImageSuccess,

  MPP_kStatus_HAL_ImageError

}

• enum gfx_rotate_target_t {

  kGFXRotateTarget_None,

  kGFXRotate_SRCSurface,

  kGFXRotate_DSTSurface

}

• enum hal_valgo_status_t {

  kStatus_HAL_ValgoSuccess,

  kStatus_HAL_ValgoMallocError,

  kStatus_HAL_ValgoInitError,

  kStatus_HAL_ValgoError,

  kStatus_HAL_ValgoStop

}

• enum display_event_t {

  kDisplayEvent_RequestFrame

}

• enum hal_display_status_t {

  kStatus_HAL_DisplaySuccess,

  kStatus_HAL_DisplayTxBusy,

  kStatus_HAL_DisplayNonBlocking,

  kStatus_HAL_DisplayError

}

• enum mpp_memory_policy_t {

  HAL_MEM_ALLOC_NONE,

  HAL_MEM_ALLOC_INPUT,

  HAL_MEM_ALLOC_OUTPUT,

  HAL_MEM_ALLOC_BOTH

}

• enum checksum_type_t {

  CHECKSUM_TYPE_PISANO,

  CHECKSUM_TYPE_CRC_ELCDIF

}

Functions

• int HAL_GfxDev_CPU_Register (gfx_dev_t ∗dev)

• int HAL_GfxDev_GPU_Register (gfx_dev_t ∗dev)

• int HAL_JPEG_CPU_Register (vdec_dev_t ∗dev)

• int HAL_JPEG_HW_Register (vdec_dev_t ∗dev)

• int setup_static_image_elt (static_image_t ∗elt)

• uint32_t calc_checksum (int size_b, void ∗pbuf)

2.1.1 Detailed Description

This section provides the detailed documentation for the MPP HAL types.

2.1.1.1 Data Structure Documentation

1. struct camera_dev_static_config_t

Structure that characterizes the camera device.

Data Fields

type	name	description
int	height	buffer height
int	width	buffer width
int	pitch	buffer pitch
int	left	left position
int	top	top position
int	right	right position
int	bottom	bottom position
mpp_rotate_degree_t	rotate	rotate degree
mpp_flip_mode_t	flip	flip
int	swapByte	swap byte per two bytes
mpp_pixel_format_t	format	pixel format
int	framerate	frame rate
int	stripe_size	stripe size in bytes
bool	stripe	stripe mode
uint32_t	n_streams	number of total output video streams
uint32_t	min_stream_req_cnt	minimum number of enqueue calls to wait for
uint32_t	crt_stream_req_cnt	number of streams requested for enqueue
mpp_exec_flag_t	req_cnt_type	flag to control stream request counting
mpp_camera_stream_cfg	stream[NUM_STREAMS]	stream configuration
bool	stream_requested[NUM_STREAMS]	flag to track if a stream is required for enqueue
bool	in_advance_enqueue	flag to indicate advance enqueue mode

2. struct camera_dev_private_capability_t

Camera device private capability.

Data Fields

type	name	description
camera_dev_callback_t	callback	callback
void ∗	param	parameter for the callback

3. struct _camera_dev

Camera devices can enqueue and dequeue frames as well as react to events from input devices via the “input← Notify” function. Camera devices can use any number of interfaces, including MIPI and CSI as long as the HAL driver implements the necessary functions found in camera_dev_operator_t. Examples of camera devices include the Orbbec U1S 3D SLM camera module and the OnSemi MT9M114 camera module.

Data Fields

int	id	unique id which is assigned by camera manager during registration
char	name[HAL_DEVICE_NAME_MAX_L	ENGTHname of] the device
const camera_dev_operator_t ∗	ops	operations
camera_dev_static_config_t	config	static configurations
camera_dev_private_capability_t	cap	private capability
void ∗	data	device private data

4. struct static_image_static_config_t

Structure that characterizes the image element.

Data Fields

type	name	description
int	height	buffer height
int	width	buffer width
int	left	left position
int	top	top position
int	right	right position
int	bottom	bottom position
mpp_pixel_format_t	format	pixel format
bool	stripe	stripe mode
int	compressed_size	compressed size in bytes

5. struct _static_image

Attributes of an image element.

Data Fields

type	name	description
int	id	unique id which is assigned by image manager
const static_image_operator_t ∗	ops	operations
static_image_static_config_t	config	static configs
int	stripe_idx	the current stripe index
uint8_t ∗	buffer	static image buffer

6. struct gfx_surface_t

Gfx surface parameters.

Data Fields

type	name	description
int	height	buffer height
int	width	buffer width
int	pitch	buffer pitch
int	left	left position
int	top	top position
int	right	right position
int	bottom	bottom position
int	swapByte	swap byte per two bytes
mpp_pixel_format_t	format	pixel format
void ∗	buf	buffer
void ∗	lock	the structure is determined by hal and set to null if not use in hal

7. struct gfx_rotate_config_t

gfx rotate configuration.

Data Fields

type	name	description
gfx_rotate_target_t	target
mpp_rotate_degree_t	degree

8. struct _gfx_dev

Data Fields

type	name	description
int	id
const gfx_dev_operator_t	ops
gfx_surface_t	src
gfx_surface_t	dst
mpp_callback_t	callback
void ∗	user_data

9. struct hal_rect_t

rectangle positions.

Data Fields

type	name	description
int	top
int	left
int	bottom
int	right

10. struct _vdec_dev

Data Fields

type	name	description
int	id
const vdec_dev_operator_t ∗	ops
mpp_callback_t	callback
void ∗	user_data

11. struct model_param_t

Structure passed to HAL as description of the binary model provided by user.

Data Fields

• const void ∗model_data

• int model_size

• float model_input_mean

• float model_input_std

• mpp_inference_params_t inference_params

• int height

• int width

• mpp_pixel_format_t format

• mpp_tensor_type_t inputType

• mpp_tensor_order_t tensor_order

• int(∗evt_callback_f )(mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data)

• void ∗cb_userdata

Field Documentation

model_data

const void∗ model_param_t::model_data pointer to model binary

model_size

int model_param_t::model_size model binary size

model_input_mean

float model_param_t::model_input_mean

model ‘mean’ of input values, used for normalization

model_input_std

float model_param_t::model_input_std

model ‘standard deviation’ of input values, used for normalization

inference_params

mpp_inference_params_t model_param_t::inference_params inference parameters

height

int model_param_t::height frame height

width

int model_param_t::width frame width

format

mpp_pixel_format_t model_param_t::format pixel format

inputType

mpp_tensor_type_t model_param_t::inputType input type

tensor_order

mpp_tensor_order_t model_param_t::tensor_order tensor order

evt_callback_f

int(∗ model_param_t::evt_callback_f) (mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data)

the callback to be called when model output is ready

cb_userdata

void∗ model_param_t::cb_userdata

pointer to user data, should be passed by callback

12. struct valgo_dev_private_capability_t

Valgo devices private capability.

Data Fields

type	name	description
void ∗	param	param for the callback

13. struct vision_frame_t

Characteristics that need to be defined by a vision algo.

Data Fields

type	name	description
int	height	frame height
int	width	frame width
int	pitch	frame pitch
mpp_pixel_format_t	format	pixel format
void ∗	input_buf	pixel input buffer

14. struct _vision_algo_dev

Attributes of a vision algo device.

Data Fields

type	name	description
int	id	unique id which is assigned by algorithm manager during the registration
char	name[HAL_DEVICE_NAME_MAX_L	ENGTHname to] identify
valgo_dev_private_capability_t	cap	private capability
const vision_algo_dev_operator_t ∗	ops	operations
vision_algo_private_data_t	priv_data	private data

15. struct _display_dev_private_capability

Structure that characterizes the display device.

Data Fields

type	name	description
int	height	buffer height
int	width	buffer width
int	pitch	buffer pitch
int	left	left position
int	top	top position
int	right	right position
int	bottom	bottom position
int	stripe_height	stripe height (0 if stripe mode is off)
bool	stripe	stripe mode
mpp_rotate_degree_t	rotate	rotate degree
mpp_pixel_format_t	format	pixel format
int	nbFrameBuffer	number of input buffers
void ∗∗	frameBuffers	array of pointers to frame buffer
mpp_callback_t	callback	callback
void ∗	user_data	parameter for the callback
void ∗	handle	Handle to the LVGL widget ‘image’.

16. struct _display_dev

Attributes of a display device. hal display device declaration.

Display devices can be used to display images, GUI overlays, etc. Examples of display devices include display panels like the RK024hh298 display, and external displays like UVC (video over USB).

Data Fields

type	name	description
int	id	unique id which is assigned by the display manager during the registration
char	name[HAL_DEVICE_NAME_MAX_L	ENGTHname of] the device
const display_dev_operator_t ∗	ops	operations
display_dev_private_capability_t	cap	private capability

17. struct hw_buf_desc_t

the hardware specific buffer requirements

Data Fields

type	name	description
int	stride	the number of bytes between 2 lines of image
int	nb_lines	the number of lines required (set to 0 if the element doesn’t require a specific number of lines)
int	alignment	alignment requirement in bytes
int	max_image_size	the number of bytes allocated
bool	cacheable	if true, HW will require cache maintenance
unsigned char ∗	addr	the aligned buffer address
unsigned char ∗	heap_p	pointer to the heap that should be freed

18. struct hal_img_decoder_setup_t!

Data Fields

type	name	description
const char ∗	vdec_dev_name
img_decoder_setup_func_t	decoder_setup_func

19. struct hal_graphics_setup_t

Data Fields

type	name	description
const char ∗	gfx_dev_name
graphic_setup_func_t	gfx_setup_func

20. struct hal_display_setup_t

Data Fields

type	name	description
const char ∗	display_name
display_setup_func_t	display_setup_func

21. struct hal_camera_setup_t

Data Fields

type	name	description
const char ∗	camera_name
camera_setup_func_t	camera_setup_func

22. struct checksum_data_t

computed checksum

Data Fields

type	name	description
checksum_type_t	type	checksum calculation method
uint32_t	value	checksum value

2.1.1.2 Macro Definition Documentation

1. HAL_GFX_DEV_CPU_NAME

#define HAL_GFX_DEV_CPU_NAME hal graphics (gfx) device declaration.

Graphics processing devices can be used to perform conversion from one image format to another, resize images and compose images on top of one another. Examples of graphics devices include the PXP (pixel pipeline) found on many i.MXRT series MCUs. Name of the graphic device using CPU operations

2. GUI_PRINTF_BUF_SIZE [1/2]

#define GUI_PRINTF_BUF_SIZE Local text buffer size.

3. GUI_PRINTF_BUF_SIZE [2/2]

#define GUI_PRINTF_BUF_SIZE Local text buffer size.

4. HAL_VDEC_DEV_NAME

#define HAL_VDEC_DEV_NAME

hal video decoder (vdec) device declaration.

Video decoder devices can be used to perform decompression of image. Examples of decoder devices include the PNG/JPEG HW or SW found on many i.MXRT series MCUs. Name of the jpeg decoder device using CPU operations

5. MAX_INPUT_PORTS

#define MAX_INPUT_PORTS

HAL public types header.

maximum number of element inputs/outputs

6. HAL_DEVICE_NAME_MAX_LENGTH

#define HAL_DEVICE_NAME_MAX_LENGTH maximum length of device name

2.1.1.3 Typedef Documentation

1. camera_dev_callback_t

typedef int(∗ camera_dev_callback_t) (const camera_dev_t ∗dev, camera_event_t event, void ∗param, uint8_t fromISR)

Callback function to notify camera manager that one frame is dequeued.

Parameters

name	description
dev	Device structure of the camera device calling this function
event	id of the event that took place
param	Parameters
fromISR	True if this operation takes place in an irq, 0 otherwise

Returns

0 if the operation was successfully

2. mpp_callback_t

typedef int(∗ mpp_callback_t) (mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data) The mpp callback function prototype.

3. img_decoder_setup_func_t

typedef int(∗ img_decoder_setup_func_t) (vdec_dev_t ∗) video decoder setup

4. graphic_setup_func_t

typedef int(∗ graphic_setup_func_t) (gfx_dev_t ∗) graphics setup

5. display_setup_func_t

typedef int(∗ display_setup_func_t) (display_dev_t ∗) display setup

6. camera_setup_func_t

typedef int(∗ camera_setup_func_t) (const char ∗, camera_dev_t ∗) camera setup

2.1.1.4 Enumeration Type Documentation

1. hal_camera_status_t

enum hal_camera_status_t Camera return status.

Enumerator

label	description
kStatus_HAL_CameraSuccess	HAL camera successful.
kStatus_HAL_CameraBusy	Camera is busy.
kStatus_HAL_CameraNoData	No data available from camera.
kStatus_HAL_CameraNonBlocking	Camera will return immediately.
kStatus_HAL_CameraError	Error occurs on HAL Camera.

2. camera_event_t

enum camera_event_t

Type of events that are supported by calling the callback function.

Enumerator

label	description
kCameraEvent_SendFrame	Camera new frame is available.
kCameraEvent_CameraDeviceInit	Camera device finished the initialization process.

3. hal_image_status_t

enum hal_image_status_t static image return status

Enumerator

label	description
MPP_kStatus_HAL_ImageSuccess	Successfully.
MPP_kStatus_HAL_ImageError	Error occurs on HAL Image.

4. gfx_rotate_target_t

enum gfx_rotate_target_t gfx rotate target

5. hal_valgo_status_t

enum hal_valgo_status_t

Valgo Error codes for hal operations.

Enumerator

label	description
kStatus_HAL_ValgoSuccess	Successfully.
kStatus_HAL_ValgoMallocError	memory allocation failed for HAL algorithm
kStatus_HAL_ValgoInitError	algorithm initialization error
kStatus_HAL_ValgoError	Error occurs in HAL algorithm.
kStatus_HAL_ValgoStop	HAL algorithm stop.

6. display_event_t

enum display_event_t

Type of events that are supported by calling the callback function.

Enumerator

label	description
kDisplayEvent_RequestFrame	Display finished sending the frame asynchronously, provide another frame.

7. hal_display_status_t

enum hal_display_status_t Error codes for display hal devices.

Enumerator

label	description
kStatus_HAL_DisplaySuccess	HAL display successful.
kStatus_HAL_DisplayTxBusy	Display tx is busy.
kStatus_HAL_DisplayNonBlocking	Display will return immediately.
kStatus_HAL_DisplayError	Error occurs on HAL Display.

8. mpp_memory_policy_t

enum mpp_memory_policy_t

The memory allocation policy of an element’s hal.

During the pipeline construction, the HAL uses this enum to tell the pipeline if it already owns input/ouput buffers. Before the pipeline starts, the memory manager will map the existing buffers to elements and allocate missing buffers from the heap.

Enumerator

label	description
HAL_MEM_ALLOC_NONE	element requires buffers to be provided by other elements, or by the pipeline
HAL_MEM_ALLOC_INPUT	element allocates its input buffer, it may require output buffers to be provided by other elements, or by the pipeline
HAL_MEM_ALLOC_OUTPUT	element allocates its output buffer, it may require input buffers to be provided by other elements, or by the pipeline
HAL_MEM_ALLOC_BOTH	element allocates both its input and output buffers

9. checksum_type_t

enum checksum_type_t checksum calculation method

Enumerator

label	description
CHECKSUM_TYPE_PISANO	checksum computed using Pisano
CHECKSUM_TYPE_CRC_ELCDIF	checksum computed CRC from ELCDIF

2.1.1.5 Function Documentation

1. HAL_GfxDev_CPU_Register()

int HAL_GfxDev_CPU_Register ( gfx_dev_t ∗ dev )

Register the graphic device with the CPU operations.

Parameters

in/out	name	description
in	dev	graphic device to register

Returns

error code (0: success, otherwise: failure)

2. HAL_GfxDev_GPU_Register()

int HAL_GfxDev_GPU_Register ( gfx_dev_t ∗ dev )

Register the graphic device with the GPU operations.

Parameters

in/out	name	description
in	dev	graphic device to register

Returns

error code (0: success, otherwise: failure)

3. HAL_JPEG_CPU_Register()

int HAL_JPEG_CPU_Register (

vdec_dev_t ∗ dev ) Register the jpeg SW decoder device.

Parameters

in/out	name	description
in	dev	decoder device to register
-	-	-

Returns

error code (0: success, otherwise: failure)

4. HAL_JPEG_HW_Register()

int HAL_JPEG_HW_Register (

vdec_dev_t ∗ dev ) Register the jpeg HW decoder device.

Parameters

in/out	name	description
in	dev	decoder device to register
-	-	-

Returns

error code (0: success, otherwise: failure)

2.2 HAL OPERATIONS

Data Structures

• struct camera_dev_operator_t

• struct static_image_operator_t

• struct gfx_dev_operator_t

• struct vdec_dev_operator_t

• struct vision_algo_dev_operator_t

• struct display_dev_operator_t

Typedefs

• typedef int(∗mpp_callback_t) (mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data)

• typedef int(∗mpp_callback_t) (mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data)

Functions

• void GUI_DrawText (uint16_t ∗lcd_buf, uint16_t fcolor, uint16_t bcolor, uint32_t width, int x, int y, const char ∗label)

• static void hal_draw_pixel565 (uint16_t ∗pDst, uint32_t x, uint32_t y, uint16_t color, uint32_t lcd_w)

• static void hal_draw_text565 (uint16_t ∗lcd_buf, uint16_t fcolor, uint16_t bcolor, uint32_t width, int x, int y, const char ∗label, int stripe_top, int stripe_bottom)

• static void hal_draw_rect565 (uint16_t ∗lcd_buf, hal_rect_t rect, mpp_color_t rgb, uint32_t width, int stripe← _top, int stripe_bottom)

• static int get_bitpp (mpp_pixel_format_t type)

• void swap_2_bytes (uint8_t ∗data, int size)

2.2.1 Detailed Description

This section provides the detailed documentation for the MPP HAL operations that needs to be implemented for each component.

2.2.1.1 Data Structure Documentation

1. struct camera_dev_operator_t

Operation that needs to be implemented by a camera device.

Data Fields

• hal_camera_status_t(∗init )(camera_dev_t ∗dev, mpp_camera_params_t ∗config, camera_dev_callback_t callback, void ∗param)

• hal_camera_status_t(∗deinit )(camera_dev_t ∗dev)

• hal_camera_status_t(∗start )(const camera_dev_t ∗dev)

• hal_camera_status_t(∗stop )(const camera_dev_t ∗dev)

• hal_camera_status_t(∗enqueue )(const camera_dev_t ∗dev, void ∗data)

• hal_camera_status_t(∗dequeue )(const camera_dev_t ∗dev, void ∗∗data, int ∗stripe, int ∗compressed_size)

• hal_camera_status_t(∗get_buf_desc )(const camera_dev_t ∗dev, hw_buf_desc_t ∗out_buf, mpp_memory_policy_t ∗policy)

• hal_camera_status_t(∗lock )(const camera_dev_t ∗dev)

• hal_camera_status_t(∗unlock )(const camera_dev_t ∗dev)

Field Documentation <a name=””_page28_x212.37_y68.87”>

init

hal_camera_status_t(∗ camera_dev_operator_t::init) (camera_dev_t ∗dev, mpp_camera_params_t ∗config, camera_dev_callback_t callback, void ∗param)

initialize the dev

deinit

hal_camera_status_t(∗ camera_dev_operator_t::deinit) (camera_dev_t ∗dev)

deinitialize the dev

start

hal_camera_status_t(∗ camera_dev_operator_t::start) (const camera_dev_t ∗dev)

start the dev

stop

hal_camera_status_t(∗ camera_dev_operator_t::stop) (const camera_dev_t ∗dev)

stop the dev

enqueue

hal_camera_status_t(∗ camera_dev_operator_t::enqueue) (const camera_dev_t ∗dev, void ∗data) enqueue a buffer to the dev

dequeue

hal_camera_status_t(∗ camera_dev_operator_t::dequeue) (const camera_dev_t ∗dev, void ∗∗data, int ∗stripe)

dequeue a buffer from the dev (blocking)

get_buf_desc

hal_camera_status_t(∗ camera_dev_operator_t::get_buf_desc) (const camera_dev_t ∗dev, hw_buf_desc_t ∗out_buf, mpp_memory_policy_t ∗policy)

get buffer descriptors and policy

lock

hal_camera_status_t(∗ camera_dev_operator_t::lock) (const camera_dev_t ∗dev) lock the device for exclusive access and operations

unlock

hal_camera_status_t(∗ camera_dev_operator_t::unlock) (const camera_dev_← t ∗dev)

unlock the device after exclusive operations

2. struct static_image_operator_t

Operation that needs to be implemented by an image element.

Data Fields

• hal_image_status_t(∗init )(static_image_t ∗elt, mpp_img_params_t ∗config, void ∗param)

• hal_image_status_t(∗dequeue )(static_image_t ∗elt, hw_buf_desc_t ∗out_buf, int ∗stripe_num)

Field Documentation

init

hal_image_status_t(∗ static_image_operator_t::init) (static_image_t ∗elt, mpp← _img_params_t ∗config, void ∗param)

initialize the elt

dequeue

hal_image_status_t(∗ static_image_operator_t::dequeue) (static_image_← t ∗elt, hw_buf_desc_t ∗out_buf, int ∗stripe_num)

dequeue a buffer from the elt

3. struct gfx_dev_operator_t

Operation that needs to be implemented by gfx device.

Data Fields

• int(∗init )(gfx_dev_t ∗dev, void ∗param)

• int(∗deinit )(gfx_dev_t ∗dev)

• int(∗ get_buf_desc )(const gfx_dev_t ∗dev, hw_buf_desc_t ∗in_buf, hw_buf_desc_t ∗out_buf, mpp_memory_policy_t ∗policy)

• int(∗ blit )(const gfx_dev_t ∗dev, const gfx_surface_t ∗pSrc, const gfx_surface_t ∗pDst, const gfx_rotate_config_t ∗pRotate, mpp_flip_mode_t flip)

• int(∗drawRect )(const gfx_dev_t ∗dev, gfx_surface_t ∗pOverlay, int x, int y, int w, int h, int color)

• int(∗drawPicture )(const gfx_dev_t ∗dev, gfx_surface_t ∗pOverlay, int x, int y, int w, int h, int alpha, const char ∗pIcon)

• int(∗drawText )(const gfx_dev_t ∗dev, gfx_surface_t ∗pOverlay, int x, int y, int textColor, int bgColor, int type, const char ∗pText)

• int(∗compose )(const gfx_dev_t ∗dev, gfx_surface_t ∗pSrc, gfx_surface_t ∗pOverlay, gfx_surface_t ∗pDst, gfx_rotate_config_t ∗pRotate, mpp_flip_mode_t flip)

• int(∗finish )(gfx_dev_t ∗dev)

4. struct vdec_dev_operator_t

Operation that needs to be implemented by vdec device.

Data Fields

• int(∗init )(vdec_dev_t ∗dev, void ∗param)

• int(∗deinit )(const vdec_dev_t ∗dev)

• int(∗ get_buf_desc )(const vdec_dev_t ∗dev, hw_buf_desc_t ∗in_buf, hw_buf_desc_t ∗out_buf, mpp_memory_policy_t ∗policy)

• int(∗decode )(const vdec_dev_t ∗dev, uint8_t ∗pSrc, uint8_t ∗pDst, int32_t jpg_size, uint32_t row_stride)

5. struct vision_algo_dev_operator_t

Operation that needs to be implemented by a vision algorithm device.

Data Fields

• hal_valgo_status_t(∗init )(vision_algo_dev_t ∗dev, model_param_t ∗param)

• hal_valgo_status_t(∗deinit )(vision_algo_dev_t ∗dev)

• hal_valgo_status_t(∗run )(const vision_algo_dev_t ∗dev, void ∗data)

• hal_valgo_status_t(∗get_buf_desc )(const vision_algo_dev_t ∗dev, hw_buf_desc_t ∗in_buf, mpp_memory_policy_t ∗policy)

Field Documentation

init

hal_valgo_status_t(∗ vision_algo_dev_operator_t::init) (vision_algo_dev_t ∗dev, model_param_t ∗param)

initialize the dev

deinit

hal_valgo_status_t(∗ vision_algo_dev_operator_t::deinit) (vision_algo_dev_t ∗dev)

deinitialize the dev

run

hal_valgo_status_t(∗ vision_algo_dev_operator_t::run) (const vision_algo_dev_t ∗dev, void ∗data)

start the dev

get_buf_desc

hal_valgo_status_t(∗ vision_algo_dev_operator_t::get_buf_desc) (const vision_algo_dev_t ∗dev, hw_buf_desc_t ∗in_buf, mpp_memory_policy_t ∗policy)

read input parameters

6. struct display_dev_operator_t

Operation that needs to be implemented by a display device.

Data Fields

• hal_display_status_t(∗init)(display_dev_t ∗dev, mpp_display_params_t ∗config, mpp_callback_t callback, void ∗user_data)

• hal_display_status_t(∗deinit )(const display_dev_t ∗dev)

• hal_display_status_t(∗start )(display_dev_t ∗dev)

• hal_display_status_t(∗stop )(display_dev_t ∗dev)

• hal_display_status_t(∗blit )(const display_dev_t ∗dev, void ∗frame, int stripe)

• hal_display_status_t(∗get_buf_desc )(const display_dev_t ∗dev, hw_buf_desc_t ∗in_buf, mpp_memory_policy_t ∗policy)

Field Documentation

init

hal_display_status_t(∗ display_dev_operator_t::init) (display_dev_t ∗dev, mpp← _display_params_t ∗config, mpp_callback_t callback, void ∗user_data)

initialize the dev

deinit

hal_display_status_t(∗ display_dev_operator_t::deinit) (const display_dev_t ∗dev)

deinitialize the dev

start

hal_display_status_t(∗ display_dev_operator_t::start) (display_dev_t ∗dev) start the dev

stop

hal_display_status_t(∗ display_dev_operator_t::stop) (display_dev_t ∗dev) stop the dev

blit

hal_display_status_t(∗ display_dev_operator_t::blit) (const display_dev_t ∗dev, void ∗frame, int stripe)

blit a buffer to the dev

get_buf_desc

hal_display_status_t(∗ display_dev_operator_t::get_buf_desc) (const display_dev_t ∗dev, hw_buf_desc_t ∗in_buf, mpp_memory_policy_t ∗policy)

get buffer descriptors and policy

2.2.1.2 Typedef Documentation

1. mpp_callback_t

typedef int(∗ mpp_callback_t) (mpp_t mpp, mpp_evt_t evt, void ∗evt_data, void ∗user_data) The mpp callback function prototype.

2.2.1.3 Function Documentation

1. GUI_DrawText()

void GUI_DrawText ( uint16_t ∗ lcd_buf, uint16_t fcolor, uint16_t bcolor, uint32_t width, int x, int y, const char ∗ label )

Draws text stored in label pointer to LCD buffer. This function copy content of data from label text buffer to the LCD.

Parameters

name	description
lcd_buf	LCD buffer address destination for drawing text
fcolor	foreground color in rgb565 format
bcolor	background color in rgb565 format
width	LCD width
x	drawing position on X axe
y	drawing position on Y axe
label	C string pointed by label

Returns

2. hal_draw_pixel565()

static void hal_draw_pixel565 ( uint16_t ∗ pDst, uint32_t x, uint32_t y, uint16_t color, uint32_t lcd_w )

Draws pixel with RGB565 color to defined point.

Parameters

name	description
pDst	image data address of destination buffer
x	drawing position on X axe
y	drawing position on Y axe
color	RGB565 encoded value
lcd_w	lcd width

3. hal_draw_text565()

static void hal_draw_text565 ( uint16_t ∗ lcd_buf, uint16_t fcolor, uint16_t bcolor, uint32_t width, int x, int y, const char ∗ label, int stripe_top, int stripe_bottom ) [static]

Draws text stored in label pointer to LCD buffer. This function copy content of data from label text buffer to the LCD.

Parameters

name	description
lcd_buf	LCD buffer address destination for drawing text
fcolor	foreground color in rgb565 format
bcolor	background color in rgb565 format
width	LCD width
x	drawing position on X axe
y	drawing position on Y axe
format	C string pointed by format

Returns

The return number of written chars to the buffer

4. hal_draw_rect565()

static void hal_draw_rect565 ( uint16_t ∗ lcd_buf, hal_rect_t rect, mpp_color_t rgb, uint32_t width, int stripe_top, int stripe_bottom ) [static]

Draws rectangle.

Parameters

in/out	name	description
in	lcd_buf	LCD buffer address destination for drawing rectangle
in	color	background color in rgb565 format
in	x	drawing position on X axe
in	y	drawing position on Y axe
in	xsize	rectangle width
in	ysize	rectangle height
in	r	0-255 red color value
in	g	0-255 green color value
in	b	0-255 blue color value
in	width	LCD width

Returns

N/A

5. **get_bitpp()**static int get_bitpp (

mpp_pixel_format_t type ) [static]

returns the number of bits per pixel per format, unknown format return 0

6. **swap_2_bytes()**void swap_2_bytes (

uint8_t ∗ data,

int size )

Swaps a buffer’s MSB and LSB bytes..

Parameters

name	description
data	pointer to the buffer to be converted(from little endian to big endian and vice-versa).
size	buffer size.

3. HAL Types

Data Structures

• struct virtual_usb_cam_config_msg_t

• struct virtual_usb_cam_req_msg_t

• struct virtual_usb_cam_rsp_msg_t

• struct virtual_usb_cam_msg_t

Macros

• #define TARGET_CAMERA0_WIDTH

• #define TARGET_CAMERA0_HEIGHT

• #define TARGET_CAMERA1_WIDTH

• #define TARGET_CAMERA1_HEIGHT

• #define TARGET_CAMERA0_RESOLUTION

• #define TARGET_CAMERA1_RESOLUTION

• #define TARGET_CAMERA_MAX_RESOLUTION

• #define TARGET_CAMERA_FPS

• #define CAMERA_DEV_BUFFER_ALIGN

• #define CORE1_EPT_ADDRESS

• #define MPP_EPT_ADDRESSS

• #define RTSP_EPT_ADDRESS

• #define APP_EP_READY_EVENT_DATA

Enumerations

• enum virtual_usb_cam_msg_type_e {

  VIRT_USB_CAM_NOMSG ,

  VIRT_USB_CAM_CONFIG,

  VIRT_USB_CAM_CONFIG_ACK, VIRT_USB_CAM_CONFIG_ERR ,

  VIRT_USB_CAM_REQRGB ,

  VIRT_USB_CAM_REQIR,

  VIRT_USB_CAM_REQRGBIR ,

  VIRT_USB_CAM_RSPRGB ,

  VIRT_USB_CAM_RSPIR ,

  VIRT_USB_CAM_RSPRGBIR ,

  VIRT_USB_CAM_ERROR }

• enum virtual_usb_cam_user_id_e {

  RTSP_USER_ID ,

  MPP_USER_ID }

• enum virtual_usb_cam_col_format_e { VIRT_USB_CAM_JPEG }

1. Detailed Description

This section provides the detailed documentation for the MPP HAL VIRTUAL CAMERA types.

2. Data Structure Documentation

3. struct virtual_usb_cam_config_msg_t

Structure that characterizes the payload of the camera config message sent from core 0 to core 1. Data Fields

uint32_t	camera_width	Width of the camera output in pixels.
uint32_t	camera_height	Height of the camera output in pixels.
virtual_usb_cam_col_format_e	color_format	Color format for the camera output (e.g., JPEG)
uint32_t	fps	Frames per second for camera capture rate.

2. struct virtual_usb_cam_req_msg_t

Structure that characterizes the payload of the camera request message containing frame buffer addresses for RGB and IR data.

Data Fields

uint32_t	rgb_frame_addr	Physical address of the RGB frame buffer.
uint32_t	rgb_max_frame_size	Maximum size allocated for RGB frame buffer.
uint32_t	ir_frame_addr	Physical address of the IR frame buffer.
uint32_t	ir_max_frame_size	Maximum size allocated for IR frame buffer.

3. struct virtual_usb_cam_rsp_msg_t

Structure that characterizes the payload of the camera request message containing frame addresses and sizes for RGB and IR data.

Data Fields

uint32_t	rgb_frame_addr	Physical address of the RGB frame buffer (looped back by the core 1 camera app)
uint32_t	rgb_frame_size	Size in bytes of the RGB frame data.
uint32_t	ir_frame_addr	Physical address of the IR frame buffer (looped back by the core 1 camera app)
uint32_t	ir_frame_size	Size in bytes of the IR frame data.

4. struct virtual_usb_cam_msg_t

Structure that characterizes the messages sent between cores. Data Fields

virtual_usb_cam_msg_type_e	msg_type	Type of message being sent (config, request, response, etc.)
virtual_usb_cam_user_id_e	user_id	Identifier for the user/component sending the message (RTSP or MPP)
union msg_payload_u	msg_payload	Union containing the actual message data based on msg_type.

3. Macro Definition Documentation

4. TARGET_CAMERA0_WIDTH

#define TARGET_CAMERA0_WIDTH

Buffer alignment requirement for camera device buffers in bytes.

2. CORE1_EPT_ADDRESS

#define CORE1_EPT_ADDRESS

Endpoint address for Core 1 inter-core communication channel.

3. MPP_EPT_ADDRESSS

#define MPP_EPT_ADDRESSS

Endpoint address for MPP (Media Processing Pipeline) inter-core communication channel. MPP might use a range of endpoints starting with 40 and up to 49 included

4. RTSP_EPT_ADDRESS

#define RTSP_EPT_ADDRESS

Endpoint address for RTSP (Real Time Streaming Protocol) inter-core communication channel.

5. APP_EP_READY_EVENT_DATA

#define APP_EP_READY_EVENT_DATA

Event data value indicating that the application endpoint is ready for communication.

4. Enumeration Type Documentation

5. virtual_usb_cam_msg_type_e

enum virtual_usb_cam_msg_type_e

Structure that characterizes the exchanged message types between core 0 and core 1.

2. virtual_usb_cam_col_format_e

enum virtual_usb_cam_col_format_e

Structure that characterizes the color format for the camera output.

2.3 HAL Setup Functions

Functions

• int hal_label_rectangle (uint8_t ∗frame, int width, int height, mpp_pixel_format_t format, mpp_labeled_rect_t ∗lr, int stripe, int stripe_max)

• int hal_landmark (uint8_t ∗frame, int width, int height, mpp_pixel_format_t format, mpp_landmark_t ∗lk, int stripe, int stripe_max)

• int hal_inference_tflite_setup (vision_algo_dev_t ∗dev)

• int hal_display_setup (const char ∗name, display_dev_t ∗dev)

• int hal_camera_setup (const char ∗name, camera_dev_t ∗dev)

• int hal_gfx_setup (const char ∗name, gfx_dev_t ∗dev)

• int hal_img_decoder_setup (const char ∗name, vdec_dev_t ∗dev)

2.3.1 Detailed Description

This section provides the detailed documentation for the HAL setup functions that should be defined by each device.

2.3.1.1 Function Documentation

1. hal_label_rectangle()

int hal_label_rectangle ( uint8_t ∗ frame, int width, int height, mpp_pixel_format_t format, mpp_labeled_rect_t ∗ lr, int stripe, int stripe_max )

Implementation of hal labeled rectangle component that draws a rectangle and a text on an input image.

Parameters

in/out	name	description
in	frame	The buffer address
in	width	Image width
in	height	Image height
in	format	Image format
in	lr	Labeled rectangle parameters
in	stripe	stripe number (0=no stripe)
in	stripe_max	max nb of stripes

Returns

0

2. **hal_landmark()**int hal_landmark (

uint8_t ∗ frame,

int width,

int height,

mpp_pixel_format_t format,

mpp_landmark_t ∗ lk,

int stripe,

int stripe_max )

Implementation of hal landmark component that draws a landmark on an input image. Parameters

in/out	name	description
in	frame	The buffer address
in	width	Image width
in	height	Image height
in	format	Image format
in	lk	landmark parameters
in	stripe	stripe number (0=no stripe)
in	stripe_max	max nb of stripes

Returns

0

3. hal_inference_tflite_setup()

int hal_inference_tflite_setup ( vision_algo_dev_t ∗ dev )

Hal setup function for inference engine Tensorflow-Lite Micro.

Parameters

in/out	name	description
in	dev	vision algo device to register

Returns

error code (0: success, otherwise: failure)

4. hal_display_setup()

int hal_display_setup ( const char ∗ name, display_dev_t ∗ dev )

Register with a display device specified by name. If name is NULL, return error.

Parameters

in/out	name	description
in	name	display name
in	dev	display device to register

Returns

error code (0: success, otherwise: failure)

5. hal_camera_setup()

int hal_camera_setup ( const char ∗ name, camera_dev_t ∗ dev )

Register with a camera device specified by name. If name is NULL, return error.

Parameters

in/out	name	description
in	name	camera name
in	dev	camera device to register

Returns

error code (0: success, otherwise: failure)

6. hal_gfx_setup()

int hal_gfx_setup ( const char ∗ name, gfx_dev_t ∗ dev )

Register with a graphic processing device specified by name.

If name is NULL, the first available graphic processing supported by Hw will be selected. The graphic device using CPU operations will be selected if name is not specified and if no graphic processing is available for the Hw.

Parameters

in/out	name	description
in	name	graphic processing device performing the image conversion
in	dev	graphic device to register

Returns

error code (0: success, otherwise: failure)

7. hal_img_decoder_setup()

int hal_img_decoder_setup ( const char ∗ name, vdec_dev_t ∗ dev )

Register with an image decoder device specified by name.

If name is NULL, the first available decoder supported by Hw will be selected. The decoder device using CPU operations will be selected if name is not specified and if no decoder is available for the Hw.

Parameters

in/out	name	description
in	name	image decoding device name
in	dev	decoder device to register

Returns

error code (0: success, otherwise: failure)