Overview - ExecuTorch CifarNet example project#
This project runs CifarNet model using ExecuTorch inference engine on the board. The example model is pre-compiled by a script in ExecuTorch examples/nxp. If you want to learn how to compile your own model, refer to the NXP-backend documentation.
What is ExecuTorch#
ExecuTorch is the PyTorch ecosystem on-device inference engine, ported for the selected NXP MCU platforms, such as RT700.
ExecuTorch is an end-to-end solution for enabling on-device inference capabilities across mobile and edge devices including wearables, embedded devices and microcontrollers. It is part of the PyTorch Edge ecosystem and enables efficient deployment of PyTorch models to edge devices. For more information, see ExecuTorch official page.
Key value propositions of ExecuTorch are:
Portability: Compatibility with a wide variety of computing platforms, from high-end mobile phones to highly constrained embedded systems and microcontrollers.
Productivity: Enabling developers to use the same toolchains and Developer Tools from PyTorch model authoring and conversion, to debugging and deployment to a wide variety of platforms.
Performance: Providing end users with a seamless and high-performance experience due to a lightweight runtime and utilizing full hardware capabilities such as CPUs, NPUs, and DSPs.
ExecuTorch workflow#
The ExecuTorch workflow works in three steps:
Exporting your PyTorch model
Compiling the exported program to an ExecuTorch program
Running the compiled program on the target device using ExecuTorch runtime library
Steps 1 and 2 are being done ahead of time (usually in Python along with your PyTorch model implementation).
To compile the exported program you can use executorch package available via pip install.
For more detailed and up-to-date instruction on how to compile your models, visit the
getting started tutorial page.
The 3rd step is performed on the target device using ExecuTorch runtime. To learn more about the ExecuTorch workflow, refer to the workflow documentation.
Supported Boards#
MIMXRT700-EVK
Hardware requirements
Mini/micro USB cable
MIMXRT700-EVK board
Personal computer
Board settings
Example running at 325MHz which need PMIC power supply.
PMIC is required to drive 1.1V on VDD2, then JP1 and JP3 on the RT700 EVK need to be shorted for proper operation
Prepare the Demo
Connect a USB cable between the host PC and the OpenSDA USB port on the target board.
Open a serial terminal with the following settings:
115200 baud rate
8 data bits
No parity
One stop bit
No flow control
Download the program to the target board.
Either press the reset button on your board or launch the debugger in your IDE to begin running the demo.
Run the Demo
The log below shows the output of the demo in the terminal window (compiled with ARM GCC):
CIFARNET example using a ExecuTorch model Model buffer loaded, has 11 methods Running method forward Setting up planned buffer 0, size 3088. Method loaded. Preparing inputs… Input prepared. Starting the model execution… Model executed successfully. Core/NPU Frequency: 324 MHz method_allocator Addr: 0x20000000 - 0x20080000 method_allocator_used: Total 0x80000 (524288 B); Used 0xdb9 (3513 B); Used/Total 0 % temp_allocator Addr: 0x20080000 - 0x20100000 temp_allocator_used: Total 0x80000 (524288 B); Used 0x3f27e4 (4138980 B); Used/Total 789 % Model Addr: 0x20430000 - 0x20448d60 Model Size: 0x18d60 (101728 B) Total Size Used: 4244221 B (Model (101728 B) + method_allocator (3513 B) + temp_allocator (4138980 B))
1 outputs:
Inference time: 1072 us
Detected: cat (99%)
Output[0][0]: -128 Output[0][1]: -128 Output[0][2]: -128 Output[0][3]: 127 Output[0][4]: -128 Output[0][5]: -128 Output[0][6]: -128 Output[0][7]: -128 Output[0][8]: -128 Output[0][9]: -128 Program complete, exiting.
FRDM-IMXRT700
Hardware requirements
Type-C USB cable
FRDMIMXRT700 board
Personal computer
Board settings
Example running at 325MHz which need PMIC power supply.
PMIC is required to drive 1.1V on VDD2, then JP1 and JP3 on the RT700 EVK need to be shorted for proper operation
Prepare the Demo
Connect a USB cable between the host PC and the OpenSDA USB port on the target board.
Open a serial terminal with the following settings:
115200 baud rate
8 data bits
No parity
One stop bit
No flow control
Download the program to the target board.
Either press the reset button on your board or launch the debugger in your IDE to begin running the demo.
Run the Demo
The log below shows the output of the demo in the terminal window (compiled with ARM GCC):
CIFARNET example using a ExecuTorch model Model buffer loaded, has 11 methods Running method forward Setting up planned buffer 0, size 3088. Method loaded. Preparing inputs… Input prepared. Starting the model execution… Model executed successfully. Core/NPU Frequency: 324 MHz method_allocator Addr: 0x20000000 - 0x20080000 method_allocator_used: Total 0x80000 (524288 B); Used 0xdb9 (3513 B); Used/Total 0 % temp_allocator Addr: 0x20080000 - 0x20100000 temp_allocator_used: Total 0x80000 (524288 B); Used 0x3f27e4 (4138980 B); Used/Total 789 % Model Addr: 0x20430000 - 0x20448d60 Model Size: 0x18d60 (101728 B) Total Size Used: 4244221 B (Model (101728 B) + method_allocator (3513 B) + temp_allocator (4138980 B))
1 outputs:
Inference time: 1072 us
Detected: cat (99%)
Output[0][0]: -128 Output[0][1]: -128 Output[0][2]: -128 Output[0][3]: 127 Output[0][4]: -128 Output[0][5]: -128 Output[0][6]: -128 Output[0][7]: -128 Output[0][8]: -128 Output[0][9]: -128 Program complete, exiting.