This chapter explains how to integrate a pretrained, quantized ML model into an Edge AI MCU based on Cortex-M and Ethos-U. Model training, quantization, and functional accuracy validation are outside the scope of this integration flow.
The starting point is a pretrained, quantized ML model that meets the application's functional requirements. Before selecting a specific Edge AI MCU, compile the model with Vela for one or more Ethos-U reference systems as described in Compare Ethos-U configurations.
Vela reports operator placement and estimates memory use, bandwidth, and NPU cycles. See Read the Vela reports. Use these results for device selection, then validate with a device-specific compile and measurements on the target hardware.
Some model zoos may provide corresponding performance and memory data for Ethos-U-based systems. Confirm that the published configuration is relevant to the candidate device before using those results.
Estimate the application's memory needs before selecting a device:
After the application runs on the target, the remaining memory can be used to tune performance. See Understand arena cache and spilling.
The diagram summarizes the integration workflow and its iteration loop. Follow the detailed steps in order because later steps depend on earlier decisions and measurements.
flowchart TD
setup["Select MCU and DFP<br/>Check DFP resources<br/>Create CMSIS-Toolbox project"] --> compile["Compile ML model<br/>for device"]
compile --> configure["Configure memory placement and linker script"]
configure --> integrate["Complete application integration"]
integrate --> validate["Validate and tune"]
validate -. Iterate .-> compile
This tutorial uses Keil Studio for VS Code, available from the VS Code Marketplace. Command-line users may use the CMSIS-Toolbox with a similar workflow.
This tutorial applies the five-step workflow to the Hello-Ethos-U examples in the ARM::CMSIS-Ethos-U pack. The three examples have the same application structure and uses the same ML models. Each example targets a different Ethos-U variant.
In Keil Studio, use Create a New Solution as described in Work with CMSIS solutions. From the table, select the target board and corresponding example that matches the Ethos-U variant in the target hardware. After creating the example, you can compile it and run it directly on the FVP simulation with the action buttons in the CMSIS view. Keil Studio automatically downloads and installs the required tools and software packs. The initial setup may take some time.
| Target board | Example | NPU/MACs | FVP simulation model |
|---|---|---|---|
| V2M-MPS3-SSE-300-FVP | Hello-Ethos-U55.csolution.yml | Ethos-U55-128 | Corstone-300 |
| V2M-MPS3-SSE-300-FVP | Hello-Ethos-U65.csolution.yml | Ethos-U65-256 | Corstone-300 |
| SSE-320 | Hello-Ethos-U85.csolution.yml | Ethos-U85-256 | Corstone-320 |
Each example includes the Hello-Ethos-U.cproject.yml file and the software layers shown in this diagram:
flowchart TD
solution["NPU-specific solution<br/>Hello-Ethos-Uxx.csolution.yml"] --> target["Target configuration<br/>device, FVP, and Board-Uxx.clayer.yml"]
solution --> project["Application project<br/>Hello-Ethos-U.cproject.yml"]
project --> sources["Application sources<br/>Source/test_main.cpp"]
project --> model["Model layer<br/>ML-MyModels.clayer.yml"]
The model layer supplies two quantized TensorFlow Lite (int8) models: hello_world, a dense network that approximates sin(x), and tiny_cnn, an image classifier that exercises convolution and pooling operations. The application runs one test input through each model on the selected NPU and checks both results against embedded reference output.
When both model checks pass, the FVP simulation model outputs:
This confirms that the example and tool setup work correctly.
Use the supplied layer as a known-good baseline to verify the system integration and tool setup. Once verified, replace it with a layer containing the application-specific ML model.
For our application, we selected the Alif Semiconductor Ensemble E7 (AE722F80F55D5LS) device and the related AppKit-E7-AIML board. We target the Ethos-U55 NPU on this device and therefore start with Hello-Ethos-U55.csolution.yml.
Open Hello-Ethos-U55.csolution.yml and add the DFP and BSP packs required by the selected device and board. Use the information in the CMSIS-Pack catalog to identify these packs. Then add a hardware target to the target-types: node before the existing FVP target. For the E7 device and AppKit, the relevant parts of the solution are:
Save the solution. Keil Studio discovers compatible board layers in the installed packs and prompts you to select one. The Alif E7 pack provides two layers for the AppKit-E7-AIML board:
This example uses the Board/AppKit-E7_M55_HP layer.
The mlops: information in the *.csolution.yml file describes the NPU, Vela configuration, model layer, and test targets used by an MLOps system. Update this information in two stages.
First select the NPU independently of its memory configuration. The Alif E7 M55_HP processor integrates an Ethos-U55 with 256 MACs, so change macs: from 128 to 256 in Hello-Ethos-U55.csolution.yml:
In Keil Studio, saving the solution runs cbuild setup and regenerates Hello-Ethos-U55.cbuild-mlops.yml. CMSIS-Toolbox combines the MLOps settings with the NPU and processor information published by the selected device and DFP.
Inspect the vela: section of the generated *.cbuild-mlops.yml file. Its ini: entry identifies the device-specific vela.ini supplied by the DFP. For example:
The generated options: entry might initially contain a system configuration or memory mode inherited from the original example. List the configurations and memory modes that the device-specific vela.ini actually provides:
Choose from the reported values by comparing the memory types, bandwidths, clock frequency, and AXI mappings with the target hardware and intended model placement. In the *.csolution.yml file, update the vela: node under mlops: with selectors that are present in the device-specific vela.ini:
Save the solution again. In the regenerated *.cbuild-mlops.yml, verify that the ini: and options: entries under vela: contain the intended configuration.
The supplied Corstone-300 simulator target uses Ethos-U55-128 by default. To test the same Ethos-U55-256 model that runs on the Alif E7, update the simulator configuration as follows:
In Board/Corstone-300/Board-U55.clayer.yml, set the compile-time driver configuration to:
In Board/Corstone-300/fvp_config_u55.txt, configure the simulated NPU:
The example contains the original quantized TensorFlow Lite models and Vela output compiled for the original Ethos-U55-128 configuration. Recompile each quantized model for the Ethos-U55-256 configuration and the device-specific system and memory mode that is reported in the generated Hello-Ethos-U55.cbuild-mlops.yml file.
These values can be directly applied to Vela. See MLOps Information.
See Vela for details about command-line options. Options such as --optimise Size can be added with the mlops.vela.misc: control in the *.csolution.yml file.
Repeat the command for every quantized model in the model layer. Replace the previous Vela output used by the application and regenerate its embedded C data if the project stores the model as a C array. Keep the original quantized .tflite file as the portable input; the _vela.tflite output is specific to the selected Ethos-U and memory configuration. For more information about this handoff, see MLOps Information.
The supplied ML-MyModels.clayer.yml has the layer type ML-Model. It selects the TensorFlow Lite Micro and Ethos-U kernel components and adds the model files to the application. The layer contains:
To use an application-specific model, copy or modify this layer, add the new model files to its groups: node, and set model.clayer in the solution's mlops: node to the resulting layer.
Keep the Vela memory mode, linker placement, and driver regions consistent, then build the system and confirm the allocations in the linker map. The diagram summarizes the required settings.
flowchart LR
mode["Vela memory mode"] --> regions["Command-stream<br/>regions"]
regions --> linker["Linker placement"]
linker --> memory["Physical memory"]
attributes["MPU/SAU and<br/>cache attributes"] --> memory
driver["Driver<br/>region settings"] --> memory
The examples place compiled model constants in the ethos_model section, the tensor arena in ethos_arena, and an optional cache buffer in ethos_cache. Map these sections to the physical memories selected by the Vela memory mode. Preserve their alignment requirements and use the linker map to confirm their addresses and sizes. See Create the linker script for the detailed mapping between Vela memory areas and linker sections.
Configure each memory region so that the CPU and NPU have the required security and access permissions. If a CPU mapping is cacheable and is not coherent with the NPU, provide the required cache clean and invalidate operations.
Set NPU_REGIONCFG_0, NPU_REGIONCFG_1, and NPU_REGIONCFG_2 as defines in the related <board>.clayer.yml file so that the driver access paths match const_mem_area, arena_mem_area, and cache_mem_area. See Match the driver configuration for the mapping.
Review the driver's weak callbacks described in Platform-specific functions and override those required by the target. These callbacks provide cache maintenance, CPU-to-NPU address remapping, run-time region selection, RTOS synchronization, and begin/end inference hooks for power control or tracing.
Follow the related guidance for data caching, mutexes and semaphores, and begin/end inference callbacks. Configure a finite inference timeout where appropriate, report NPU faults, and complete the driver bring-up checklist before relying on the integration in an application.
Verify correctness, memory use, and performance on the target hardware.
When tuning, change one parameter at a time, such as the Vela memory mode, arena_cache_size, or optimization strategy, then recompile and repeat the checks. Use Read the Vela reports and Compare Ethos-U configurations to record and compare the results.
During bring-up, provide a watchdog or RTOS timeout so that a missing completion interrupt does not block the application indefinitely. If an inference times out, capture the driver logs and NPU state before resetting the NPU. Check the following areas:
Ethos-U65 and Ethos-U85 can use a memory mode in which arena_mem_area and cache_mem_area resolve to different memory types. The tensor arena can reside in external writable memory while a faster SRAM region is used for staging. This arrangement is not applicable to Ethos-U55 because its AXI1 interface is read-only.
Select a compatible memory mode and set arena_cache_size to the available scratch-fast capacity. Reserve the cache_mem_area in the linker script and configure its MPU/SAU, cache, and driver region settings consistently. See Understand arena cache and spilling and Create the linker script.
Assume that the selected target provides NPU-accessible external DRAM and that the tensor arena currently resides in SRAM. Moving it to DRAM requires these coordinated changes: