This chapter is the technical reference for the Ethos-U driver interface.
The Ethos-U driver operations and features at a glance:
The input to the Ethos-U driver must provide:
The driver does not compile models, allocate the ML framework's tensor arena, choose a memory mode in vela.ini, or place sections in physical memory.
For the first execution on a target, follow the Driver bring-up checklist.
The driver source is maintained in the Ethos-U core-driver repository. The software pack ARM::CMSIS-Ethos-U includes this source code unchanged and makes it available to CMSIS-based applications as a software component. The same source can therefore be obtained directly from the repository for use with other build environments. The pack also includes CMSIS-RTOS2, FreeRTOS and cache-management interfaces as optional source code templates.
| File or directory | Content |
|---|---|
| include/ | Public driver, device, data-type, and PMU header files. |
| src/ | Common driver code, variant-specific backend and PMU implementations, default configuration headers, register interfaces, and private headers. |
| zephyr/ | Metadata for using the driver as a Zephyr module. |
| CMakeLists.txt | Build description for integrating or building the driver with CMake. |
| README.md | Standalone build instructions and driver API examples. |
| LICENSE.txt, SECURITY.md | License and security information for the driver source. |
The pack ARM::CMSIS-Ethos-U provides the software component ARM::Machine Learning:NPU Support:Ethos-U Driver in multiple variants. For using the driver add one variant of the component as shown below:
The Ethos-U driver is configured with preprocessor #define statements. The following tables list the supported configuration statements.
A single-variant build supports one Ethos-U variant selected at compile time. Define exactly one Ethos-U family together with its MAC configuration. Do not define ETHOSU_MULTI_VARIANT. When using the CMSIS pack, selection of a single-variant Ethos-U Driver component defines the corresponding Ethos-U family automatically.
| Definition | Purpose |
|---|---|
| ETHOSU55, ETHOSU65, or ETHOSU85 | Select the Ethos-U family. Define exactly one. The CMSIS component selection defines this automatically. |
| ETHOSU_MACS | Select the MAC configuration, for example 128 for an Ethos-U55-128 variant. |
A multi-variant build includes support for Ethos-U55, Ethos-U65, and Ethos-U85. The Ethos-U variant and MAC configuration are selected for each driver instance at run time. Do not define the single-variant ETHOSU55, ETHOSU65, ETHOSU85, or ETHOSU_MACS statements in this build. When using the CMSIS pack, selection of the Multi-Variant Ethos-U Driver component defines ETHOSU_MULTI_VARIANT automatically.
| Definition | Purpose |
|---|---|
| ETHOSU_MULTI_VARIANT | Enable multi-variant support. The CMSIS component selection defines this automatically. This changes the initialization, reservation, platform-operation, and PMU APIs available to the application. |
The following #define statements apply in either a single-variant or multi-variant build. In a multi-variant build, the memory-routing statements initialize the default configuration for each included Ethos-U family.
NPU_QCONFIG and NPU_REGIONCFG_0 through NPU_REGIONCFG_7 provide default memory-access selectors; they do not contain addresses. NPU_QCONFIG selects the access configuration used to fetch the command stream. The suffix in NPU_REGIONCFG_n identifies the corresponding base_addr[n] region, while the value selects its access configuration. On Ethos-U55 and Ethos-U65, the value selects an AXI_LIMITx access profile. On Ethos-U85, it selects a MEM_ATTR entry. See Command stream regions and base pointers.
| Definition | Purpose |
|---|---|
| ETHOSU_MAX_WAITERS | Set the maximum number of distinct Ethos-U variants tracked by driver reservation. The default is 4. |
| ETHOSU_SEMAPHORE_WAIT_INFERENCE | Set the timeout passed to the platform semaphore while waiting for inference completion. The default is ETHOSU_SEMAPHORE_WAIT_FOREVER; the platform defines the time unit. See Mutex and semaphores. |
| NPU_QCONFIG | Set the default memory-access selector for fetching the command stream. |
| NPU_REGIONCFG_0 through NPU_REGIONCFG_7 | Set the default memory-access selector for each corresponding base_addr[0] through base_addr[7] region. |
The following #define statements configure driver text logging through the C standard I/O streams. See Logging for the generated macro reference and implementation details.
| Definition | Purpose |
|---|---|
| ETHOSU_LOG_ENABLE | Enable or disable logging. The default is 1. |
| ETHOSU_LOG_SEVERITY | Select the most verbose compiled log level: ETHOSU_LOG_ERR, ETHOSU_LOG_WARN, ETHOSU_LOG_INFO, or ETHOSU_LOG_DEBUG. The default is ETHOSU_LOG_WARN. |
The configuration headers provide default values for the following hardware #define statements. A silicon vendor can supply values validated for its NPU, interconnect, memory system, cache policy, and security configuration to adjust the driver defaults. Do not tune these settings independently. See Command stream regions and base pointers.
| Ethos-U variant | Configuration definitions |
|---|---|
| Ethos-U55 and Ethos-U65 | AXI_LIMITx_MAX_BEATS_BYTES, AXI_LIMITx_MEM_TYPE, AXI_LIMITx_MAX_OUTSTANDING_READS, and AXI_LIMITx_MAX_OUTSTANDING_WRITES, where x is 0 through 3. |
| Ethos-U85 | NPU_MAC_PWR_RAMP_CYCLES, NPU_MEM_ATTR_0 through NPU_MEM_ATTR_3, AXI_LIMIT_SRAM_MAX_OUTSTANDING_READ, AXI_LIMIT_SRAM_MAX_OUTSTANDING_WRITE, AXI_LIMIT_SRAM_MAX_BEATS, AXI_LIMIT_EXT_MAX_OUTSTANDING_READ, AXI_LIMIT_EXT_MAX_OUTSTANDING_WRITE, and AXI_LIMIT_EXT_MAX_BEATS. |
For Ethos-U55 and Ethos-U65, each AXI_LIMITx register defines a complete AXI access profile, despite the register name suggesting that it contains only transaction limits. The corresponding AXI_LIMITx_* #define statements configure its maximum burst size, memory type, and maximum number of outstanding read and write transactions. In particular, AXI_LIMITx_MEM_TYPE sets the memory type used to encode the AXI AxCACHE signals.
Vela generates command streams that refer to memory regions. The system configuration and memory mode selected in the vela.ini file determine the expected memory area for each memory region used by the compiled ML model.
The Ethos-U NPU and driver support eight base pointers and the corresponding REGIONCFG[0] ... REGIONCFG[7] selectors. Vela currently uses only three regions:
Consequently, only regions 0 through 2 are referenced in the following section.
The memory mode selects which Vela logical alias, Axi0 or Axi1, each memory area uses. The following table shows the default vela.ini settings. Region 2 (cache_mem_area) is used only when region 1 (arena_mem_area) uses a different logical alias.
| Vela memory mode | Region 0 (const_mem_area) | Region 1 (arena_mem_area) | Region 2 (cache_mem_area) |
|---|---|---|---|
| Sram_Only | Axi0 | Axi0 | Axi0 (not used) |
| Shared_Sram | Axi1 | Axi0 | Axi0 (not used) |
| Dedicated_Sram | Axi1 | Axi1 | Axi0 |
The system configuration maps each logical alias to a Vela memory type. The following tables show two example mappings from the default vela.ini settings.
Example using --system-config Ethos_U55_High_End_Embedded:
| Vela memory mode | Region 0 (const_mem_area) | Region 1 (arena_mem_area) | Region 2 (cache_mem_area) |
|---|---|---|---|
| Sram_Only | Sram | Sram | Not used |
| Shared_Sram | OffChipFlash | Sram | Not used |
| Dedicated_Sram | OffChipFlash | OffChipFlash (invalid) | Sram |
Example using --system-config Ethos_U85_SYS_DRAM_Mid:
| Vela memory mode | Region 0 (const_mem_area) | Region 1 (arena_mem_area) | Region 2 (cache_mem_area) |
|---|---|---|---|
| Sram_Only | Sram | Sram | Not used |
| Shared_Sram | Dram | Sram | Not used |
| Dedicated_Sram | Dram | Dram | Sram |
The linker maps each Vela memory type to physical memory. Even when region 0 is in Dram, it may be placed in ROM (Flash) because it contains constants. Driver base addresses and access settings must match the memory placement selected by Vela and the linker.
The simplest setup uses a single-variant driver with one fixed memory-access configuration for all ML models. The NPU_QCONFIG definition selects the memory-access configuration used to fetch the command stream. Each NPU_REGIONCFG_x definition selects the memory-access configuration for region x. In the Vela configurations above, x is 0, 1, or 2.
Selector encodings are NPU-specific; see Ethos-U55 and Ethos-U65 and Ethos-U85 below.
The memory-access configuration can also be selected at run time:
The inference invocation functions (ethosu_invoke_v3(), ethosu_invoke_async(), and ethosu_invoke_auto()) receive custom_data_ptr, which points to the Vela-generated custom-operator payload. This payload contains metadata and the command stream. The base_addr argument points to an array containing the actual base address of each memory region, and num_base_addr specifies the number of entries.
Vela uses memory region 2 for fast scratch memory. The fast_memory and fast_memory_size arguments to ethosu_init() or ethosu_init_ex() specify the physical location and size of this memory.
The following table maps invocation parameters to the corresponding NPU registers and driver definitions.
| Invocation function parameter | Address register | Access-configuration selector | Default definition |
|---|---|---|---|
| custom_data_ptr | QBASE | QCONFIG | NPU_QCONFIG |
| base_addr[n] | BASEP[n] | REGIONCFG[n] | NPU_REGIONCFG_n |
Ethos-U55 and Ethos-U65 provide the AXI0 and AXI1 access paths.
| Memory placement | AXI port |
|---|---|
| SRAM | AXI0 |
| DRAM/Flash | AXI1 |
The driver provides the following default values:
| Ethos-U variant | NPU_QCONFIG | NPU_REGIONCFG_0 | NPU_REGIONCFG_1 | NPU_REGIONCFG_2 |
|---|---|---|---|---|
| Ethos-U55 | 2 -> AXI1 | 3 -> AXI1 | 0 -> AXI0 | 1 -> AXI0 |
| Ethos-U65 | 2 -> AXI1 | 3 -> AXI1 | 0 -> AXI0 | 1 -> AXI0 |
For Ethos-U55 and Ethos-U65, QCONFIG and the REGIONCFG[0..7] fields accept values 0 through 3. The value selects an AXI port, an outstanding transaction counter, and the corresponding AXI_LIMITx access profile.
| Value | AXI port | AXI access profile | Transaction counter |
|---|---|---|---|
| 0 | AXI0 | AXI_LIMIT0 | AXI0_OUTSTANDING_COUNTER0 |
| 1 | AXI0 | AXI_LIMIT1 | AXI0_OUTSTANDING_COUNTER1 |
| 2 | AXI1 | AXI_LIMIT2 | AXI1_OUTSTANDING_COUNTER2 |
| 3 | AXI1 | AXI_LIMIT3 | AXI1_OUTSTANDING_COUNTER3 |
Transaction counters enforce the limits in the corresponding AXI_LIMITx profile. For their use in performance analysis, see Monitoring AXI transaction latency.
Each AXI_LIMITx access profile contains the burst split alignment, the memory type used to encode AxCACHE, and the maximum number of outstanding read and write transactions. These fields are configured by AXI_LIMITx_MAX_BEATS_BYTES, AXI_LIMITx_MEM_TYPE, AXI_LIMITx_MAX_OUTSTANDING_READS, and AXI_LIMITx_MAX_OUTSTANDING_WRITES, respectively.
The AXI ports are referred to as AXI_SRAM and AXI_EXT.
| Memory placement | AXI port |
|---|---|
| SRAM | AXI_SRAM |
| DRAM/Flash | AXI_EXT |
The driver provides the following default values:
| Ethos-U variant | NPU_QCONFIG | NPU_REGIONCFG_0 | NPU_REGIONCFG_1 | NPU_REGIONCFG_2 |
|---|---|---|---|---|
| Ethos-U85 | 2 -> MEM_ATTR_2 -> AXI_EXT | 3 -> MEM_ATTR_3 -> AXI_EXT | 0 -> MEM_ATTR_0 -> AXI_SRAM | 1 -> MEM_ATTR_1 -> AXI_SRAM |
For Ethos-U85, each NPU_MEM_ATTR_n #define supplies the packed value written to the corresponding MEM_ATTR[n] register. The suffix n identifies the entry; it is not the value assigned to the #define. Each entry selects the AXI port and specifies the memory domain and the memory type used to encode the AxCACHE signals.
QCONFIG and each REGIONCFG field contain an index from 0 through 3 that selects one of these MEM_ATTR entries. The default MEM_ATTR_0 and MEM_ATTR_1 entries use AXI_SRAM, while MEM_ATTR_2 and MEM_ATTR_3 use AXI_EXT. For example, NPU_QCONFIG defaults to 2, selecting MEM_ATTR_2; the default packed value of NPU_MEM_ATTR_2 is (1 << 2), or 4, which selects AXI_EXT.
These default MEM_ATTR values are set by the driver to replicate the default Ethos-U55 and Ethos-U65 behavior, making it easier to correlate configurations across variants, but the MEM_ATTR values can be configured by the user.
Ethos-U85 AXI limits are configured in the AXI_SRAM and AXI_EXT registers. There is one AXI_SRAM register and one AXI_EXT register, so those limit settings apply to all ports in each group.
Ethos-U85 AXI information:
| U85 configuration (MACs/CC) | Number of SRAM ports | Maximum outstanding reads per port | Maximum outstanding writes per port |
|---|---|---|---|
| 128 | 2 | 12 | 16 |
| 256 | 2 | 12 | 16 |
| 512 | 2 | 12 | 16 |
| 1024 | 2 | 12 | 16 |
| 2048 | 4 | 12 | 16 |
| U85 configuration (MACs/CC) | Number of EXT ports | Maximum outstanding reads per port | Maximum outstanding writes per port |
|---|---|---|---|
| 128 | 1 | 32 | 32 |
| 256 | 1 | 32 | 32 |
| 512 | 1 | 64 | 32 |
| 1024 | 2 | 64 | 32 |
| 2048 | 2 | 64 | 32 |
The driver API is defined in include/ethosu_driver.h with related types in include/ethosu_types.h.
| API Function | Build | Description |
|---|---|---|
| ethosu_init() | Single-variant | Initialize and register an NPU instance using the compile-time device selection. |
| ethosu_init_ex() | Multi-variant | Initialize and register an NPU instance using a device descriptor, run-time configuration, and optional per-driver user operations. |
| ethosu_deinit() | Both | Unregister an idle NPU instance and release its synchronization resources. |
| ethosu_invoke_v3() | Both | Submit an inference to a specified driver and wait synchronously for completion. |
| ethosu_invoke_async(), ethosu_wait() | Both | Submit an inference asynchronously, then poll or block for completion. |
| ethosu_invoke_auto() | Multi-variant | Read the network's NPU requirements, reserve a matching driver, run the inference, and release the driver. |
| ethosu_get_product_config_from_cop_data() | Both | Read the Ethos-U product and MAC configuration from a Vela custom-operator payload. |
| ethosu_irq_handler() | Both | Handle an NPU completion or fault interrupt for a driver instance. |
| ethosu_get_driver_version(), ethosu_get_hw_info() | Both | Inspect the driver version and an NPU instance's hardware information. |
| ethosu_soft_reset() | Both | Reset an NPU instance and restore its configuration. |
| ethosu_request_power(), ethosu_release_power() | Both | Manage reference-counted NPU power requests. |
| ethosu_reserve_driver() | Single-variant | Block until an instance of the compile-time NPU variant is available and reserve it. |
| ethosu_reserve_driver_ex() | Both | Block until an instance matching the requested product and MAC configuration is available and reserve it. |
| ethosu_release_driver() | Both | Release a reserved driver instance. |
See Driver functions for the complete generated API and Driver structures for public data types.
The driver provides default implementations for platform-specific functions listed below. The default weak function implementation of the driver should be carefully review and overwritten when needed.
| API Function | When an overwrite is needed |
|---|---|
| ethosu_flush_dcache(), ethosu_invalidate_dcache() | CPU-cached memory is shared with the NPU and requires platform-specific cache maintenance. |
| ethosu_address_remap() | The CPU and NPU use different addresses for the same storage. |
| ethosu_config_select() | Memory-region attributes depend on the address or run-time placement. |
| Mutex and semaphore functions in Platform-specific functions | Multiple threads or NPUs can use the driver and require platform-specific RTOS locking. |
| ethosu_inference_begin(), ethosu_inference_end() | Inference tracing, power control, or application callbacks are required. |
Cache policy, linker placement, and region configuration are system-level decisions. Detailed guidance is in the chapter Integration.
Inferences can be invoked in two manners: synchronously or asynchronously. The two types of invocation can be freely mixed in a single application.
The typical usage of the driver is the synchronous invocation as shown below:
The following simplified sequence diagram shows the synchronous invocation in an RTOS environment:
sequenceDiagram
participant application as ML thread
participant driver as Driver
participant npu as Ethos-U NPU
participant rtos as RTOS
participant threads as Other threads
application->>driver: ethosu_invoke_v3()
driver->>npu: Program regions and start
driver->>rtos: ethosu_semaphore_take(): Block ML thread
rtos->>threads: Schedule other ready threads
Note over threads,npu: Other threads can run while the NPU is busy
npu->>driver: Completion or fault interrupt
driver->>rtos: ethosu_semaphore_give(): Make ML thread ready
rtos-->>driver: Resume ML thread
driver-->>application: Inference result
With an RTOS implementation, ethosu_semaphore_take() blocks the calling thread and allows the RTOS scheduler to run other ready threads while the Ethos-U NPU executes the inference. The Ethos-U interrupt handler calls ethosu_semaphore_give() when the NPU completes or reports a fault, allowing the synchronous invocation to resume. The platform must provide the RTOS-specific semaphore functions described in Mutex and semaphores.
In some cases the asynchronous invocation is needed which allows in the same thread the execution of other work after starting the inference on the Ethos-U NPU.
The following simplified sequence diagram shows the asynchronous invocation:
sequenceDiagram
participant application as ML thread
participant driver as Driver
participant npu as Ethos-U NPU
application->>driver: ethosu_invoke_async()
driver->>npu: Program regions and start
driver-->>application: Submission result
Note over application,npu: ML thread can do other work while the NPU is busy
application->>driver: ethosu_wait()
npu->>driver: Completion or fault interrupt
driver-->>application: Wait completes
Initialize each driver instance with ethosu_init(). This registers the instance and makes it available for running inference. Call ethosu_deinit() to unregister and tear down the instance.
Create one driver instance for each NPU device. All registered NPUs must use the same compile-time NPU configuration because a driver build supports only one configuration.
Use this checklist after initializing the driver and before integrating large application graphs:
After these driver checks, continue with the end-to-end Integration workflow and Validate and tune before treating platform bring-up as complete.
Cache maintenance is required when the CPU and NPU share memory that is cached by the CPU. The driver provides weak no-op implementations of two hooks. If the ML runtime does not manage the cache, override both hooks:
The hooks receive the base_addr regions, but not the command stream. If the command stream is in cached memory written by the CPU, the application or ML runtime must clean it before invoking the inference. If the ML runtime already performs all required cache maintenance, the default weak no-op functions are sufficient.
For example, a platform can use the driver cache functions as follows:
The driver uses the following synchronization objects to reserve an NPU instance and wait for inference completion:
| Synchronization object | Used by | Purpose |
|---|---|---|
| Global driver mutex | Driver registration, deregistration, reservation, and release | Protects the registered-driver list and each driver's reserved flag. It ensures that concurrent threads cannot reserve the same NPU instance. |
| Global availability semaphore | Driver registration, deregistration, reservation, and release | Counts the number of driver instances currently available. ethosu_reserve_driver() waits on this semaphore when every NPU is reserved. |
| Per-driver completion semaphore | ethosu_wait() and ethosu_irq_handler() | Blocks a thread while its NPU is running. The NPU interrupt gives the semaphore after recording successful completion or a fault. |
The default synchronization functions support a single-threaded bare-metal application. An RTOS or multicore application must provide the platform-specific mutex and semaphore functions. The pack provides code templates for CMSIS-RTOS2 and native FreeRTOS implementations. See Platform-specific functions for the hook signatures.
The driver provides weak hooks that applications can override to receive inference begin and end callbacks:
The user_arg passed to ethosu_invoke_v3() or ethosu_invoke_async() is forwarded to both callbacks. For example:
For a practical use of these callbacks, see the PMU example.
The driver exposes the Ethos-U PMU through pmu_ethosu.h. The API supports a 64-bit cycle counter and programmable event counters. Ethos-U55 and Ethos-U65 builds provide four event counters; Ethos-U85 builds provide eight. Use ETHOSU_PMU_Get_NumEventCountersForDrv() when code must work with more than one Ethos-U target or with a multi-variant build.
The target-specific enum ethosu_pmu_event_type lists the supported events. They include NPU and MAC activity or stalls, weight-decoder and activation-output activity, memory transactions and stalls, latency ranges, and ECC events. The available event names differ between Ethos-U55/U65 and Ethos-U85. Use the symbolic enum values with ETHOSU_PMU_Set_EVTYPER(); do not program hardware event numbers directly.
On Ethos-U55 and Ethos-U65, PMCAXI_CHAN.AXI_CNT_SEL selects one of the outstanding transaction counters listed under Ethos-U55 and Ethos-U65, and PMCAXI_CHAN.CH_SEL selects the AXI channel to monitor. The ETHOSU_PMU_AXI_LATENCY_* events count transactions matching both selections. Other AXI PMU events, such as accepted or completed transactions, data beats, and stalls, are reported per AXI port rather than per outstanding transaction counter.
The main API groups are:
| API Function | Description |
|---|---|
| ETHOSU_PMU_Enable(), ETHOSU_PMU_Disable() | Enable or disable the PMU |
| ETHOSU_PMU_Get_NumEventCounters(), ETHOSU_PMU_Get_NumEventCountersForDrv() | Get the number of counters for the compile-time NPU or a specific driver |
| ETHOSU_PMU_Set_EVTYPER(), ETHOSU_PMU_Get_EVTYPER() | Select an event |
| ETHOSU_PMU_CYCCNT_Reset(), ETHOSU_PMU_EVCNTR_ALL_Reset() | Reset counters |
| ETHOSU_PMU_CNTR_Enable(), ETHOSU_PMU_CNTR_Disable() | Enable or disable counters |
| ETHOSU_PMU_Get_CCNTR(), ETHOSU_PMU_Get_EVCNTR() | Read counters |
| ETHOSU_PMU_Get_CNTR_OVS(), ETHOSU_PMU_Set_CNTR_OVS(), ETHOSU_PMU_Set_CNTR_IRQ_Enable(), ETHOSU_PMU_Set_CNTR_IRQ_Disable() | Handle overflow |
Counter masks use ETHOSU_PMU_CNT1_Msk and the other event-counter masks, plus ETHOSU_PMU_CCNT_Msk for the cycle counter. Enabling the PMU requests NPU power; disabling it releases that request, so always pair the two operations.
Override ethosu_inference_begin() to select and reset events immediately before an inference, and override ethosu_inference_end() to read the counters and disable the PMU afterward. These callbacks receive the same user_arg that was passed to ethosu_invoke_v3() or ethosu_invoke_async(), which can identify where the results should be stored. PMU measurements are hardware observations; compare them with Vela compiler estimates, but do not treat the estimates as cycle-accurate measurements.
The Technical Reference Manual for each NPU provides the event encoding and a hardware-level description of what each event counts:
Use the event descriptions together with the following guidelines:
Event counters are 32-bit and can overflow during a long measurement. Check ETHOSU_PMU_Get_CNTR_OVS() when this is possible. Because only four or eight events can be measured at once, collect additional events in repeated runs with identical model, input, memory, and system settings.
The following example counts NPU-active and NPU-idle events and records the NPU cycle count for one inference. Pass a pointer to struct pmu_results as the user_arg to ethosu_invoke_v3() or ethosu_invoke_async(). The callbacks use ETHOSU_PMU_Get_NumEventCountersForDrv() so the same pattern can be extended with additional events on targets that provide more counters.