Public API functions in ethosu_driver.h. More...

Functions
int	ethosu_init (struct ethosu_driver drv, void const base_address, const void *fast_memory, const size_t fast_memory_size, uint32_t secure_enable, uint32_t privilege_enable)
int	ethosu_init_ex (struct ethosu_driver drv, const struct ethosu_device_desc dev_desc, struct ethosu_device_config dev_config, struct ethosu_device_user_ops dev_user_ops, void const base_address, const void fast_memory, const size_t fast_memory_size, uint32_t secure_enable, uint32_t privilege_enable)
void	ethosu_deinit (struct ethosu_driver *drv)
int	ethosu_soft_reset (struct ethosu_driver *drv)
int	ethosu_request_power (struct ethosu_driver *drv)
void	ethosu_release_power (struct ethosu_driver *drv)
void	ethosu_get_driver_version (struct ethosu_driver_version *ver)
void	ethosu_get_hw_info (struct ethosu_driver drv, struct ethosu_hw_info hw)
int	ethosu_invoke_v3 (struct ethosu_driver drv, const void custom_data_ptr, const int custom_data_size, uint64_t const base_addr, const size_t base_addr_size, const int num_base_addr, void *user_arg)
int	ethosu_invoke_async (struct ethosu_driver drv, const void custom_data_ptr, const int custom_data_size, uint64_t const base_addr, const size_t base_addr_size, const int num_base_addr, void *user_arg)
int	ethosu_invoke_auto (const void custom_data_ptr, const int custom_data_size, uint64_t const base_addr, const size_t base_addr_size, const int num_base_addr, void user_arg)
int	ethosu_get_product_config_from_cop_data (const void custom_data_ptr, const int custom_data_size, uint32_t product_out, uint32_t *log2_macs_out)
int	ethosu_wait (struct ethosu_driver *drv, bool block)
struct ethosu_driver *	ethosu_reserve_driver (void)
struct ethosu_driver *	ethosu_reserve_driver_ex (uint32_t product, uint32_t log2_macs)
void	ethosu_release_driver (struct ethosu_driver *drv)
void	ethosu_irq_handler (struct ethosu_driver *drv)

Description

Public API functions in ethosu_driver.h.

These functions manage driver instances and run inference jobs. See Driver Usage for synchronous and asynchronous invocation examples.

Function Documentation

◆ ethosu_init()

int ethosu_init	(	struct ethosu_driver *	drv,
		void *const	base_address,
		const void *	fast_memory,
		const size_t	fast_memory_size,
		uint32_t	secure_enable,
		uint32_t	privilege_enable )

Initialize the Ethos-U driver. All driver handles should be initialised before any other driver function is called.

Parameters

drv	Pointer to driver handle
base_address	NPU register base address
fast_memory	Fast memory area, used for Ethos-U65/U85 with spilling
fast_memory_size	Size in bytes of fast memory area
secure_enable	Configure NPU in secure- or non-secure mode
privilege_enable	Configure NPU in privileged- or non-privileged mode

Returns: 0 on success, else negative error code

Initializes and registers driver instance for one NPU. Fast memory is used for Ethos-U65/U85 spilling.

When the platform overrides the weak mutex or semaphore hooks, the hooks must be able to create synchronization objects during this call.

Initialized handle must not be copied, moved or initialized more than once without an intervening call to ethosu_deinit().

Note: Before the first inference, route and enable the NPU interrupt so that the target interrupt handler calls ethosu_irq_handler().

◆ ethosu_init_ex()

int ethosu_init_ex	(	struct ethosu_driver *	drv,
		const struct ethosu_device_desc *	dev_desc,
		struct ethosu_device_config *	dev_config,
		struct ethosu_device_user_ops *	dev_user_ops,
		void *const	base_address,
		const void *	fast_memory,
		const size_t	fast_memory_size,
		uint32_t	secure_enable,
		uint32_t	privilege_enable )

Initialize the Ethos-U driver (multi variant mode). All driver handles should be initialised before any other driver function is called.

Parameters

drv	Pointer to driver handle
dev_desc	Pointer to device descriptor
dev_config	Pointer to device config
dev_user_ops	Pointer to device user ops (can be NULL)
base_address	NPU register base address
fast_memory	Fast memory area, used for Ethos-U65/U85 with spilling
fast_memory_size	Size in bytes of fast memory area
secure_enable	Configure NPU in secure- or non-secure mode
privilege_enable	Configure NPU in privileged- or non-privileged mode

Returns: 0 on success, else negative error code

Initializes and registers one NPU instance in a multi-variant build. The device descriptor selects the NPU backend, the device configuration supplies the register settings, and the optional user operations provide per-driver address-remapping and memory-configuration callbacks.

The lifecycle and interrupt-routing requirements are the same as for ethosu_init.

◆ ethosu_deinit()

void ethosu_deinit ( struct ethosu_driver * drv )

Deinitialize the Ethos-U driver. Caller must make sure that the driver handle is not in use, and that there are no outstanding reserve calls for it.

Parameters

drv	Pointer to driver handle

Deregisters the driver and destroys allocated resources. The driver must be idle. Do not call this function while the handle is reserved by another task or while an asynchronous job is running or waiting to be finalized. Storage for the handle must remain valid until ethosu_deinit() completes.

◆ ethosu_soft_reset()

int ethosu_soft_reset ( struct ethosu_driver * drv )

Soft resets the Ethos-U device.

Parameters

drv	Pointer to driver handle

Returns: 0 on success, else negative error code

Resets the NPU, restores its configuration, and reapplies power and clock settings.

◆ ethosu_request_power()

int ethosu_request_power ( struct ethosu_driver * drv )

Request to disable Q-channel power gating of the Ethos-U device. Power requests are ref.counted. Increases count. (Note: clock gating is made to follow power gating)

Parameters

drv	Pointer to driver handle

Returns: 0 on success, else negative error code

Adds a reference-counted request that disables Q-channel clock and power gating. The first request soft-resets the NPU.

◆ ethosu_release_power()

void ethosu_release_power ( struct ethosu_driver * drv )

Release disable request for Q-channel power gating of the Ethos-U device. Power requests are ref.counted. Decreases count.

Parameters

drv	Pointer to driver handle

Releases one request and enables Q-channel clock and power gating when the count reaches zero.

◆ ethosu_get_driver_version()

void ethosu_get_driver_version ( struct ethosu_driver_version * ver )

Get Ethos-U driver version.

Parameters

ver	Driver version struct

Retrieves the driver version.

After function return, structure ethosu_driver_version members hold driver version. Member

major: contains the value of ETHOSU_DRIVER_VERSION_MAJOR
minor: contains the value of ETHOSU_DRIVER_VERSION_MINOR
patch: contains the value of ETHOSU_DRIVER_VERSION_PATCH

◆ ethosu_get_hw_info()

void ethosu_get_hw_info	(	struct ethosu_driver *	drv,
		struct ethosu_hw_info *	hw )

Get Ethos-U hardware information.

Parameters

drv	Pointer to driver handle
hw	Hardware information struct

Retrieves the NPU hardware information.

Example:

static struct ethosu_driver EthosDriver;
struct ethosu_hw_info       hw_info;
 
ethosu_get_hw_info(&EthosDriver, &hw_info);
 
printf("Ethos-U version info:\n");
printf("\tArch:       v%u.%u.%u\n", hw_info.version.arch_major_rev,
                                    hw_info.version.arch_minor_rev,
                                    hw_info.version.arch_patch_rev);
printf("\tMACs/cc:    %u\n", (uint32_t)(1 << hw_info.cfg.macs_per_cc));
printf("\tCmd stream: v%u\n", hw_info.cfg.cmd_stream_version);

◆ ethosu_invoke_v3()

int ethosu_invoke_v3	(	struct ethosu_driver *	drv,
		const void *	custom_data_ptr,
		const int	custom_data_size,
		uint64_t *const	base_addr,
		const size_t *	base_addr_size,
		const int	num_base_addr,
		void *	user_arg )

Invoke command stream.

Note that when multi variant mode is enabled, calling this function with drv == NULL will call ethosu_invoke_auto function.

Parameters

drv	Pointer to driver handle
custom_data_ptr	Custom data payload
custom_data_size	Size in bytes of custom data
base_addr	Array of base address pointers
base_addr_size	Size in bytes of each address in base_addr
num_base_addr	Number of elements in base_addr array
user_arg	User argument, will be passed to ethosu_inference_begin() and ethosu_inference_end()

Returns: 0 on success, else negative error code

Blocks until inference completes.

custom_data_ptr must contain a trusted Vela Custom Operator Payload 1. The driver verifies the payload identifier, word alignment, optimizer compatibility, and supported driver actions, but it is not a parser for untrusted or arbitrarily malformed input.

base_addr and base_addr_size must have matching entries for at most eight 16-byte-aligned base addresses. num_base_addr must be in the range 0 through

The command stream must also be 16-byte aligned.

The sizes are used for cache maintenance and fast-memory validation. They do not create hardware bounds around NPU accesses and do not prove that offsets in the command stream stay within a supplied region.

When fast memory is configured and base-address index 2 exists, the driver validates its size and replaces base_addr[2] with the fast-memory address. base_addr is mutable for this reason.

user_arg is passed to the inference callbacks. Synchronous and asynchronous invocation may be mixed sequentially; a driver supports one outstanding job.

◆ ethosu_invoke_async()

int ethosu_invoke_async	(	struct ethosu_driver *	drv,
		const void *	custom_data_ptr,
		const int	custom_data_size,
		uint64_t *const	base_addr,
		const size_t *	base_addr_size,
		const int	num_base_addr,
		void *	user_arg )

Invoke command stream using async interface. Must be followed by call(s) to ethosu_wait() upon successful return.

See also: ethosu_invoke_v3 for documentation.

Parameters

drv	Pointer to driver handle.
custom_data_ptr	Pointer to the command stream payload.
custom_data_size	Size in bytes of the command stream payload.
base_addr	Array of base address pointers for tensor arenas and weight buffers.
base_addr_size	Array of sizes in bytes, one entry per base address.
num_base_addr	Number of entries in base_addr and base_addr_size.
user_arg	User argument passed to ethosu_inference_begin() and ethosu_inference_end().

Returns: 0 on success, else negative error code.

Starts one job and returns before inference completion.

After a successful call, keep the payload and both base-address arrays valid until ethosu_wait() returns 0 or -1. Complete the job with polling or blocking calls to ethosu_wait(). The memory referenced by the base-address array must also remain allocated and under NPU ownership for that interval.

◆ ethosu_invoke_auto()

int ethosu_invoke_auto	(	const void *	custom_data_ptr,
		const int	custom_data_size,
		uint64_t *const	base_addr,
		const size_t *	base_addr_size,
		const int	num_base_addr,
		void *	user_arg )

Call this to automatically find a suitable driver matching what the network has been compiled for. Note that this will potentially block waiting for a driver to become available, as it does implicit reserve- and release of a matching driver.

See also: ethosu_invoke_v3 for documentation, except it doesn't take a driver arg.

Parameters

custom_data_ptr	Pointer to the command stream payload.
custom_data_size	Size in bytes of the command stream payload.
base_addr	Array of base address pointers for tensor arenas and weight buffers.
base_addr_size	Array of sizes in bytes, one entry per base address.
num_base_addr	Number of entries in base_addr and base_addr_size.
user_arg	User argument passed to ethosu_inference_begin() and ethosu_inference_end().

Returns: 0 on success, else negative error code.

Reads the required product and MAC configuration from the Vela custom operator payload, reserves a matching driver, invokes the inference synchronously, and releases the driver.

The payload, base-address, cache-maintenance, and callback requirements are the same as for ethosu_invoke_v3.

◆ ethosu_get_product_config_from_cop_data()

int ethosu_get_product_config_from_cop_data	(	const void *	custom_data_ptr,
		const int	custom_data_size,
		uint32_t *	product_out,
		uint32_t *	log2_macs_out )

Parses the provided custom operator payload data to find which product configuration the network is compiled for. The output data can for instance be passed to ethosu_reserve_driver_ex().

Parameters

custom_data_ptr	Custom data payload
custom_data_size	Size in bytes of custom data
product_out	Output variable where product type is placed on successful data parsing
log2_macs_out	Output variable where mac configuration is placed on successful data parsing

Returns: 0 on success, -1 on error

Reads the Ethos-U product and encoded MAC configuration required by a Vela custom operator payload. On success, the returned values can be passed to ethosu_reserve_driver_ex.

◆ ethosu_wait()

int ethosu_wait	(	struct ethosu_driver *	drv,
		bool	block )

Wait for inference to complete (block=true) Poll status or finish up if inference is complete (block=false) (This function is only intended to be used in conjuction with ethosu_invoke_async)

Parameters

drv	Pointer to driver handle
block	If call should block if inference is running

Returns: -2 on inference not invoked, -1 on inference error, 0 on success, 1 on inference running

Finalizes a job started by ethosu_invoke_async(). When block=true, waits for completion; when block=false, returns 1 while the job is running.

The timeout value used by a blocking inference wait is ETHOSU_SEMAPHORE_WAIT_INFERENCE. Its unit and timeout behavior are defined by the platform implementation of ethosu_semaphore_take(); the default value requests an indefinite wait.

Finalizing a completed or timed-out job invalidates the configured cache regions, calls ethosu_inference_end(), releases the inference power request, and resets the internal job state.

If called from another thread concurrently with ethosu_invoke_async(), caller ordering must guarantee wait is entered only after successful async invoke completion.

◆ ethosu_reserve_driver()

struct ethosu_driver * ethosu_reserve_driver ( void )

Reserves a driver to execute inference with. Call will block until a driver is available.

Returns: Pointer to driver handle.

In multi-NPU usage, this call is serialized with mutex operations and may block on a semaphore when no driver instance is free.

Waiting for an available driver instance uses the global driver semaphore with forever-wait semantics. At least one driver must have been successfully initialized and registered before this function is called.

◆ ethosu_reserve_driver_ex()

struct ethosu_driver * ethosu_reserve_driver_ex	(	uint32_t	product,
		uint32_t	log2_macs )

Reserves a driver to execute inference with. Call will block until a driver is available. Macros for the product (ETHOSU_PRODUCT_U*) and number of macs (ETHOSU_MACS_*) parameters can be found in ethosu_type.h

Parameters

product	Ethos-U product
log2_macs	log2(num macs)

Returns: Pointer to driver handle.

Blocks until a registered driver matching the requested Ethos-U product and encoded MAC configuration is available, then reserves and returns it.

Use an ETHOSU_PRODUCT_U* value for product and an ETHOSU_MACS_* value for log2_macs.

◆ ethosu_release_driver()

void ethosu_release_driver ( struct ethosu_driver * drv )

Release driver that was previously reserved with

See also: ethosu_reserve_driver or; ethosu_reserve_driver_ex

Parameters

drv	Pointer to driver handle

Marks the driver available and wakes a task waiting for a driver. Complete an asynchronous job with ethosu_wait() before release; releasing a running job may reset the NPU without calling ethosu_inference_end().

◆ ethosu_irq_handler()

void ethosu_irq_handler ( struct ethosu_driver * drv )

Interrupt handler to be called on IRQ from Ethos-U

Parameters

drv	Pointer to driver handle

This function must be called from the target NPU interrupt handler.

It acknowledges the interrupt, records the job result, and unblocks tasks waiting for the job completion.

Functions

Description

Function Documentation

◆ ethosu_init()

◆ ethosu_init_ex()

◆ ethosu_deinit()

◆ ethosu_soft_reset()

◆ ethosu_request_power()

◆ ethosu_release_power()

◆ ethosu_get_driver_version()

◆ ethosu_get_hw_info()

◆ ethosu_invoke_v3()

◆ ethosu_invoke_async()

◆ ethosu_invoke_auto()

◆ ethosu_get_product_config_from_cop_data()

◆ ethosu_wait()

◆ ethosu_reserve_driver()

◆ ethosu_reserve_driver_ex()

◆ ethosu_release_driver()

◆ ethosu_irq_handler()