IRuntime Class Reference

#include <IRuntime.hpp>

Classes
struct	CreationOptions

Public Member Functions
Status	LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network)
	Loads a complete network into the IRuntime. More...
Status	LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network, std::string &errorMessage)
	Load a complete network into the IRuntime. More...
Status	LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network, std::string &errorMessage, const INetworkProperties &networkProperties)
TensorInfo	GetInputTensorInfo (NetworkId networkId, LayerBindingId layerId) const
TensorInfo	GetOutputTensorInfo (NetworkId networkId, LayerBindingId layerId) const
Status	EnqueueWorkload (NetworkId networkId, const InputTensors &inputTensors, const OutputTensors &outputTensors)
	Evaluates a network using input in inputTensors and outputs filled into outputTensors. More...
Status	Execute (IWorkingMemHandle &workingMemHandle, const InputTensors &inputTensors, const OutputTensors &outputTensors)
	This is an experimental function. More...
void	Schedule (NetworkId networkId, const InputTensors &inputTensors, const OutputTensors &outputTensors, const QosExecPriority priority, std::shared_ptr< IAsyncExecutionCallback > callback)
	This is an experimental function Schedule a thread safe execution by taking the input tensors and an execution priority for Quality of Service. More...
Status	UnloadNetwork (NetworkId networkId)
	Unloads a network from the IRuntime. More...
const IDeviceSpec &	GetDeviceSpec () const
std::unique_ptr< IWorkingMemHandle >	CreateWorkingMemHandle (NetworkId networkId)
	Create a new unique WorkingMemHandle object. More...
const std::shared_ptr< IProfiler >	GetProfiler (NetworkId networkId) const
	Gets the profiler corresponding to the given network id. More...
void	RegisterDebugCallback (NetworkId networkId, const DebugCallbackFunction &func)
	Registers a callback function to debug layers performing custom computations on intermediate tensors. More...

Static Public Member Functions
static IRuntime *	CreateRaw (const CreationOptions &options)
static IRuntimePtr	Create (const CreationOptions &options)
static void	Destroy (IRuntime *runtime)

Protected Member Functions
	IRuntime ()
	IRuntime (const IRuntime::CreationOptions &options)
	~IRuntime ()

Protected Attributes
std::unique_ptr< RuntimeImpl >	pRuntimeImpl

Detailed Description

Definition at line 73 of file IRuntime.hpp.

Constructor & Destructor Documentation

IRuntime() [1/2]

IRuntime ( )

protected

Definition at line 26 of file Runtime.cpp.

Referenced by IRuntime::CreateRaw().

: pRuntimeImpl( new RuntimeImpl(armnn::IRuntime::CreationOptions())) {}

IRuntime() [2/2]

IRuntime ( const IRuntime::CreationOptions &  options )

protected

Definition at line 28 of file Runtime.cpp.

References IRuntime::~IRuntime().

: pRuntimeImpl(new RuntimeImpl(options)) {}

~IRuntime()

~IRuntime ( )

protecteddefault

Referenced by IRuntime::IRuntime().

Member Function Documentation

Create()

IRuntimePtr Create ( const CreationOptions &  options )

static

Definition at line 37 of file Runtime.cpp.

References IRuntime::CreateRaw(), and IRuntime::Destroy().

Referenced by armnn::experimental::AsyncEndToEndTestImpl(), armnn::experimental::AsyncThreadedEndToEndTestImpl(), BOOST_AUTO_TEST_CASE(), BOOST_FIXTURE_TEST_CASE(), GetSoftmaxProfilerJson(), InferenceModel< IParser, TDataType >::InferenceModel(), and main().

{
    return IRuntimePtr(CreateRaw(options), &IRuntime::Destroy);
}

CreateRaw()

IRuntime * CreateRaw ( const CreationOptions &  options )

static

Definition at line 32 of file Runtime.cpp.

References IRuntime::IRuntime().

Referenced by IRuntime::Create().

{
    return new IRuntime(options);
}

CreateWorkingMemHandle()

std::unique_ptr< IWorkingMemHandle > CreateWorkingMemHandle

(

NetworkId

networkId

)

Create a new unique WorkingMemHandle object.

Create multiple handles if you wish to have overlapped Execution by calling this function from different threads.

Definition at line 110 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->CreateWorkingMemHandle(networkId);
}

Destroy()

void Destroy ( IRuntime *  runtime )

static

Definition at line 42 of file Runtime.cpp.

Referenced by IRuntime::Create().

{
    delete runtime;
}

EnqueueWorkload()

Status EnqueueWorkload	(	NetworkId	networkId,
		const InputTensors &	inputTensors,
		const OutputTensors &	outputTensors
	)

Evaluates a network using input in inputTensors and outputs filled into outputTensors.

Definition at line 77 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->EnqueueWorkload(networkId, inputTensors, outputTensors);
}

Execute()

Status Execute	(	IWorkingMemHandle &	workingMemHandle,
		const InputTensors &	inputTensors,
		const OutputTensors &	outputTensors
	)

This is an experimental function.

Evaluates a network using input in inputTensors and outputs filled into outputTensors. This function performs a thread safe execution of the network. Returns once execution is complete. Will block until this and any other thread using the same workingMem object completes.

Definition at line 84 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->Execute(workingMemHandle, inputTensors, outputTensors);
}

GetDeviceSpec()

const IDeviceSpec & GetDeviceSpec

(

)

const

Definition at line 105 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->GetDeviceSpec();
}

GetInputTensorInfo()

TensorInfo GetInputTensorInfo	(	NetworkId	networkId,
		LayerBindingId	layerId
	)		const

Definition at line 67 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->GetInputTensorInfo(networkId, layerId);
}

GetOutputTensorInfo()

TensorInfo GetOutputTensorInfo	(	NetworkId	networkId,
		LayerBindingId	layerId
	)		const

Definition at line 72 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->GetOutputTensorInfo(networkId, layerId);
}

GetProfiler()

const std::shared_ptr< IProfiler > GetProfiler

(

NetworkId

networkId

)

const

Gets the profiler corresponding to the given network id.

Parameters

networkId

The id of the network for which to get the profile.

Returns

A pointer to the requested profiler, or nullptr if not found.

Definition at line 115 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->GetProfiler(networkId);
}

LoadNetwork() [1/3]

Status LoadNetwork	(	NetworkId &	networkIdOut,
		IOptimizedNetworkPtr	network
	)

Loads a complete network into the IRuntime.

Parameters

[out]	networkIdOut	- Unique identifier for the network is returned in this reference.
[in]	network	- Complete network to load into the IRuntime. The runtime takes ownership of the network once passed in.

Returns

armnn::Status

Definition at line 47 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

Referenced by RuntimeImpl::LoadNetwork().

{
    return pRuntimeImpl->LoadNetwork(networkIdOut, std::move(network));
}

LoadNetwork() [2/3]

Status LoadNetwork	(	NetworkId &	networkIdOut,
		IOptimizedNetworkPtr	network,
		std::string &	errorMessage
	)

Load a complete network into the IRuntime.

Parameters

[out]	networkIdOut	Unique identifier for the network is returned in this reference.
[in]	network	Complete network to load into the IRuntime.
[out]	errorMessage	Error message if there were any errors. The runtime takes ownership of the network once passed in.

Returns

armnn::Status

Definition at line 52 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->LoadNetwork(networkIdOut, std::move(network), errorMessage);
}

LoadNetwork() [3/3]

Status LoadNetwork	(	NetworkId &	networkIdOut,
		IOptimizedNetworkPtr	network,
		std::string &	errorMessage,
		const INetworkProperties &	networkProperties
	)

Definition at line 59 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->LoadNetwork(networkIdOut, std::move(network), errorMessage, networkProperties);
}

RegisterDebugCallback()

void RegisterDebugCallback	(	NetworkId	networkId,
		const DebugCallbackFunction &	func
	)

Registers a callback function to debug layers performing custom computations on intermediate tensors.

Parameters

networkId	The id of the network to register the callback.
func	callback function to pass to the debug layer.

Definition at line 120 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->RegisterDebugCallback(networkId, func);
}

Schedule()

void Schedule	(	NetworkId	networkId,
		const InputTensors &	inputTensors,
		const OutputTensors &	outputTensors,
		const QosExecPriority	priority,
		std::shared_ptr< IAsyncExecutionCallback >	callback
	)

This is an experimental function Schedule a thread safe execution by taking the input tensors and an execution priority for Quality of Service.

The output tensors will then be filled and the callback object will notify that the execution has either succeeded or failed.

Definition at line 91 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    pRuntimeImpl->Schedule(networkId, inputTensors, outputTensors, priority, cb);
}

UnloadNetwork()

Status UnloadNetwork

(

NetworkId

networkId

)

Unloads a network from the IRuntime.

At the moment this only removes the network from the m_Impl->m_Network. This might need more work in the future to be AndroidNN compliant.

Parameters

[in]

networkId

- Unique identifier for the network to be unloaded. Generated in LoadNetwork().

Returns

armnn::Status

Definition at line 100 of file Runtime.cpp.

References IRuntime::pRuntimeImpl.

{
    return pRuntimeImpl->UnloadNetwork(networkId);
}

Member Data Documentation

pRuntimeImpl

std::unique_ptr<RuntimeImpl> pRuntimeImpl

protected

Definition at line 232 of file IRuntime.hpp.

Referenced by IRuntime::CreateWorkingMemHandle(), IRuntime::EnqueueWorkload(), IRuntime::Execute(), IRuntime::GetDeviceSpec(), IRuntime::GetInputTensorInfo(), IRuntime::GetOutputTensorInfo(), IRuntime::GetProfiler(), IRuntime::LoadNetwork(), IRuntime::RegisterDebugCallback(), IRuntime::Schedule(), and IRuntime::UnloadNetwork().

---

The documentation for this class was generated from the following files:

• include/armnn/IRuntime.hpp
• src/armnn/Runtime.cpp