21.05
|
#include <Runtime.hpp>
Public Member Functions | |
Status | LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network) |
Loads a complete network into the Runtime. More... | |
Status | LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network, std::string &errorMessage) |
Load a complete network into the IRuntime. More... | |
Status | LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network, std::string &errorMessage, const INetworkProperties &networkProperties) |
TensorInfo | GetInputTensorInfo (NetworkId networkId, LayerBindingId layerId) const |
TensorInfo | GetOutputTensorInfo (NetworkId networkId, LayerBindingId layerId) const |
Status | EnqueueWorkload (NetworkId networkId, const InputTensors &inputTensors, const OutputTensors &outputTensors) |
void | Schedule (NetworkId networkId, const InputTensors &inputTensors, const OutputTensors &outputTensors, const QosExecPriority priority, std::shared_ptr< IAsyncExecutionCallback > callback) |
This is an experimental function. More... | |
Status | Execute (IWorkingMemHandle &workingMemHandle, const InputTensors &inputTensors, const OutputTensors &outputTensors) |
This is an experimental function. More... | |
Status | UnloadNetwork (NetworkId networkId) |
Unloads a network from the Runtime. More... | |
const IDeviceSpec & | GetDeviceSpec () const |
const std::shared_ptr< IProfiler > | GetProfiler (NetworkId networkId) const |
Gets the profiler corresponding to the given network id. More... | |
std::unique_ptr< IWorkingMemHandle > | CreateWorkingMemHandle (NetworkId networkId) |
Create a new unique WorkingMemHandle object. More... | |
void | RegisterDebugCallback (NetworkId networkId, const DebugCallbackFunction &func) |
Registers a callback function to debug layers performing custom computations on intermediate tensors. More... | |
RuntimeImpl (const IRuntime::CreationOptions &options) | |
Creates a runtime for workload execution. More... | |
~RuntimeImpl () | |
void | ReportStructure () |
Public Member Functions inherited from IReportStructure | |
virtual | ~IReportStructure () |
Friends | |
void | RuntimeLoadedNetworksReserve (RuntimeImpl *runtime) |
profiling::ProfilingService & | GetProfilingService (RuntimeImpl *runtime) |
Definition at line 30 of file Runtime.hpp.
RuntimeImpl | ( | const IRuntime::CreationOptions & | options | ) |
Creates a runtime for workload execution.
Definition at line 269 of file Runtime.cpp.
References ProfilingService::AddBackendProfilingContext(), DeviceSpec::AddSupportedBackends(), ARMNN_ASSERT, ARMNN_LOG, ARMNN_VERSION, armnn::BackendRegistryInstance(), ProfilingService::ConfigureProfilingService(), BackendRegistry::GetFactory(), armnn::GetTimeDuration(), armnn::GetTimeNow(), armnn::info, IRuntime::CreationOptions::m_DynamicBackendsPath, IRuntime::CreationOptions::ExternalProfilingOptions::m_EnableProfiling, IRuntime::CreationOptions::m_ProfilingOptions, IRuntime::CreationOptions::ExternalProfilingOptions::m_TimelineEnabled, BackendRegistry::SetProfilingService(), and ProfilingService::WaitForProfilingServiceActivation().
Referenced by RuntimeImpl::GetDeviceSpec().
~RuntimeImpl | ( | ) |
Definition at line 338 of file Runtime.cpp.
References ARMNN_LOG, armnn::BackendRegistryInstance(), DeviceSpec::ClearDynamicBackends(), DynamicBackendUtils::DeregisterDynamicBackends(), DeviceSpec::GetDynamicBackends(), armnn::GetTimeDuration(), armnn::GetTimeNow(), armnn::info, BackendRegistry::SetProfilingService(), and RuntimeImpl::UnloadNetwork().
Referenced by RuntimeImpl::GetDeviceSpec().
std::unique_ptr< IWorkingMemHandle > CreateWorkingMemHandle | ( | NetworkId | networkId | ) |
Create a new unique WorkingMemHandle object.
Create multiple handles if you wish to have overlapped Execution by calling this function from different threads.
Definition at line 492 of file Runtime.cpp.
References ARMNN_LOG, ARMNN_SCOPED_PROFILING_EVENT, LoadedNetwork::CreateWorkingMemHandle(), armnn::error, LoadedNetwork::FreeWorkingMemory(), ProfilerManager::GetInstance(), LoadedNetwork::GetProfiler(), LoadedNetwork::IsAsyncEnabled(), ProfilerManager::RegisterProfiler(), and armnn::Undefined.
Referenced by RuntimeImpl::GetDeviceSpec().
Status EnqueueWorkload | ( | NetworkId | networkId, |
const InputTensors & | inputTensors, | ||
const OutputTensors & | outputTensors | ||
) |
Definition at line 405 of file Runtime.cpp.
References ARMNN_LOG, ARMNN_SCOPED_PROFILING_EVENT, LoadedNetwork::EnqueueWorkload(), armnn::error, armnn::Failure, LoadedNetwork::FreeWorkingMemory(), ProfilerManager::GetInstance(), LoadedNetwork::GetProfiler(), LoadedNetwork::IsAsyncEnabled(), ProfilerManager::RegisterProfiler(), and armnn::Undefined.
Referenced by BOOST_AUTO_TEST_CASE(), and VerifyPostOptimisationStructureTestImpl().
Status Execute | ( | IWorkingMemHandle & | workingMemHandle, |
const InputTensors & | inputTensors, | ||
const OutputTensors & | outputTensors | ||
) |
This is an experimental function.
Evaluates a network using input in inputTensors and outputs filled into outputTensors. This function performs a thread safe execution of the network. Returns once execution is complete. Will block until this and any other thread using the same workingMem object completes.
Definition at line 438 of file Runtime.cpp.
References ARMNN_LOG, ARMNN_SCOPED_PROFILING_EVENT, armnn::error, LoadedNetwork::Execute(), armnn::Failure, ProfilerManager::GetInstance(), IWorkingMemHandle::GetNetworkId(), LoadedNetwork::GetProfiler(), LoadedNetwork::IsAsyncEnabled(), ProfilerManager::RegisterProfiler(), and armnn::Undefined.
|
inline |
Definition at line 88 of file Runtime.hpp.
References RuntimeImpl::CreateWorkingMemHandle(), RuntimeImpl::GetProfiler(), RuntimeImpl::GetProfilingService, RuntimeImpl::RegisterDebugCallback(), RuntimeImpl::ReportStructure(), RuntimeImpl::RuntimeImpl(), RuntimeImpl::RuntimeLoadedNetworksReserve, and RuntimeImpl::~RuntimeImpl().
Referenced by BOOST_AUTO_TEST_CASE(), and VerifyPostOptimisationStructureTestImpl().
TensorInfo GetInputTensorInfo | ( | NetworkId | networkId, |
LayerBindingId | layerId | ||
) | const |
Definition at line 394 of file Runtime.cpp.
References LoadedNetwork::GetInputTensorInfo().
Referenced by BOOST_AUTO_TEST_CASE(), and VerifyPostOptimisationStructureTestImpl().
TensorInfo GetOutputTensorInfo | ( | NetworkId | networkId, |
LayerBindingId | layerId | ||
) | const |
Definition at line 399 of file Runtime.cpp.
References LoadedNetwork::GetOutputTensorInfo().
Referenced by BOOST_AUTO_TEST_CASE(), and VerifyPostOptimisationStructureTestImpl().
Gets the profiler corresponding to the given network id.
networkId | The id of the network for which to get the profile. |
Definition at line 242 of file Runtime.cpp.
Referenced by RuntimeImpl::GetDeviceSpec().
Status LoadNetwork | ( | NetworkId & | networkIdOut, |
IOptimizedNetworkPtr | network | ||
) |
Loads a complete network into the Runtime.
[out] | networkIdOut | - Unique identifier for the network is returned in this reference. |
[in] | network | - Complete network to load into the Runtime. The runtime takes ownership of the network once passed in. |
Definition at line 130 of file Runtime.cpp.
References IRuntime::LoadNetwork().
Referenced by BOOST_AUTO_TEST_CASE(), and VerifyPostOptimisationStructureTestImpl().
Status LoadNetwork | ( | NetworkId & | networkIdOut, |
IOptimizedNetworkPtr | network, | ||
std::string & | errorMessage | ||
) |
Load a complete network into the IRuntime.
[out] | networkIdOut | Unique identifier for the network is returned in this reference. |
[in] | network | Complete network to load into the IRuntime. |
[out] | errorMessage | Error message if there were any errors. The runtime takes ownership of the network once passed in. |
Definition at line 136 of file Runtime.cpp.
References IRuntime::LoadNetwork(), and armnn::Undefined.
Status LoadNetwork | ( | NetworkId & | networkIdOut, |
IOptimizedNetworkPtr | network, | ||
std::string & | errorMessage, | ||
const INetworkProperties & | networkProperties | ||
) |
Definition at line 144 of file Runtime.cpp.
References armnn::Failure, LoadedNetwork::MakeLoadedNetwork(), and armnn::Success.
void RegisterDebugCallback | ( | NetworkId | networkId, |
const DebugCallbackFunction & | func | ||
) |
Registers a callback function to debug layers performing custom computations on intermediate tensors.
networkId | The id of the network to register the callback. |
func | callback function to pass to the debug layer. |
Definition at line 523 of file Runtime.cpp.
References DeviceSpec::AddSupportedBackends(), DynamicBackendUtils::CreateDynamicBackends(), DynamicBackendUtils::GetBackendPaths(), DynamicBackendUtils::GetSharedObjects(), LoadedNetwork::RegisterDebugCallback(), and DynamicBackendUtils::RegisterDynamicBackends().
Referenced by RuntimeImpl::GetDeviceSpec().
|
virtual |
Implements IReportStructure.
Definition at line 254 of file Runtime.cpp.
Referenced by RuntimeImpl::GetDeviceSpec().
void Schedule | ( | NetworkId | networkId, |
const InputTensors & | inputTensors, | ||
const OutputTensors & | outputTensors, | ||
const QosExecPriority | priority, | ||
std::shared_ptr< IAsyncExecutionCallback > | callback | ||
) |
This is an experimental function.
Schedule a thread safe execution by taking the input tensors and an execution priority for Quality of Service. The output tensors will then be filled and the callback object will notify that the execution has either succeeded or failed.
Definition at line 462 of file Runtime.cpp.
References ARMNN_SCOPED_PROFILING_EVENT, ProfilerManager::GetInstance(), LoadedNetwork::GetProfiler(), LoadedNetwork::IsAsyncEnabled(), ProfilerManager::RegisterProfiler(), LoadedNetwork::Schedule(), and armnn::Undefined.
Unloads a network from the Runtime.
At the moment this only removes the network from the m_Impl->m_Network. This might need more work in the future to be AndroidNN compliant.
[in] | networkId | Unique identifier for the network to be unloaded. Generated in LoadNetwork(). |
Definition at line 190 of file Runtime.cpp.
References ARMNN_LOG, LabelsAndEventClasses::ARMNN_PROFILING_EOL_EVENT_CLASS, armnn::debug, armnn::Failure, TimelineUtilityMethods::GetTimelineUtils(), armnn::Success, and armnn::warning.
Referenced by BOOST_AUTO_TEST_CASE(), and RuntimeImpl::~RuntimeImpl().
|
friend |
Definition at line 35 of file TestUtils.cpp.
Referenced by RuntimeImpl::GetDeviceSpec().
|
friend |
Definition at line 30 of file RuntimeTests.cpp.
Referenced by RuntimeImpl::GetDeviceSpec().