ArmNN
 24.08
TosaRefBackend Class Reference

#include <TosaRefBackend.hpp>

Inheritance diagram for TosaRefBackend:
[legend]
Collaboration diagram for TosaRefBackend:
[legend]

Public Member Functions

 TosaRefBackend ()=default
 
 ~TosaRefBackend ()=default
 
const BackendIdGetId () const override
 
IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager () const override
 
IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
 
IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry) const override
 
IBackendInternal::IBackendContextPtr CreateBackendContext (const IRuntime::CreationOptions &) const override
 Create the runtime context of the backend. More...
 
IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext (const IRuntime::CreationOptions &creationOptions, IBackendProfilingPtr &backendProfiling) override
 Create context specifically used for profiling interaction from backends. More...
 
IBackendInternal::ILayerSupportSharedPtr GetLayerSupport () const override
 
OptimizationViews OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
 
std::vector< ITensorHandleFactory::FactoryIdGetHandleFactoryPreferences () const override
 (Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
 
void RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry) override
 (Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
 
std::unique_ptr< ICustomAllocatorGetDefaultAllocator () const override
 Returns the default memory allocator for the backend. More...
 
- Public Member Functions inherited from IBackendInternal
 ~IBackendInternal () override=default
 Allow backends created by the factory function to be destroyed through IBackendInternal. More...
 
virtual IWorkloadFactoryPtr CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const
 
virtual IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const
 
virtual IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const
 
virtual IBackendSpecificModelContextPtr CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const
 
virtual ILayerSupportSharedPtr GetLayerSupport (const ModelOptions &modelOptions) const
 
virtual OptimizationViews OptimizeSubgraphView (const SubgraphView &subgraph) const
 
bool SupportsTensorAllocatorAPI () const
 
ITensorHandleFactory::FactoryId GetBackwardCompatibleFavoriteHandleFactory ()
 
virtual void RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags)
 (Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
 
virtual BackendCapabilities GetCapabilities () const
 Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
 
virtual bool UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & > errMsg)
 Signals the backend to use a custom memory allocator provided by the user. More...
 
virtual unsigned int GetNumberOfCacheFiles () const
 Returns the number of files cached if backend supports caching. More...
 
virtual ExecutionData CreateExecutionData (WorkingMemDescriptor &workingMemDescriptor) const
 Returns ExecutionData for the backend. More...
 
virtual void UpdateExecutionData (ExecutionData &executionData, WorkingMemDescriptor &workingMemDescriptor) const
 Update the ExecutionData for a layer. More...
 

Static Public Member Functions

static const BackendIdGetIdStatic ()
 
- Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion GetApiVersion ()
 Returns the version of the Backend API. More...
 

Additional Inherited Members

- Public Types inherited from IBackendInternal
using IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
 
using IBackendContextPtr = std::unique_ptr< IBackendContext >
 
using IBackendProfilingContextPtr = std::shared_ptr< arm::pipe::IBackendProfilingContext >
 This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
 
using IBackendProfilingPtr = std::unique_ptr< arm::pipe::IBackendProfiling >
 
using ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
 
using IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
 
using IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
 
using IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
 
- Protected Member Functions inherited from IBackendInternal
 IBackendInternal ()=default
 Creation must be done through a specific backend interface. More...
 
- Protected Member Functions inherited from IBackend
 IBackend ()
 
virtual ~IBackend ()
 

Detailed Description

Definition at line 12 of file TosaRefBackend.hpp.

Constructor & Destructor Documentation

◆ TosaRefBackend()

TosaRefBackend ( )
default

◆ ~TosaRefBackend()

~TosaRefBackend ( )
default

Member Function Documentation

◆ CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions ) const
overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 60 of file TosaRefBackend.cpp.

61 {
62  return IBackendContextPtr{};
63 }

◆ CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions creationOptions,
IBackendProfilingPtr backendProfiling 
)
overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 65 of file TosaRefBackend.cpp.

67 {
69 }

◆ CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 71 of file TosaRefBackend.cpp.

72 {
73  return std::make_unique<TosaRefMemoryManager>();
74 }

◆ CreateWorkloadFactory() [1/2]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry tensorHandleFactoryRegistry) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 44 of file TosaRefBackend.cpp.

46 {
47  auto memoryManager = std::make_shared<TosaRefMemoryManager>();
48 
49  tensorHandleFactoryRegistry.RegisterMemoryManager(memoryManager);
50 
51  auto factory = std::make_unique<TosaRefTensorHandleFactory>(memoryManager);
52  // Register copy and import factory pair
53  tensorHandleFactoryRegistry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
54  // Register the factory
55  tensorHandleFactoryRegistry.RegisterFactory(std::move(factory));
56 
57  return std::make_unique<TosaRefWorkloadFactory>(PolymorphicPointerDowncast<TosaRefMemoryManager>(memoryManager));
58 }

References TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

◆ CreateWorkloadFactory() [2/2]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr memoryManager = nullptr) const
overridevirtual

Implements IBackendInternal.

Definition at line 38 of file TosaRefBackend.cpp.

40 {
41  return std::make_unique<TosaRefWorkloadFactory>(PolymorphicPointerDowncast<TosaRefMemoryManager>(memoryManager));
42 }

◆ GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const
overridevirtual

Returns the default memory allocator for the backend.

Returns
- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 187 of file TosaRefBackend.cpp.

188 {
189  return std::make_unique<DefaultAllocator>();
190 }

◆ GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const
overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 168 of file TosaRefBackend.cpp.

169 {
170  return std::vector<ITensorHandleFactory::FactoryId> { TosaRefTensorHandleFactory::GetIdStatic() };
171 }

References TosaRefTensorHandleFactory::GetIdStatic().

◆ GetId()

const BackendId& GetId ( ) const
inlineoverridevirtual

Implements IBackend.

Definition at line 19 of file TosaRefBackend.hpp.

20  {
21  return GetIdStatic();
22  }

References TosaRefBackend::GetIdStatic().

Referenced by TosaRefBackend::OptimizeSubgraphView().

◆ GetIdStatic()

const BackendId & GetIdStatic ( )
static

Definition at line 32 of file TosaRefBackend.cpp.

33 {
34  static const BackendId s_Id{TosaRefBackendId()};
35  return s_Id;
36 }

References armnn::TosaRefBackendId().

Referenced by TosaRefBackend::GetId().

◆ GetLayerSupport()

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const
overridevirtual

Implements IBackendInternal.

Definition at line 76 of file TosaRefBackend.cpp.

77 {
78  static ILayerSupportSharedPtr layerSupport{new TosaRefLayerSupport};
79  return layerSupport;
80 }

◆ OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView subgraph,
const ModelOptions modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 82 of file TosaRefBackend.cpp.

84 {
85  OptimizationViews optimizationViews(modelOptions);
86 
87  auto handler = std::make_unique<TosaSerializationHandler>();
88 
89  std::vector<std::string> graphInputs;
90  std::vector<std::string> graphOutputs;
91 
92  std::vector<TosaSerializationOperator*> operators;
93  std::vector<TosaSerializationTensor*> tensors;
94 
95  auto it = subgraph.begin();
96  while (it != subgraph.end())
97  {
98  Layer& base = *(PolymorphicDowncast<Layer*>(*it));
99 
100  if(base.GetType() == armnn::LayerType::Input ||
102  {
103  ++it;
104  continue;
105  }
106 
107  tosa::TosaSerializationBasicBlock* mappings = GetTosaMappingFromLayer(&base);
108 
109  // Loop through inputs to see if there are any graph inputs, if so save them.
110  // If it's an input to the graph "input" can be found in the string.
111  for (const std::string& blockInputName : mappings->GetInputs())
112  {
113  if (blockInputName.find("input") != std::string::npos)
114  {
115  graphInputs.push_back(blockInputName);
116  }
117  }
118 
119  // Loop through outputs to see if there are any graph outputs, if so save them.
120  // If it's an output to the graph "output" can be found in the string.
121  for (const std::string& blockOutputName : mappings->GetOutputs())
122  {
123  if (blockOutputName.find("output") != std::string::npos)
124  {
125  graphOutputs.push_back(blockOutputName);
126  }
127  }
128 
129  auto blockOperators = mappings->GetOperators();
130  operators.insert(operators.end(), blockOperators.begin(), blockOperators.end());
131 
132  auto blockTensors = mappings->GetTensors();
133  tensors.insert(tensors.end(), blockTensors.begin(), blockTensors.end());
134 
135  ++it;
136  }
137 
138  // Add all mappings to main block.
139  auto* block = new TosaSerializationBasicBlock("main", "main", operators, tensors, graphInputs, graphOutputs);
140 
141  std::vector<TosaSerializationBasicBlock*> blocks;
142  blocks.emplace_back(block);
143 
144  // Add blocks to the main region.
145  auto* region = new TosaSerializationRegion("main", blocks);
146  handler->GetRegions().emplace_back(region);
147 
148  auto compiledBlob =
149  std::make_unique<PreCompiledObjectPtr>(handler.release(), DeleteAsType<TosaSerializationHandler>);
150 
151  IConnectableLayer* preCompiledLayer = optimizationViews.GetINetwork()->AddPrecompiledLayer(
153  std::move(*compiledBlob),
155  "TOSA_Pre_Compiled_Layer");
156 
157  // Copy the output tensor infos from sub-graph
158  for (unsigned int i = 0; i < subgraph.GetNumOutputSlots(); i++)
159  {
160  preCompiledLayer->GetOutputSlot(i).SetTensorInfo(subgraph.GetIOutputSlot(i)->GetTensorInfo());
161  }
162 
163  optimizationViews.AddSubstitution({ std::move(subgraph), SubgraphView(preCompiledLayer) });
164  return optimizationViews;
165 }

References INetwork::AddPrecompiledLayer(), OptimizationViews::AddSubstitution(), SubgraphView::begin(), SubgraphView::end(), TosaRefBackend::GetId(), OptimizationViews::GetINetwork(), SubgraphView::GetIOutputSlot(), SubgraphView::GetNumInputSlots(), SubgraphView::GetNumOutputSlots(), IConnectableLayer::GetOutputSlot(), IOutputSlot::GetTensorInfo(), GetTosaMappingFromLayer(), Layer::GetType(), armnn::Input, armnn::Output, and IOutputSlot::SetTensorInfo().

◆ RegisterTensorHandleFactories()

void RegisterTensorHandleFactories ( class TensorHandleFactoryRegistry )
overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 173 of file TosaRefBackend.cpp.

174 {
175  auto memoryManager = std::make_shared<TosaRefMemoryManager>();
176 
177  registry.RegisterMemoryManager(memoryManager);
178 
179  auto factory = std::make_unique<TosaRefTensorHandleFactory>(memoryManager);
180 
181  // Register copy and import factory pair
182  registry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
183  // Register the factory
184  registry.RegisterFactory(std::move(factory));
185 }

References TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().


The documentation for this class was generated from the following files:
armnn::TosaRefBackendId
constexpr const char * TosaRefBackendId()
Definition: TosaRefBackendId.hpp:10
armnn::Optional
Definition: Optional.hpp:270
armnn::SubgraphView::GetIOutputSlot
const IOutputSlot * GetIOutputSlot(unsigned int index) const
Definition: SubgraphView.cpp:256
armnn::SubgraphView::GetNumInputSlots
unsigned int GetNumInputSlots() const
Definition: SubgraphView.cpp:271
armnn::IOutputSlot::GetTensorInfo
virtual const TensorInfo & GetTensorInfo() const =0
armnn::TosaRefLayerSupport
Definition: TosaRefLayerSupport.hpp:12
armnn::IBackendInternal::IBackendContextPtr
std::unique_ptr< IBackendContext > IBackendContextPtr
Definition: IBackendInternal.hpp:90
armnn::Layer
Definition: Layer.hpp:230
GetTosaMappingFromLayer
TosaSerializationBasicBlock * GetTosaMappingFromLayer(const Layer *layer)
Definition: TosaMappings.cpp:213
armnn::TosaRefBackend::GetId
const BackendId & GetId() const override
Definition: TosaRefBackend.hpp:19
armnn::SubgraphView::begin
IConnectableLayerIterator begin()
Definition: SubgraphView.cpp:286
armnn::IOutputSlot::SetTensorInfo
virtual void SetTensorInfo(const TensorInfo &tensorInfo)=0
armnn::SubgraphView
The SubgraphView class represents a subgraph of a Graph.
Definition: SubgraphView.hpp:31
armnn::OptimizationViews
Definition: OptimizationViews.hpp:17
armnn::TosaRefTensorHandleFactory::GetIdStatic
static const FactoryId & GetIdStatic()
Definition: TosaRefTensorHandleFactory.cpp:16
armnn::TosaRefBackend::GetIdStatic
static const BackendId & GetIdStatic()
Definition: TosaRefBackend.cpp:32
armnn::IBackendInternal::IBackendProfilingContextPtr
std::shared_ptr< arm::pipe::IBackendProfilingContext > IBackendProfilingContextPtr
This is the bridge between backend and backend profiling we'll keep it in the backend namespace.
Definition: IBackendInternal.hpp:92
armnn::Layer::GetType
LayerType GetType() const override
Returns the armnn::LayerType of this layer.
Definition: Layer.hpp:286
armnn::SubgraphView::GetNumOutputSlots
unsigned int GetNumOutputSlots() const
Definition: SubgraphView.cpp:276
armnn::SubgraphView::end
IConnectableLayerIterator end()
Definition: SubgraphView.cpp:291
armnn::BackendId
Definition: BackendId.hpp:75
armnn::IConnectableLayer::GetOutputSlot
virtual const IOutputSlot & GetOutputSlot(unsigned int index) const =0
Get the const output slot handle by slot index.
armnn::IConnectableLayer
Interface for a layer that is connectable to other layers via InputSlots and OutputSlots.
Definition: INetwork.hpp:80
armnn::ILayerSupportSharedPtr
std::shared_ptr< ILayerSupport > ILayerSupportSharedPtr
Definition: ILayerSupport.hpp:40
armnn::LayerType::Input
@ Input
armnn::PreCompiledDescriptor
A PreCompiledDescriptor for the PreCompiledLayer.
Definition: Descriptors.hpp:1367
armnn::LayerType::Output
@ Output