ArmNN
 26.07
Public Member Functions | Static Public Member Functions | List of all members
NeonBackend Class Reference

#include <NeonBackend.hpp>

Inheritance diagram for NeonBackend:
[legend]
Collaboration diagram for NeonBackend:
[legend]

Public Member Functions

 NeonBackend ()=default
 
 ~NeonBackend ()=default
 
const BackendId & GetId () const override
 
IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager () const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry) const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const override
 
IBackendInternal::IBackendContextPtr CreateBackendContext (const IRuntime::CreationOptions &) const override
 Create the runtime context of the backend. More...
 
IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext (const IRuntime::CreationOptions &, IBackendProfilingPtr &backendProfiling) override
 Create context specifically used for profiling interaction from backends. More...
 
IBackendInternal::ILayerSupportSharedPtr GetLayerSupport () const override
 
IBackendInternal::ILayerSupportSharedPtr GetLayerSupport (const ModelOptions &modelOptions) const override
 
OptimizationViews OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
 
std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences () const override
 (Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
 
void RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry) override
 (Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
 
IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const override
 
BackendCapabilities GetCapabilities () const override
 Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
 
std::unique_ptr< ICustomAllocator > GetDefaultAllocator () const override
 Returns the default memory allocator for the backend. More...
 
- Public Member Functions inherited from IBackendInternal
 ~IBackendInternal () override=default
 Allow backends created by the factory function to be destroyed through IBackendInternal. More...
 
virtual IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const
 
virtual OptimizationViews OptimizeSubgraphView (const SubgraphView &subgraph) const
 
bool SupportsTensorAllocatorAPI () const
 
ITensorHandleFactory::FactoryId GetBackwardCompatibleFavoriteHandleFactory ()
 
virtual void RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags)
 (Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
 
virtual bool UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & > errMsg)
 Signals the backend to use a custom memory allocator provided by the user. More...
 
virtual unsigned int GetNumberOfCacheFiles () const
 Returns the number of files cached if backend supports caching. More...
 

Static Public Member Functions

static const BackendId & GetIdStatic ()
 
- Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion GetApiVersion ()
 Returns the version of the Backend API. More...
 

Additional Inherited Members

- Public Types inherited from IBackendInternal
using IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
 
using IBackendContextPtr = std::unique_ptr< IBackendContext >
 
using IBackendProfilingContextPtr = std::shared_ptr< arm::pipe::IBackendProfilingContext >
 This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
 
using IBackendProfilingPtr = std::unique_ptr< arm::pipe::IBackendProfiling >
 
using ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
 
using IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
 
using IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
 
using IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
 
- Protected Member Functions inherited from IBackendInternal
 IBackendInternal ()=default
 Creation must be done through a specific backend interface. More...
 
- Protected Member Functions inherited from IBackend
 IBackend ()
 
virtual ~IBackend ()
 

Detailed Description

Definition at line 29 of file NeonBackend.hpp.

Constructor & Destructor Documentation

◆ NeonBackend()

NeonBackend ( )
default

◆ ~NeonBackend()

~NeonBackend ( )
default

Member Function Documentation

◆ CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions &  ) const
overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 109 of file NeonBackend.cpp.

110 {
111  return IBackendContextPtr{};
112 }
armnn::IBackendInternal::IBackendContextPtr
std::unique_ptr< IBackendContext > IBackendContextPtr
Definition: IBackendInternal.hpp:88

◆ CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions &  creationOptions,
IBackendProfilingPtr &  backendProfiling 
)
overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 114 of file NeonBackend.cpp.

116 {
117  return IBackendProfilingContextPtr{};
118 }
armnn::IBackendInternal::IBackendProfilingContextPtr
std::shared_ptr< arm::pipe::IBackendProfilingContext > IBackendProfilingContextPtr
This is the bridge between backend and backend profiling we'll keep it in the backend namespace.
Definition: IBackendInternal.hpp:90

◆ CreateBackendSpecificModelContext()

IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext ( const ModelOptions &  modelOptions) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 120 of file NeonBackend.cpp.

122 {
123  return IBackendSpecificModelContextPtr{new NeonBackendModelContext{modelOptions}};
124 }
armnn::IBackendInternal::IBackendSpecificModelContextPtr
std::shared_ptr< IBackendModelContext > IBackendSpecificModelContextPtr
Definition: IBackendInternal.hpp:94

Referenced by NeonBackend::CreateWorkloadFactory(), and NeonBackend::GetLayerSupport().

◆ CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 52 of file NeonBackend.cpp.

53 {
54  return std::make_unique<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
55  BaseMemoryManager::MemoryAffinity::Offset);
56 }

References BaseMemoryManager::Offset.

◆ CreateWorkloadFactory() [1/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 72 of file NeonBackend.cpp.

74 {
75  auto memoryManager = std::make_shared<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
76  BaseMemoryManager::MemoryAffinity::Offset);
77 
78  tensorHandleFactoryRegistry.RegisterMemoryManager(memoryManager);
79 
80  auto factory = std::make_unique<NeonTensorHandleFactory>(memoryManager);
81  // Register copy and import factory pair
82  tensorHandleFactoryRegistry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
83  // Register the factory
84  tensorHandleFactoryRegistry.RegisterFactory(std::move(factory));
85 
86 
87  return std::make_unique<NeonWorkloadFactory>(
88  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager));
89 }

References BaseMemoryManager::Offset, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

◆ CreateWorkloadFactory() [2/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry,
const ModelOptions &  modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 91 of file NeonBackend.cpp.

93 {
94  auto memoryManager = std::make_shared<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
95  BaseMemoryManager::MemoryAffinity::Offset);
96 
97  tensorHandleFactoryRegistry.RegisterMemoryManager(memoryManager);
98 
99  auto factory = std::make_unique<NeonTensorHandleFactory>(memoryManager);
100  // Register copy and import factory pair
101  tensorHandleFactoryRegistry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
102  // Register the factory
103  tensorHandleFactoryRegistry.RegisterFactory(std::move(factory));
104 
105  return std::make_unique<NeonWorkloadFactory>(
106  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
107 }
armnn::NeonBackend::CreateBackendSpecificModelContext
IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext(const ModelOptions &modelOptions) const override
Definition: NeonBackend.cpp:120

References NeonBackend::CreateBackendSpecificModelContext(), BaseMemoryManager::Offset, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

◆ CreateWorkloadFactory() [3/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr &  memoryManager = nullptr) const
overridevirtual

Implements IBackendInternal.

Definition at line 58 of file NeonBackend.cpp.

60 {
61  return std::make_unique<NeonWorkloadFactory>(
62  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager));
63 }

◆ CreateWorkloadFactory() [4/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IMemoryManagerSharedPtr &  memoryManager,
const ModelOptions &  modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 65 of file NeonBackend.cpp.

67 {
68  return std::make_unique<NeonWorkloadFactory>(
69  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
70 }

References NeonBackend::CreateBackendSpecificModelContext().

◆ GetCapabilities()

BackendCapabilities GetCapabilities ( ) const
inlineoverridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 68 of file NeonBackend.hpp.

69  {
70  return cpuAccCapabilities;
71  };
armnn::cpuAccCapabilities
const BackendCapabilities cpuAccCapabilities("CpuAcc", { {"NonConstWeights", true}, {"ProtectedContentAllocation", false}, {"ConstantTensorsAsInputs", true}, {"PreImportIOTensors", false}, {"ExternallyManagedMemory", true}, {"MultiAxisPacking", false}, {"SingleAxisPacking", true}, {"HasFp16", arm_compute::CPUInfo::get().has_fp16()}, {"AllOrNothing", false} })

References armnn::cpuAccCapabilities.

◆ GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const
overridevirtual

Returns the default memory allocator for the backend.

Returns
- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 639 of file NeonBackend.cpp.

640 {
641  return std::make_unique<DefaultAllocator>();
642 }

◆ GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const
overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 620 of file NeonBackend.cpp.

621 {
622  return std::vector<ITensorHandleFactory::FactoryId>() = { NeonTensorHandleFactory::GetIdStatic() };
623 }
armnn::NeonTensorHandleFactory::GetIdStatic
static const FactoryId & GetIdStatic()
Definition: NeonTensorHandleFactory.cpp:89

References NeonTensorHandleFactory::GetIdStatic().

◆ GetId()

const BackendId& GetId ( ) const
inlineoverridevirtual

Implements IBackend.

Definition at line 36 of file NeonBackend.hpp.

36 { return GetIdStatic(); }
armnn::NeonBackend::GetIdStatic
static const BackendId & GetIdStatic()
Definition: NeonBackend.cpp:46

References NeonBackend::GetIdStatic().

◆ GetIdStatic()

const BackendId & GetIdStatic ( )
static

Definition at line 46 of file NeonBackend.cpp.

47 {
48  static const BackendId s_Id{NeonBackendId()};
49  return s_Id;
50 }
armnn::NeonBackendId
constexpr const char * NeonBackendId()
Definition: NeonBackendId.hpp:10

References armnn::NeonBackendId().

Referenced by NeonBackend::GetId().

◆ GetLayerSupport() [1/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const
overridevirtual

Implements IBackendInternal.

Definition at line 126 of file NeonBackend.cpp.

127 {
128  static ILayerSupportSharedPtr layerSupport
129  {
130  new NeonLayerSupport(IBackendInternal::IBackendSpecificModelContextPtr{})
131  };
132  return layerSupport;
133 }
armnn::IBackendInternal::ILayerSupportSharedPtr
std::shared_ptr< ILayerSupport > ILayerSupportSharedPtr
Definition: IBackendInternal.hpp:92

◆ GetLayerSupport() [2/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( const ModelOptions &  modelOptions) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 135 of file NeonBackend.cpp.

136 {
137  return ILayerSupportSharedPtr{new NeonLayerSupport(CreateBackendSpecificModelContext(modelOptions))};
138 }

References NeonBackend::CreateBackendSpecificModelContext().

◆ OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView &  subgraph,
const ModelOptions &  modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 140 of file NeonBackend.cpp.

142 {
143  OptimizationViews optimizationViews(modelOptions);
144 
145  auto it = subgraph.end();
146  std::map<LayerGuid, Layer*> untouched;
147 
148  while (it != subgraph.begin())
149  {
150  --it;
151  Layer& base = *(PolymorphicDowncast<Layer*>(*it));
152  untouched.insert({base.GetGuid(), &base});
153  }
154 
155  it = subgraph.end();
156  while (it != subgraph.begin())
157  {
158  --it;
159  Layer& base = *(PolymorphicDowncast<Layer*>(*it));
160 
161  // Fuse activation into previous layer if supported by backend
162  if ((base.GetType() == LayerType::DepthwiseConvolution2d || base.GetType() == LayerType::Convolution2d
163  || base.GetType() == LayerType::BatchNormalization || base.GetType() == LayerType::FullyConnected
164  || base.GetType() == LayerType::Addition || base.GetType() == LayerType::Multiplication
165  || base.GetType() == LayerType::Subtraction || base.GetType() == LayerType::Division
166  || base.GetType() == LayerType::ElementwiseBinary)
167  && (base.GetAdditionalInformation<ActivationDescriptor>() == nullptr))
168  {
169  for (auto output = base.BeginOutputSlots(); output != base.EndOutputSlots(); ++output)
170  {
171  if (output->GetNumConnections() == 1)
172  {
173  for (auto&& childInput : output->GetConnections())
174  {
175  if ((childInput->GetOwningLayer().GetType() == LayerType::Activation) &&
176  (checkDataTypeInputandOutput(childInput->GetOwningLayer())))
177  {
178  Layer& child = childInput->GetOwningLayer();
179 
180  auto* activationLayer = PolymorphicDowncast<ActivationLayer*>(&child);
181  // Before we proceed make sure that this activation layer is in the subgraph. It could be
182  // the first layer in the next subgraph.
183  if (untouched.find(activationLayer->GetGuid()) == untouched.end())
184  {
185  // We can't fuse a layer that's outside the subgraph.
186  break;
187  }
188  const std::string name = std::string("fused-") + child.GetName() + std::string("-into-") +
189  base.GetName();
190 
191  // Get params from activation layer
192  ActivationDescriptor activationDesc = activationLayer->GetParameters();
193 
194  if (base.GetType() == LayerType::Convolution2d)
195  {
196  Convolution2dLayer* baseLayer = PolymorphicDowncast<Convolution2dLayer*>(&base);
197 
198  Optional<TensorInfo> biases;
199 
200  if (baseLayer->GetParameters().m_BiasEnabled)
201  {
202  biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
203  }
204 
205  arm_compute::Status status = NeonConvolution2dWorkloadValidate(
206  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
207  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
208  baseLayer->GetParameters(),
209  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
210  biases,
211  false,
212  &activationDesc);
213 
214  if (status)
215  {
216  FuseConvolution2dLayer<Convolution2dLayer>(optimizationViews,
217  baseLayer,
218  activationLayer,
219  activationDesc,
220  name);
221  untouched.erase(baseLayer->GetGuid());
222  untouched.erase(activationLayer->GetGuid());
223  }
224  }
225  else if (base.GetType() == LayerType::DepthwiseConvolution2d)
226  {
227  DepthwiseConvolution2dLayer* baseLayer =
228  PolymorphicDowncast<DepthwiseConvolution2dLayer*>(&base);
229 
230  Optional<TensorInfo> biases;
231 
232  if (baseLayer->GetParameters().m_BiasEnabled)
233  {
234  biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
235  }
236 
237  arm_compute::Status status = NeonDepthwiseConvolutionWorkloadValidate(
238  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
239  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
240  baseLayer->GetParameters(),
241  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
242  biases,
243  &activationDesc);
244 
245  if (status)
246  {
247  FuseDepthwiseConvolution2dLayer<DepthwiseConvolution2dLayer>(optimizationViews,
248  baseLayer,
249  activationLayer,
250  activationDesc,
251  name);
252  untouched.erase(baseLayer->GetGuid());
253  untouched.erase(activationLayer->GetGuid());
254  }
255  }
256  else if (base.GetType() == LayerType::FullyConnected)
257  {
258  FullyConnectedLayer* baseLayer = PolymorphicDowncast<FullyConnectedLayer*>(&base);
259  FullyConnectedDescriptor descriptor = baseLayer->GetParameters();
260 
261  // As bias is optional only try to get TensorInfo from input if bias is enabled.
262  Optional<TensorInfo> biases;
263  if (descriptor.m_BiasEnabled)
264  {
265  biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
266  }
267 
268  arm_compute::Status status = NeonFullyConnectedWorkloadValidate(
269  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
270  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
271  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
272  biases,
273  baseLayer->GetParameters(),
274  &activationDesc);
275 
276  if (status)
277  {
278  FuseFullyConnectedLayer<FullyConnectedLayer>(optimizationViews,
279  baseLayer,
280  activationLayer,
281  activationDesc,
282  name);
283  untouched.erase(baseLayer->GetGuid());
284  untouched.erase(activationLayer->GetGuid());
285  }
286  }
287  else if (base.GetType() == LayerType::BatchNormalization)
288  {
289  BatchNormalizationLayer* baseLayer =
290  PolymorphicDowncast<BatchNormalizationLayer*>(&base);
291 
292  arm_compute::Status status = NeonBatchNormalizationValidate(
293  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
294  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
295  baseLayer->m_Mean->GetTensorInfo(),
296  baseLayer->m_Variance->GetTensorInfo(),
297  baseLayer->m_Beta->GetTensorInfo(),
298  baseLayer->m_Gamma->GetTensorInfo(),
299  baseLayer->GetParameters(),
300  &activationDesc);
301 
302  if (status)
303  {
304  BatchNormalizationLayer* replacementLayer =
305  FuseBatchNormalizationLayer<BatchNormalizationLayer>(optimizationViews,
306  baseLayer,
307  activationLayer,
308  activationDesc,
309  name);
310 
311  replacementLayer->m_Beta = std::move(baseLayer->m_Beta);
312  replacementLayer->m_Gamma = std::move(baseLayer->m_Gamma);
313  replacementLayer->m_Mean = std::move(baseLayer->m_Mean);
314  replacementLayer->m_Variance = std::move(baseLayer->m_Variance);
315  untouched.erase(baseLayer->GetGuid());
316  untouched.erase(activationLayer->GetGuid());
317  }
318  }
319  else if (base.GetType() == LayerType::Addition)
320  {
321  AdditionLayer* baseLayer = PolymorphicDowncast<AdditionLayer*>(&base);
322 
323  arm_compute::Status status = NeonAdditionWorkloadValidate(
324  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
325  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
326  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
327  &activationDesc);
328 
329  if (status)
330  {
331  FuseAdditionLayer<AdditionLayer>(optimizationViews,
332  baseLayer,
333  activationLayer,
334  activationDesc,
335  name);
336  untouched.erase(baseLayer->GetGuid());
337  untouched.erase(activationLayer->GetGuid());
338  }
339  }
340  else if (base.GetType() == LayerType::Division)
341  {
342  DivisionLayer* baseLayer = PolymorphicDowncast<DivisionLayer*>(&base);
343 
344  arm_compute::Status status = NeonDivisionWorkloadValidate(
345  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
346  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
347  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
348  &activationDesc);
349 
350  if (status)
351  {
352  FuseDivisionLayer<DivisionLayer>(optimizationViews,
353  baseLayer,
354  activationLayer,
355  activationDesc,
356  name);
357  untouched.erase(baseLayer->GetGuid());
358  untouched.erase(activationLayer->GetGuid());
359  }
360  }
361  else if (base.GetType() == LayerType::Multiplication)
362  {
363  MultiplicationLayer* baseLayer = PolymorphicDowncast<MultiplicationLayer*>(&base);
364 
365  arm_compute::Status status = NeonMultiplicationWorkloadValidate(
366  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
367  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
368  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
369  &activationDesc);
370 
371  if (status)
372  {
373  FuseMultiplicationLayer<MultiplicationLayer>(optimizationViews,
374  baseLayer,
375  activationLayer,
376  activationDesc,
377  name);
378  untouched.erase(baseLayer->GetGuid());
379  untouched.erase(activationLayer->GetGuid());
380  }
381  }
382  else if (base.GetType() == LayerType::Subtraction)
383  {
384  SubtractionLayer* baseLayer = PolymorphicDowncast<SubtractionLayer*>(&base);
385 
386  arm_compute::Status status = NeonSubtractionWorkloadValidate(
387  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
388  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
389  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
390  &activationDesc);
391 
392  if (status)
393  {
394  FuseSubtractionLayer<SubtractionLayer>(optimizationViews,
395  baseLayer,
396  activationLayer,
397  activationDesc,
398  name);
399  untouched.erase(baseLayer->GetGuid());
400  untouched.erase(activationLayer->GetGuid());
401  }
402  }
403  else if (base.GetType() == LayerType::ElementwiseBinary)
404  {
405  ElementwiseBinaryLayer* baseLayer = PolymorphicDowncast<ElementwiseBinaryLayer*>(&base);
406 
407  if (baseLayer->GetParameters().m_Operation == BinaryOperation::Add)
408  {
409  arm_compute::Status status = NeonAdditionWorkloadValidate(
410  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
411  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
412  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
413  &activationDesc);
414 
415  if (status)
416  {
417  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
418  baseLayer,
419  activationLayer,
420  activationDesc,
421  BinaryOperation::Add,
422  name);
423  untouched.erase(baseLayer->GetGuid());
424  untouched.erase(activationLayer->GetGuid());
425  }
426  }
427  else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Div)
428  {
429  arm_compute::Status status = NeonDivisionWorkloadValidate(
430  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
431  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
432  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
433  &activationDesc);
434 
435  if (status)
436  {
437  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
438  baseLayer,
439  activationLayer,
440  activationDesc,
441  BinaryOperation::Div,
442  name);
443  untouched.erase(baseLayer->GetGuid());
444  untouched.erase(activationLayer->GetGuid());
445  }
446  }
447  else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Mul)
448  {
449  arm_compute::Status status = NeonMultiplicationWorkloadValidate(
450  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
451  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
452  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
453  &activationDesc);
454 
455  if (status)
456  {
457  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
458  baseLayer,
459  activationLayer,
460  activationDesc,
461  BinaryOperation::Mul,
462  name);
463  untouched.erase(baseLayer->GetGuid());
464  untouched.erase(activationLayer->GetGuid());
465  }
466  }
467  else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Sub)
468  {
469  arm_compute::Status status = NeonSubtractionWorkloadValidate(
470  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
471  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
472  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
473  &activationDesc);
474 
475  if (status)
476  {
477  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
478  baseLayer,
479  activationLayer,
480  activationDesc,
481  BinaryOperation::Sub,
482  name);
483  untouched.erase(baseLayer->GetGuid());
484  untouched.erase(activationLayer->GetGuid());
485  }
486  }
487  // No fusion available for other BinaryOperations
488  }
489  }
490  }
491  }
492  }
493  }
494 
495  // Separate reduce layer with multiple axes into multiple reduce layers with 1 axis.
496  if (base.GetType() == LayerType::Reduce)
497  {
498  ReduceLayer* baseLayer = PolymorphicDowncast<ReduceLayer*>(&base);
499  ReduceDescriptor reduceDescriptor = baseLayer->GetParameters();
500 
501  if (!reduceDescriptor.m_vAxis.empty() && reduceDescriptor.m_vAxis.size() > 1)
502  {
503  // Add new layers to the graph and connect them.
504  std::vector<IConnectableLayer*> layers = ChainReduceLayers<ReduceLayer>(optimizationViews,
505  baseLayer,
506  reduceDescriptor);
507 
508  // Replace existing baselayer with new subgraph.
509  ReplaceLayers<ReduceLayer>(optimizationViews, baseLayer, layers);
510  untouched.erase(baseLayer->GetGuid());
511  }
512  }
513 
514  // Remove Reshape where possible
515  if (base.GetType() == LayerType::Reshape)
516  {
517  ReshapeLayer* baseLayer = PolymorphicDowncast<ReshapeLayer*>(&base);
518 
519  // Cannot remove a Reshape if it's connected to any layer that has an NCHW layout
520  if (ConnectedToLayerWithNCHW(baseLayer))
521  {
522  continue;
523  }
524  RemoveReshapeLayer(baseLayer, untouched, optimizationViews);
525  }
526 
527  // Replace Add/Mul/Add where possible
528  Layer* layerList[4] = {nullptr, nullptr, nullptr, nullptr};
529  const std::vector<ActivationFunction> validActivates = { ActivationFunction::ReLu,
530  ActivationFunction::BoundedReLu };
531  if (IsLayerSequence<BinaryOperation>(base,
532  BinaryOperation::Add, BinaryOperation::Mul, BinaryOperation::Add,
533  layerList,
534  true, // handleValidActivates
535  validActivates))
536  {
537  bool fuseReLu = false;
538  unsigned int numInputs = 0;
539  unsigned int numOutputs = 0;
540  std::vector<TensorInfo> inputInfos;
541  std::vector<TensorInfo> outputInfos;
542  const ActivationDescriptor* activationDescriptor = nullptr;
543 
544  if (BuildAddMulAddTensorInfoLists<Layer>(layerList,
545  numInputs,
546  numOutputs,
547  inputInfos,
548  outputInfos,
549  activationDescriptor,
550  fuseReLu))
551  {
552  // Create the new Add/Mul/Add layer and set the Relu activation function
553  FusedDescriptor fusedDescriptor(numInputs, numOutputs, FusedKernelType::AddMulAdd);
554  arm_compute::Status status = NeonFusedWorkloadValidate({inputInfos.begin(), inputInfos.end()},
555  {outputInfos.begin(), outputInfos.end()},
556  fusedDescriptor,
557  activationDescriptor);
558  if (status)
559  {
560  std::string fusedName;
561  GetFusedName(layerList, fusedName);
562 
563  IConnectableLayer* addMulAddLayer =
564  optimizationViews.GetINetwork()->AddFusedLayer(fusedDescriptor, fusedName.c_str());
565 
566  if (fuseReLu)
567  {
568  FusedLayer* addMulAddFusedLayer = PolymorphicDowncast<FusedLayer*>(addMulAddLayer);
569  addMulAddFusedLayer->SetAdditionalInfoForObject(
570  std::make_shared<ActivationDescriptor>(*activationDescriptor));
571  }
572 
573  // Update the graph
574  std::vector<IConnectableLayer*> originalLayers;
575  for (unsigned int layerIdx = 0; layerIdx < 4; ++layerIdx)
576  {
577  if (layerList[layerIdx])
578  {
579  originalLayers.push_back(layerList[layerIdx]);
580  }
581  }
582 
583  std::vector<SlotList> inputLayersSlotLists, outputLayersSlotLists;
584  BuildAddMulAddSlotLists<SlotList>(fuseReLu,
585  outputInfos.size() > 1,
586  inputLayersSlotLists,
587  outputLayersSlotLists);
588 
589  ReplaceMultipleLayers<FusedLayer>(optimizationViews,
590  originalLayers,
591  PolymorphicDowncast<FusedLayer*>(addMulAddLayer),
592  inputLayersSlotLists,
593  outputLayersSlotLists);
594 
595  // Remove unused layers
596  for (unsigned int layerIdx = 0; layerIdx < 4; ++layerIdx)
597  {
598  if (layerList[layerIdx])
599  {
600  untouched.erase(layerList[layerIdx]->GetGuid());
601  }
602  }
603  }
604  }
605  }
606  }
607 
608  if (optimizationViews.GetSubstitutions().empty() && optimizationViews.GetDeletedSubgraphs().empty())
609  {
610  optimizationViews.AddUntouchedSubgraph(SubgraphView(subgraph));
611  }
612  else
613  {
614  ReportUntouchedLayers(optimizationViews, untouched);
615  }
616 
617  return optimizationViews;
618 }
armnn::GetFusedName
void GetFusedName(Layer *layerList[4], std::string &fusedName)
Definition: NeonBackendOptimizationUtils.hpp:71
armnn::NeonAdditionWorkloadValidate
arm_compute::Status NeonAdditionWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonAdditionWorkload.cpp:20
armnn::NeonConvolution2dWorkloadValidate
arm_compute::Status NeonConvolution2dWorkloadValidate(const TensorInfo &input, const TensorInfo &output, const Convolution2dDescriptor &descriptor, const TensorInfo &weights, const Optional< TensorInfo > &biases, bool isFastMathEnabled, const ActivationDescriptor *activationDescriptor)
Definition: NeonConvolution2dWorkload.cpp:24
armnn::ActivationFunction::BoundedReLu
@ BoundedReLu
min(a, max(b, input)) ReLu1 & ReLu6.
armnn::ReportUntouchedLayers
void ReportUntouchedLayers(OptimizationViews &optimizationViews, std::map< LayerGuid, Layer * > untouched)
Definition: SubgraphUtils.hpp:243
armnn::NeonDepthwiseConvolutionWorkloadValidate
arm_compute::Status NeonDepthwiseConvolutionWorkloadValidate(const TensorInfo &input, const TensorInfo &output, const DepthwiseConvolution2dDescriptor &descriptor, const TensorInfo &weights, const Optional< TensorInfo > &biases, const ActivationDescriptor *activationDescriptor)
Definition: NeonDepthwiseConvolutionWorkload.cpp:29
armnn::NeonDivisionWorkloadValidate
arm_compute::Status NeonDivisionWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonDivisionWorkload.cpp:18
armnn::Status
Status
enumeration
Definition: Types.hpp:43
armnn::NeonSubtractionWorkloadValidate
arm_compute::Status NeonSubtractionWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonSubtractionWorkload.cpp:22
armnn::NeonFusedWorkloadValidate
arm_compute::Status NeonFusedWorkloadValidate(const std::vector< std::reference_wrapper< TensorInfo >> &inputInfos, const std::vector< std::reference_wrapper< TensorInfo >> &outputInfos, const FusedDescriptor &fusedDescriptor, const ActivationDescriptor *activationDescriptor)
Definition: NeonFusedWorkload.cpp:22
armnn::NeonBatchNormalizationValidate
arm_compute::Status NeonBatchNormalizationValidate(const TensorInfo &input, const TensorInfo &output, const TensorInfo &mean, const TensorInfo &var, const TensorInfo &beta, const TensorInfo &gamma, const BatchNormalizationDescriptor &descriptor, const ActivationDescriptor *activationDescriptor)
Definition: NeonBatchNormalizationWorkload.cpp:24
armnn::ConnectedToLayerWithNCHW
bool ConnectedToLayerWithNCHW(Layer *baseLayer)
Checks if the Layer is connected to any Layer that has an NCHW layout.
Definition: SubgraphUtils.hpp:292
armnn::NeonMultiplicationWorkloadValidate
arm_compute::Status NeonMultiplicationWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonMultiplicationWorkload.cpp:19
armnn::NeonFullyConnectedWorkloadValidate
arm_compute::Status NeonFullyConnectedWorkloadValidate(const TensorInfo &input, const TensorInfo &output, const TensorInfo &weights, const Optional< TensorInfo > &biases, const FullyConnectedDescriptor &descriptor, const ActivationDescriptor *activationDescriptor)
Definition: NeonFullyConnectedWorkload.cpp:24
armnn::RemoveReshapeLayer
void RemoveReshapeLayer(ReshapeLayer *baseLayer, std::map< LayerGuid, Layer * > &untouched, OptimizationViews &optimizationViews)
Definition: SubgraphUtils.hpp:335

References armnn::Activation, armnn::Add, INetwork::AddFusedLayer(), armnn::Addition, armnn::AddMulAdd, OptimizationViews::AddUntouchedSubgraph(), armnn::BatchNormalization, SubgraphView::begin(), Layer::BeginOutputSlots(), armnn::BoundedReLu, armnn::ConnectedToLayerWithNCHW(), armnn::Convolution2d, armnn::DepthwiseConvolution2d, armnn::Div, armnn::Division, armnn::ElementwiseBinary, SubgraphView::end(), Layer::EndOutputSlots(), armnn::FullyConnected, Layer::GetAdditionalInformation(), InputSlot::GetConnectedOutputSlot(), OptimizationViews::GetDeletedSubgraphs(), armnn::GetFusedName(), Layer::GetGuid(), OptimizationViews::GetINetwork(), Layer::GetInputSlot(), Layer::GetName(), LayerWithParameters< Parameters >::GetParameters(), OptimizationViews::GetSubstitutions(), OutputSlot::GetTensorInfo(), Layer::GetType(), BatchNormalizationLayer::m_Beta, FullyConnectedDescriptor::m_BiasEnabled, Convolution2dDescriptor::m_BiasEnabled, DepthwiseConvolution2dDescriptor::m_BiasEnabled, BatchNormalizationLayer::m_Gamma, BatchNormalizationLayer::m_Mean, ElementwiseBinaryDescriptor::m_Operation, BatchNormalizationLayer::m_Variance, ReduceDescriptor::m_vAxis, armnn::Mul, armnn::Multiplication, armnn::NeonAdditionWorkloadValidate(), armnn::NeonBatchNormalizationValidate(), armnn::NeonConvolution2dWorkloadValidate(), armnn::NeonDepthwiseConvolutionWorkloadValidate(), armnn::NeonDivisionWorkloadValidate(), armnn::NeonFullyConnectedWorkloadValidate(), armnn::NeonFusedWorkloadValidate(), armnn::NeonMultiplicationWorkloadValidate(), armnn::NeonSubtractionWorkloadValidate(), armnn::Reduce, armnn::ReLu, armnn::RemoveReshapeLayer(), armnn::ReportUntouchedLayers(), armnn::Reshape, Layer::SetAdditionalInfoForObject(), armnn::Sub, and armnn::Subtraction.

◆ RegisterTensorHandleFactories()

void RegisterTensorHandleFactories ( class TensorHandleFactoryRegistry &  )
overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 625 of file NeonBackend.cpp.

626 {
627  auto memoryManager = std::make_shared<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
628  BaseMemoryManager::MemoryAffinity::Offset);
629 
630  registry.RegisterMemoryManager(memoryManager);
631 
632  auto factory = std::make_unique<NeonTensorHandleFactory>(memoryManager);
633  // Register copy and import factory pair
634  registry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
635  // Register the factory
636  registry.RegisterFactory(std::move(factory));
637 }

References BaseMemoryManager::Offset, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

The documentation for this class was generated from the following files: