ClBackend Class Reference

#include <ClBackend.hpp>

Inheritance diagram for ClBackend:

Collaboration diagram for ClBackend:

Classes
class	ClBackendCustomAllocatorMemoryRegion
class	ClBackendCustomAllocatorWrapper

Public Member Functions
	ClBackend ()
	ClBackend (std::shared_ptr< ICustomAllocator > allocator)
	~ClBackend ()=default
const BackendId &	GetId () const override
IBackendInternal::IMemoryManagerUniquePtr	CreateMemoryManager () const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (TensorHandleFactoryRegistry &registry) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const override
std::vector< ITensorHandleFactory::FactoryId >	GetHandleFactoryPreferences () const override
	(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.
IBackendInternal::IBackendContextPtr	CreateBackendContext (const IRuntime::CreationOptions &) const override
	Create the runtime context of the backend.
IBackendInternal::IBackendProfilingContextPtr	CreateBackendProfilingContext (const IRuntime::CreationOptions &, IBackendProfilingPtr &backendProfiling) override
	Create context specifically used for profiling interaction from backends.
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport () const override
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport (const ModelOptions &modelOptions) const override
OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
IBackendInternal::IBackendSpecificModelContextPtr	CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const override
std::unique_ptr< ICustomAllocator >	GetDefaultAllocator () const override
	Returns the default memory allocator for the backend.
BackendCapabilities	GetCapabilities () const override
	Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.
virtual bool	UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & > errMsg) override
	Signals the backend to use a custom memory allocator provided by the user.
virtual unsigned int	GetNumberOfCacheFiles () const override
	Returns the number of files cached if backend supports caching.
Public Member Functions inherited from IBackendInternal
	~IBackendInternal () override=default
	Allow backends created by the factory function to be destroyed through IBackendInternal.
virtual OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph) const
bool	SupportsTensorAllocatorAPI () const
ITensorHandleFactory::FactoryId	GetBackwardCompatibleFavoriteHandleFactory ()

Static Public Member Functions
static const BackendId &	GetIdStatic ()
Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion	GetApiVersion ()
	Returns the version of the Backend API.

Public Attributes
std::shared_ptr< ClBackendCustomAllocatorWrapper >	m_CustomAllocator
bool	m_UsingCustomAllocator = false

Additional Inherited Members
Public Types inherited from IBackendInternal
using	IWorkloadFactoryPtr = std::unique_ptr<IWorkloadFactory>
using	IBackendContextPtr = std::unique_ptr<IBackendContext>
using	IBackendProfilingContextPtr = std::shared_ptr<arm::pipe::IBackendProfilingContext>
	This is the bridge between backend and backend profiling we'll keep it in the backend namespace.
using	IBackendProfilingPtr = std::unique_ptr<arm::pipe::IBackendProfiling>
using	ILayerSupportSharedPtr = std::shared_ptr<ILayerSupport>
using	IBackendSpecificModelContextPtr = std::shared_ptr<IBackendModelContext>
using	IMemoryManagerUniquePtr = std::unique_ptr<IMemoryManager>
using	IMemoryManagerSharedPtr = std::shared_ptr<IMemoryManager>
Protected Member Functions inherited from IBackendInternal
	IBackendInternal ()=default
	Creation must be done through a specific backend interface.
Protected Member Functions inherited from IBackend
	IBackend ()
virtual	~IBackend ()

Detailed Description

Definition at line 24 of file ClBackend.hpp.

Constructor & Destructor Documentation

ClBackend() [1/2]

ClBackend ( )

inline

Definition at line 27 of file ClBackend.hpp.

: m_CustomAllocator(nullptr) {};

References m_CustomAllocator.

ClBackend() [2/2]

ClBackend ( std::shared_ptr< ICustomAllocator > allocator )

inline

Definition at line 28 of file ClBackend.hpp.

    {
        std::string err;
        UseCustomMemoryAllocator(allocator, err);
    }

References UseCustomMemoryAllocator().

~ClBackend()

~ClBackend ( )

default

Member Function Documentation

CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions & ) const

overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 235 of file ClBackend.cpp.

{
    return IBackendContextPtr{new ClBackendContext{options}};
}

CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions & creationOptions, IBackendProfilingPtr & backendProfiling )

overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 240 of file ClBackend.cpp.

{
    return IBackendProfilingContextPtr{};
}

CreateBackendSpecificModelContext()

IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext ( const ModelOptions & modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 246 of file ClBackend.cpp.

{
    return IBackendSpecificModelContextPtr{new ClBackendModelContext{modelOptions}};
}

Referenced by CreateWorkloadFactory(), CreateWorkloadFactory(), CreateWorkloadFactory(), GetLayerSupport(), and OptimizeSubgraphView().

CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 50 of file ClBackend.cpp.

{
    if (m_UsingCustomAllocator)
    {
        return std::make_unique<ClMemoryManager>(m_CustomAllocator);
    }
    return std::make_unique<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
}

References m_CustomAllocator, and m_UsingCustomAllocator.

CreateWorkloadFactory() [1/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry & tensorHandleFactoryRegistry, const ModelOptions & modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 101 of file ClBackend.cpp.

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc));
 
    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));
 
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

References CreateBackendSpecificModelContext(), m_CustomAllocator, m_UsingCustomAllocator, armnn::Malloc, armnn::PolymorphicPointerDowncast(), TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

CreateWorkloadFactory() [2/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry & tensorHandleFactoryRegistry, const ModelOptions & modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 129 of file ClBackend.cpp.

{
    // To allow force import if inputFlags/outputFlags are Undefined, set it as Malloc
    if (inputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        inputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    if (outputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        outputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
            inputFlags, outputFlags);
 
    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));
 
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

References CreateBackendSpecificModelContext(), m_CustomAllocator, m_UsingCustomAllocator, armnn::Malloc, armnn::PolymorphicPointerDowncast(), TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), TensorHandleFactoryRegistry::RegisterMemoryManager(), and armnn::Undefined.

CreateWorkloadFactory() [3/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr & memoryManager = nullptr ) const

overridevirtual

Implements IBackendInternal.

Definition at line 59 of file ClBackend.cpp.

{
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager));
}

References armnn::PolymorphicPointerDowncast().

CreateWorkloadFactory() [4/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IMemoryManagerSharedPtr & memoryManager, const ModelOptions & modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 66 of file ClBackend.cpp.

{
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

References CreateBackendSpecificModelContext(), and armnn::PolymorphicPointerDowncast().

CreateWorkloadFactory() [5/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( TensorHandleFactoryRegistry & registry ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 73 of file ClBackend.cpp.

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc));
 
    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));
 
    return std::make_unique<ClWorkloadFactory>(
            PolymorphicPointerDowncast<ClMemoryManager>(memoryManager));
}

References m_CustomAllocator, m_UsingCustomAllocator, armnn::Malloc, armnn::PolymorphicPointerDowncast(), TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

GetCapabilities()

BackendCapabilities GetCapabilities ( ) const

overridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 275 of file ClBackend.cpp.

{
    // add new capabilities here..
    return BackendCapabilities ("GpuAcc",
                                                 {
                                                     {"NonConstWeights", true},
                                                     {"ProtectedContentAllocation", true},
                                                     {"ConstantTensorsAsInputs", true},
                                                     {"PreImportIOTensors", false},
                                                     {"ExternallyManagedMemory", true},
                                                     {"MultiAxisPacking", false},
                                                     {"SingleAxisPacking", true},
                                                     {"AllOrNothing", false},
                                                     {"HasFp16", arm_compute::CLKernelLibrary::get().fp16_supported()}
                                                 });
}

GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const

overridevirtual

Returns the default memory allocator for the backend.

Returns

- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 270 of file ClBackend.cpp.

{
    return std::make_unique<ClBackendDefaultAllocator>();
}

GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const

overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 169 of file ClBackend.cpp.

{
    return std::vector<ITensorHandleFactory::FactoryId> {ClTensorHandleFactory::GetIdStatic(),
                                                         ClImportTensorHandleFactory::GetIdStatic()};
}

References ClImportTensorHandleFactory::GetIdStatic(), and ClTensorHandleFactory::GetIdStatic().

GetId()

const BackendId & GetId ( ) const

inlineoverridevirtual

Implements IBackend.

Definition at line 36 of file ClBackend.hpp.

{ return GetIdStatic(); }

References GetIdStatic().

GetIdStatic()

const BackendId & GetIdStatic ( )

static

Definition at line 44 of file ClBackend.cpp.

{
    static const BackendId s_Id{ClBackendId()};
    return s_Id;
}

References armnn::ClBackendId().

Referenced by GetId().

GetLayerSupport() [1/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const

overridevirtual

Implements IBackendInternal.

Definition at line 252 of file ClBackend.cpp.

{
    static ILayerSupportSharedPtr layerSupport
        {
            new ClLayerSupport(IBackendInternal::IBackendSpecificModelContextPtr{})
        };
    return layerSupport;
}

GetLayerSupport() [2/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( const ModelOptions & modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 261 of file ClBackend.cpp.

{
    static ILayerSupportSharedPtr layerSupport
    {
        new ClLayerSupport(CreateBackendSpecificModelContext(modelOptions))
    };
    return layerSupport;
}

References CreateBackendSpecificModelContext().

GetNumberOfCacheFiles()

virtual unsigned int GetNumberOfCacheFiles ( ) const

inlineoverridevirtual

Returns the number of files cached if backend supports caching.

Returns

- Returns 0 if backend does not support caching otherwise number of files cached

Reimplemented from IBackendInternal.

Definition at line 94 of file ClBackend.hpp.

{ return 1; }

OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView & subgraph, const ModelOptions & modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 292 of file ClBackend.cpp.

{
    OptimizationViews optimizationViews(modelOptions);
 
    auto it = subgraph.end();
    bool isFastMathEnabled = false;
    std::map<LayerGuid, Layer*> untouched;
 
    while (it != subgraph.begin())
    {
        --it;
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));
        untouched.insert({base.GetGuid(), &base});
    }
 
    it = subgraph.end();
#if defined(ARMCOMPUTECL_ENABLED)
    IBackendInternal::IBackendSpecificModelContextPtr modelContextPtr = CreateBackendSpecificModelContext(modelOptions);
 
    if (modelContextPtr)
    {
        auto clModelOptions = dynamic_cast<ClBackendModelContext*>(modelContextPtr.get());
        if (clModelOptions)
        {
            isFastMathEnabled = clModelOptions->IsFastMathEnabled();
        }
    }
#endif
    while (it != subgraph.begin())
    {
        --it;
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));
 
        // Fuse activation into previous layer if supported by backend
        if ((base.GetType() == LayerType::DepthwiseConvolution2d || base.GetType() == LayerType::Convolution2d
            || base.GetType() == LayerType::BatchNormalization || base.GetType() == LayerType::FullyConnected
            || base.GetType() == LayerType::Addition || base.GetType() == LayerType::Multiplication
            || base.GetType() == LayerType::Subtraction || base.GetType() == LayerType::Division
            || base.GetType() == LayerType::ElementwiseBinary)
            && (base.GetAdditionalInformation<ActivationDescriptor>() == nullptr))
        {
            for (auto output = base.BeginOutputSlots(); output != base.EndOutputSlots(); ++output)
            {
                if (output->GetNumConnections() == 1)
                {
                    for (auto&& childInput : output->GetConnections())
                    {
                        if ((childInput->GetOwningLayer().GetType() == LayerType::Activation) &&
                            (checkDataTypeInputandOutput(childInput->GetOwningLayer())))
                        {
                            Layer& child = childInput->GetOwningLayer();
 
                            auto* activationLayer = PolymorphicDowncast<ActivationLayer*>(&child);
                            // Before we proceed make sure that this activation layer is in the subgraph. It could be
                            // the first layer in the next subgraph.
                            if (untouched.find(activationLayer->GetGuid()) == untouched.end())
                            {
                                // We can't fuse a layer that's outside the subgraph.
                                break;
                            }
 
                            const std::string name = std::string("fused-") + child.GetName() + std::string("-into-") +
                                                     base.GetName();
 
                            // Get params from activation layer
                            ActivationDescriptor activationDesc = activationLayer->GetParameters();
 
                            if (base.GetType() == LayerType::Convolution2d)
                            {
                                Convolution2dLayer* baseLayer = PolymorphicDowncast<Convolution2dLayer*>(&base);
 
                                Optional<TensorInfo> biases;
 
                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
                                }
 
                                arm_compute::Status status = ClConvolution2dWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        biases,
                                        isFastMathEnabled,
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseConvolution2dLayer<Convolution2dLayer>(optimizationViews,
                                                                               baseLayer,
                                                                               activationLayer,
                                                                               activationDesc,
                                                                               name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::DepthwiseConvolution2d)
                            {
                                DepthwiseConvolution2dLayer* baseLayer =
                                        PolymorphicDowncast<DepthwiseConvolution2dLayer*>(&base);
 
                                Optional<TensorInfo> biases;
 
                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
                                }
 
                                arm_compute::Status status = ClDepthwiseConvolutionWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        biases,
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseDepthwiseConvolution2dLayer<DepthwiseConvolution2dLayer>(optimizationViews,
                                                                                                 baseLayer,
                                                                                                 activationLayer,
                                                                                                 activationDesc,
                                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::FullyConnected)
                            {
                                FullyConnectedLayer* baseLayer = PolymorphicDowncast<FullyConnectedLayer*>(&base);
                                FullyConnectedDescriptor descriptor = baseLayer->GetParameters();
 
                                // As bias is optional only try to get TensorInfo from input if bias is enabled.
                                Optional<TensorInfo> biases;
                                if (descriptor.m_BiasEnabled)
                                {
                                    biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
                                }
 
                                arm_compute::Status status = ClFullyConnectedWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        biases,
                                        baseLayer->GetParameters(),
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseFullyConnectedLayer<FullyConnectedLayer>(optimizationViews,
                                                                                 baseLayer,
                                                                                 activationLayer,
                                                                                 activationDesc,
                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::BatchNormalization)
                            {
                                BatchNormalizationLayer* baseLayer =
                                        PolymorphicDowncast<BatchNormalizationLayer*>(&base);
 
                                arm_compute::Status status = ClBatchNormalizationValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->m_Mean->GetTensorInfo(),
                                        baseLayer->m_Variance->GetTensorInfo(),
                                        baseLayer->m_Beta->GetTensorInfo(),
                                        baseLayer->m_Gamma->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        &activationDesc);
 
                                if (status)
                                {
                                    BatchNormalizationLayer* replacementLayer =
                                        FuseBatchNormalizationLayer<BatchNormalizationLayer>(optimizationViews,
                                                                                             baseLayer,
                                                                                             activationLayer,
                                                                                             activationDesc,
                                                                                             name);
 
                                    replacementLayer->m_Beta     = std::move(baseLayer->m_Beta);
                                    replacementLayer->m_Gamma    = std::move(baseLayer->m_Gamma);
                                    replacementLayer->m_Mean     = std::move(baseLayer->m_Mean);
                                    replacementLayer->m_Variance = std::move(baseLayer->m_Variance);
 
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Addition)
                            {
                                AdditionLayer* baseLayer = PolymorphicDowncast<AdditionLayer*>(&base);
 
                                arm_compute::Status status = ClAdditionValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseAdditionLayer<AdditionLayer>(optimizationViews,
                                                                     baseLayer,
                                                                     activationLayer,
                                                                     activationDesc,
                                                                     name);
 
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Division)
                            {
                                DivisionLayer* baseLayer = PolymorphicDowncast<DivisionLayer*>(&base);
 
                                arm_compute::Status status = ClDivisionWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseDivisionLayer<DivisionLayer>(optimizationViews,
                                                                     baseLayer,
                                                                     activationLayer,
                                                                     activationDesc,
                                                                     name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Multiplication)
                            {
                                MultiplicationLayer* baseLayer = PolymorphicDowncast<MultiplicationLayer*>(&base);
 
                                arm_compute::Status status = ClMultiplicationWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseMultiplicationLayer<MultiplicationLayer>(optimizationViews,
                                                                                 baseLayer,
                                                                                 activationLayer,
                                                                                 activationDesc,
                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Subtraction)
                            {
                                SubtractionLayer* baseLayer = PolymorphicDowncast<SubtractionLayer*>(&base);
 
                                arm_compute::Status status = ClSubtractionValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);
 
                                if (status)
                                {
                                    FuseSubtractionLayer<SubtractionLayer>(optimizationViews,
                                                                           baseLayer,
                                                                           activationLayer,
                                                                           activationDesc,
                                                                           name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::ElementwiseBinary)
                            {
                                ElementwiseBinaryLayer* baseLayer = PolymorphicDowncast<ElementwiseBinaryLayer*>(&base);
 
                                if (baseLayer->GetParameters().m_Operation == BinaryOperation::Add)
                                {
                                    arm_compute::Status status = ClAdditionValidate(
                                            baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                            activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            &activationDesc);
 
                                    if (status)
                                    {
                                        FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
                                                                                           baseLayer,
                                                                                           activationLayer,
                                                                                           activationDesc,
                                                                                           BinaryOperation::Add,
                                                                                           name);
                                        untouched.erase(baseLayer->GetGuid());
                                        untouched.erase(activationLayer->GetGuid());
                                    }
                                }
                                else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Div)
                                {
                                    arm_compute::Status status = ClDivisionWorkloadValidate(
                                            baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                            activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            &activationDesc);
 
                                    if (status)
                                    {
                                        FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
                                                                                           baseLayer,
                                                                                           activationLayer,
                                                                                           activationDesc,
                                                                                           BinaryOperation::Div,
                                                                                           name);
                                        untouched.erase(baseLayer->GetGuid());
                                        untouched.erase(activationLayer->GetGuid());
                                    }
                                }
                                else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Mul)
                                {
                                    arm_compute::Status status = ClMultiplicationWorkloadValidate(
                                            baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                            activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            &activationDesc);
 
                                    if (status)
                                    {
                                        FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
                                                                                           baseLayer,
                                                                                           activationLayer,
                                                                                           activationDesc,
                                                                                           BinaryOperation::Mul,
                                                                                           name);
                                        untouched.erase(baseLayer->GetGuid());
                                        untouched.erase(activationLayer->GetGuid());
                                    }
                                }
                                else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Sub)
                                {
                                    arm_compute::Status status = ClSubtractionValidate(
                                            baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                            activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                            &activationDesc);
 
                                    if (status)
                                    {
                                        FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
                                                                                           baseLayer,
                                                                                           activationLayer,
                                                                                           activationDesc,
                                                                                           BinaryOperation::Sub,
                                                                                           name);
                                        untouched.erase(baseLayer->GetGuid());
                                        untouched.erase(activationLayer->GetGuid());
                                    }
                                }
                                // No fusion available for other BinaryOperations
                            }
                        }
                    }
                }
            }
        }
 
        // Separate reduce layer with multiple axes into multiple reduce layers with 1 axis.
        if (base.GetType() == LayerType::Reduce)
        {
            ReduceLayer* baseLayer            = PolymorphicDowncast<ReduceLayer*>(&base);
            ReduceDescriptor reduceDescriptor = baseLayer->GetParameters();
 
            if (!reduceDescriptor.m_vAxis.empty() && reduceDescriptor.m_vAxis.size() > 1)
            {
                // Add new layers to the graph and connect them.
                std::vector<IConnectableLayer*> layers = ChainReduceLayers<ReduceLayer>(optimizationViews,
                                                                                        baseLayer,
                                                                                        reduceDescriptor);
 
                // Replace existing baselayer with new subgraph.
                ReplaceLayers<ReduceLayer>(optimizationViews, baseLayer, layers);
                untouched.erase(baseLayer->GetGuid());
            }
        }
 
        // Remove Reshape where possible
        if (base.GetType() == LayerType::Reshape)
        {
            ReshapeLayer* baseLayer = PolymorphicDowncast<ReshapeLayer*>(&base);
 
            // Cannot remove a Reshape if it's connected to any layer that has an NCHW layout
            if (ConnectedToLayerWithNCHW(baseLayer))
            {
                continue;
            }
            RemoveReshapeLayer(baseLayer, untouched, optimizationViews);
        }
        // Special case to fuse padding into average pooling 2d for quantized datatype.
        // Required to be done as a backend specific optimization as Neon does not support this special case.
        if (base.GetType() == LayerType::Pooling2d)
        {
            Pooling2dLayer* baseLayer = PolymorphicDowncast<Pooling2dLayer*>(&base);
            Pooling2dDescriptor poolingDescriptor = baseLayer->GetParameters();
            if (baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetOwningLayer().GetType() == LayerType::Pad)
            {
                PadLayer* padLayer = PolymorphicDowncast<PadLayer*>(
                    &baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetOwningLayer());
                if (padLayer->GetOutputSlot(0).GetNumConnections() == 1 &&
                    optimizations::pad_fold::TryFoldPadIntoLayer2d(padLayer->GetParameters(),
                                                                   poolingDescriptor,
                                                                   padLayer->GetOutputSlot().GetTensorInfo(),
                                                                   true))
                {
                    FoldPadLayer2d<Pooling2dLayer, Pooling2dDescriptor>(optimizationViews, baseLayer,
                                                             poolingDescriptor, padLayer);
                    untouched.erase(baseLayer->GetGuid());
                    untouched.erase(padLayer->GetGuid());
                }
            }
        }
    }
 
    if (optimizationViews.GetSubstitutions().empty() && optimizationViews.GetDeletedSubgraphs().empty())
    {
        optimizationViews.AddUntouchedSubgraph(SubgraphView(subgraph));
    }
    else
    {
        ReportUntouchedLayers(optimizationViews, untouched);
    }
 
    return optimizationViews;
}

References armnn::Activation, armnn::Add, armnn::Addition, OptimizationViews::AddUntouchedSubgraph(), armnn::BatchNormalization, SubgraphView::begin(), Layer::BeginOutputSlots(), armnn::ChainReduceLayers(), armnn::ClAdditionValidate(), armnn::ClBatchNormalizationValidate(), armnn::ClConvolution2dWorkloadValidate(), armnn::ClDepthwiseConvolutionWorkloadValidate(), armnn::ClDivisionWorkloadValidate(), armnn::ClFullyConnectedWorkloadValidate(), armnn::ClMultiplicationWorkloadValidate(), armnn::ClSubtractionValidate(), armnn::ConnectedToLayerWithNCHW(), armnn::Convolution2d, CreateBackendSpecificModelContext(), armnn::DepthwiseConvolution2d, armnn::Div, armnn::Division, armnn::ElementwiseBinary, SubgraphView::end(), Layer::EndOutputSlots(), armnn::FoldPadLayer2d(), armnn::FullyConnected, armnn::FuseAdditionLayer(), armnn::FuseBatchNormalizationLayer(), armnn::FuseConvolution2dLayer(), armnn::FuseDepthwiseConvolution2dLayer(), armnn::FuseDivisionLayer(), armnn::FuseElementwiseBinaryLayer(), armnn::FuseFullyConnectedLayer(), armnn::FuseMultiplicationLayer(), armnn::FuseSubtractionLayer(), Layer::GetAdditionalInformation(), InputSlot::GetConnectedOutputSlot(), OptimizationViews::GetDeletedSubgraphs(), Layer::GetGuid(), Layer::GetInputSlot(), Layer::GetName(), OutputSlot::GetNumConnections(), Layer::GetOutputSlot(), OutputSlot::GetOwningLayer(), LayerWithParameters< Parameters >::GetParameters(), OptimizationViews::GetSubstitutions(), OutputSlot::GetTensorInfo(), Layer::GetType(), BatchNormalizationLayer::m_Beta, Convolution2dDescriptor::m_BiasEnabled, DepthwiseConvolution2dDescriptor::m_BiasEnabled, FullyConnectedDescriptor::m_BiasEnabled, BatchNormalizationLayer::m_Gamma, BatchNormalizationLayer::m_Mean, ElementwiseBinaryDescriptor::m_Operation, BatchNormalizationLayer::m_Variance, ReduceDescriptor::m_vAxis, armnn::Mul, armnn::Multiplication, armnn::Pad, armnn::PolymorphicDowncast(), armnn::Pooling2d, armnn::Reduce, armnn::RemoveReshapeLayer(), armnn::ReplaceLayers(), armnn::ReportUntouchedLayers(), armnn::Reshape, armnn::Sub, armnn::Subtraction, and armnn::optimizations::pad_fold::TryFoldPadIntoLayer2d().

RegisterTensorHandleFactories() [1/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry & )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 175 of file ClBackend.cpp.

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc));
 
    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));
 
}

References m_CustomAllocator, m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

RegisterTensorHandleFactories() [2/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry & registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 200 of file ClBackend.cpp.

{
    // To allow force import if inputFlags/outputFlags are Undefined, set it as Malloc
    if (inputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        inputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    if (outputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        outputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
            inputFlags, outputFlags);
 
    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));
}

References m_CustomAllocator, m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), TensorHandleFactoryRegistry::RegisterMemoryManager(), and armnn::Undefined.

UseCustomMemoryAllocator()

virtual bool UseCustomMemoryAllocator ( std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & > errMsg )

inlineoverridevirtual

Signals the backend to use a custom memory allocator provided by the user.

Parameters

allocator	- a pointer to the provided ICustomAllocator to use with this backend
errMsg	- Optional string variable to return error messages

Returns

- Returns true if switching to custom allocator was successful

Reimplemented from IBackendInternal.

Definition at line 82 of file ClBackend.hpp.

    {
        IgnoreUnused(errMsg);
        ARMNN_LOG(info) << "Using Custom Allocator for ClBackend";
 
        // Set flag to signal the backend to use a custom memory allocator
        m_CustomAllocator = std::make_shared<ClBackendCustomAllocatorWrapper>(std::move(allocator));
        m_UsingCustomAllocator = true;
        return m_UsingCustomAllocator;
    }

References ARMNN_LOG, armnn::IgnoreUnused(), armnn::info, m_CustomAllocator, and m_UsingCustomAllocator.

Referenced by ClBackend().

Member Data Documentation

m_CustomAllocator

std::shared_ptr<ClBackendCustomAllocatorWrapper> m_CustomAllocator

Definition at line 283 of file ClBackend.hpp.

Referenced by ClBackend(), CreateMemoryManager(), CreateWorkloadFactory(), CreateWorkloadFactory(), CreateWorkloadFactory(), RegisterTensorHandleFactories(), RegisterTensorHandleFactories(), and UseCustomMemoryAllocator().

m_UsingCustomAllocator

bool m_UsingCustomAllocator = false

Definition at line 284 of file ClBackend.hpp.

Referenced by CreateMemoryManager(), CreateWorkloadFactory(), CreateWorkloadFactory(), CreateWorkloadFactory(), RegisterTensorHandleFactories(), RegisterTensorHandleFactories(), and UseCustomMemoryAllocator().

---

The documentation for this class was generated from the following files:

• src/backends/cl/ClBackend.hpp
• src/backends/cl/ClBackend.cpp