GpuFsaBackend Class Reference

#include <GpuFsaBackend.hpp>

Inheritance diagram for GpuFsaBackend:

Collaboration diagram for GpuFsaBackend:

Classes
class	ClBackendCustomAllocatorMemoryRegion
class	GpuFsaBackendCustomAllocatorWrapper

Public Member Functions
	GpuFsaBackend ()
	GpuFsaBackend (std::shared_ptr< ICustomAllocator > allocator)
	~GpuFsaBackend ()=default
const BackendId &	GetId () const override
IBackendInternal::IMemoryManagerUniquePtr	CreateMemoryManager () const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (TensorHandleFactoryRegistry &registry) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const override
std::vector< ITensorHandleFactory::FactoryId >	GetHandleFactoryPreferences () const override
	(Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
IBackendInternal::IBackendContextPtr	CreateBackendContext (const IRuntime::CreationOptions &) const override
	Create the runtime context of the backend. More...
IBackendInternal::IBackendProfilingContextPtr	CreateBackendProfilingContext (const IRuntime::CreationOptions &, IBackendProfilingPtr &backendProfiling) override
	Create context specifically used for profiling interaction from backends. More...
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport () const override
OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
std::unique_ptr< ICustomAllocator >	GetDefaultAllocator () const override
	Returns the default memory allocator for the backend. More...
BackendCapabilities	GetCapabilities () const override
	Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
virtual bool	UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & >) override
	Signals the backend to use a custom memory allocator provided by the user. More...
Public Member Functions inherited from IBackendInternal
	~IBackendInternal () override=default
	Allow backends created by the factory function to be destroyed through IBackendInternal. More...
virtual IWorkloadFactoryPtr	CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const
virtual IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const
virtual IBackendSpecificModelContextPtr	CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const
virtual ILayerSupportSharedPtr	GetLayerSupport (const ModelOptions &modelOptions) const
virtual OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph) const
bool	SupportsTensorAllocatorAPI () const
ITensorHandleFactory::FactoryId	GetBackwardCompatibleFavoriteHandleFactory ()
virtual unsigned int	GetNumberOfCacheFiles () const
	Returns the number of files cached if backend supports caching. More...
virtual ExecutionData	CreateExecutionData (WorkingMemDescriptor &workingMemDescriptor) const
	Returns ExecutionData for the backend. More...
virtual void	UpdateExecutionData (ExecutionData &executionData, WorkingMemDescriptor &workingMemDescriptor) const
	Update the ExecutionData for a layer. More...

Static Public Member Functions
static const BackendId &	GetIdStatic ()
Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion	GetApiVersion ()
	Returns the version of the Backend API. More...

Public Attributes
std::shared_ptr< GpuFsaBackendCustomAllocatorWrapper >	m_CustomAllocator
bool	m_UsingCustomAllocator = false

Additional Inherited Members
Public Types inherited from IBackendInternal
using	IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
using	IBackendContextPtr = std::unique_ptr< IBackendContext >
using	IBackendProfilingContextPtr = std::shared_ptr< arm::pipe::IBackendProfilingContext >
	This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
using	IBackendProfilingPtr = std::unique_ptr< arm::pipe::IBackendProfiling >
using	ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
using	IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
using	IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
using	IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
Protected Member Functions inherited from IBackendInternal
	IBackendInternal ()=default
	Creation must be done through a specific backend interface. More...
Protected Member Functions inherited from IBackend
	IBackend ()
virtual	~IBackend ()

Detailed Description

Definition at line 54 of file GpuFsaBackend.hpp.

Constructor & Destructor Documentation

GpuFsaBackend() [1/2]

GpuFsaBackend ( )

inline

Definition at line 57 of file GpuFsaBackend.hpp.

: m_CustomAllocator(nullptr) {};

GpuFsaBackend() [2/2]

GpuFsaBackend ( std::shared_ptr< ICustomAllocator >  allocator )

inline

Definition at line 58 of file GpuFsaBackend.hpp.

    {
        UseCustomMemoryAllocator(allocator, armnn::EmptyOptional());
    }

References GpuFsaBackend::UseCustomMemoryAllocator().

~GpuFsaBackend()

~GpuFsaBackend ( )

default

Member Function Documentation

CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions &  ) const

overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 198 of file GpuFsaBackend.cpp.

{
    return IBackendContextPtr{new GpuFsaBackendContext{options}};
}

CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions &  creationOptions, IBackendProfilingPtr &  backendProfiling  )

overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 203 of file GpuFsaBackend.cpp.

{
    return IBackendProfilingContextPtr{};
}

CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 75 of file GpuFsaBackend.cpp.

{
    if (m_UsingCustomAllocator)
    {
        return std::make_unique<GpuFsaMemoryManager>(m_CustomAllocator);
    }
    return std::make_unique<GpuFsaMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
}

References GpuFsaBackend::m_CustomAllocator, and GpuFsaBackend::m_UsingCustomAllocator.

CreateWorkloadFactory() [1/3]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry, const ModelOptions &  modelOptions, MemorySourceFlags  inputFlags, MemorySourceFlags  outputFlags  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 111 of file GpuFsaBackend.cpp.

{
 
    // To allow force import if inputFlags/outputFlags are Undefined, set it as Malloc
    if (inputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        inputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    if (outputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        outputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
 
    std::shared_ptr<GpuFsaMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<GpuFsaTensorHandleFactory>(memoryManager);
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
 
    return std::make_unique<GpuFsaWorkloadFactory>(PolymorphicPointerDowncast<GpuFsaMemoryManager>(memoryManager));
}

References GpuFsaBackend::m_CustomAllocator, GpuFsaBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterFactory(), TensorHandleFactoryRegistry::RegisterMemoryManager(), and armnn::Undefined.

CreateWorkloadFactory() [2/3]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr &  memoryManager = nullptr ) const

overridevirtual

Implements IBackendInternal.

Definition at line 84 of file GpuFsaBackend.cpp.

{
    return std::make_unique<GpuFsaWorkloadFactory>(PolymorphicPointerDowncast<GpuFsaMemoryManager>(memoryManager));
}

CreateWorkloadFactory() [3/3]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( TensorHandleFactoryRegistry &  registry ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 90 of file GpuFsaBackend.cpp.

{
    std::shared_ptr<GpuFsaMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<GpuFsaTensorHandleFactory>(memoryManager);
 
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
 
    return std::make_unique<GpuFsaWorkloadFactory>(PolymorphicPointerDowncast<GpuFsaMemoryManager>(memoryManager));
}

References GpuFsaBackend::m_CustomAllocator, GpuFsaBackend::m_UsingCustomAllocator, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

GetCapabilities()

BackendCapabilities GetCapabilities ( ) const

inlineoverridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 98 of file GpuFsaBackend.hpp.

    {
        return gpuFsaCapabilities;
    };

References armnn::gpuFsaCapabilities.

GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const

overridevirtual

Returns the default memory allocator for the backend.

Returns

- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 215 of file GpuFsaBackend.cpp.

{
    return std::make_unique<GpuFsaBackendDefaultAllocator>();
}

GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const

overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 146 of file GpuFsaBackend.cpp.

{
    return std::vector<ITensorHandleFactory::FactoryId> { GpuFsaTensorHandleFactory::GetIdStatic() };
}

References GpuFsaTensorHandleFactory::GetIdStatic().

GetId()

const BackendId& GetId ( ) const

inlineoverridevirtual

Implements IBackend.

Definition at line 65 of file GpuFsaBackend.hpp.

{ return GetIdStatic(); }

References GpuFsaBackend::GetIdStatic().

Referenced by GpuFsaBackend::OptimizeSubgraphView().

GetIdStatic()

const BackendId & GetIdStatic ( )

static

Definition at line 69 of file GpuFsaBackend.cpp.

{
    static const BackendId s_Id{GpuFsaBackendId()};
    return s_Id;
}

References armnn::GpuFsaBackendId().

Referenced by GpuFsaBackend::GetId().

GetLayerSupport()

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const

overridevirtual

Implements IBackendInternal.

Definition at line 209 of file GpuFsaBackend.cpp.

{
    static ILayerSupportSharedPtr layerSupport{new GpuFsaLayerSupport};
    return layerSupport;
}

OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView &  subgraph, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 220 of file GpuFsaBackend.cpp.

{
    OptimizationViews optimizationViews(modelOptions);
 
    using namespace arm_compute::experimental::dynamic_fusion;
 
    auto it = subgraph.end();
    std::map<LayerGuid, Layer*> untouched;
    while (it != subgraph.begin())
    {
        --it;
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));
        untouched.insert({base.GetGuid(), &base});
    }
 
    GpuFsaLayerSupport supportChecker;
    it = subgraph.end();
    arm_compute::CLCompileContext* compileCtx = &(arm_compute::CLKernelLibrary::get().get_compile_context());
 
    // Setup the GpuWokloadContext which will exist for the lifetime of the Graph. This contains the TensorInfos
    std::shared_ptr<GpuWorkloadContext> workloadContext = std::make_shared<GpuWorkloadContext>(compileCtx);
    while (it != subgraph.begin())
    {
        --it;
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));
        // Create a GpuFsaPreCompiledBlob, this contains all of the information needed to execute an operator
        GpuFsaPreCompiledBlob* preCompiledBlobPtr = new GpuFsaPreCompiledBlob();
        preCompiledBlobPtr->workloadContext = workloadContext;
        preCompiledBlobPtr->sketch = std::make_unique<GpuWorkloadSketch>(workloadContext.get());
 
        // Configure and setup the sketch for each supported op. Their data will be wrapped into a PreCompiled layer
        switch (base.GetType())
        {
            case (LayerType::Activation):
            {
                auto desc = PolymorphicDowncast<const ActivationDescriptor*>(&base.GetParameters());
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                GpuFsaActivationCreateOp(preCompiledBlobPtr, input, *desc);
                break;
            }
            case (LayerType::Cast):
            {
                auto input  = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto output = base.GetOutputSlot(0).GetTensorInfo();
                GpuFsaCastCreateOp(preCompiledBlobPtr, input, output);
                break;
            }
            case (LayerType::Convolution2d):
            {
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto weights = base.GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo();
 
                auto desc = PolymorphicDowncast<const Convolution2dDescriptor*>(&base.GetParameters());
                if (desc->m_BiasEnabled)
                {
                    auto bias = base.GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
                    GpuFsaConvolution2dCreateOp(preCompiledBlobPtr,
                                                input,
                                                *desc,
                                                weights,
                                                bias);
                }
                else
                {
                    GpuFsaConvolution2dCreateOp(preCompiledBlobPtr,
                                                input,
                                                *desc,
                                                weights,
                                                EmptyOptional());
                }
                break;
            }
            case (LayerType::BatchMatMul):
            {
                auto input0 = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto input1 = base.GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo();
                auto desc = PolymorphicDowncast<const BatchMatMulDescriptor*>(&base.GetParameters());
                GpuFsaBatchMatMulCreateOp(preCompiledBlobPtr, input0, input1, *desc);
                break;
            }
            case (LayerType::DepthwiseConvolution2d):
            {
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto weights = base.GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo();
 
                auto desc = PolymorphicDowncast<const DepthwiseConvolution2dDescriptor*>(&base.GetParameters());
                if (desc->m_BiasEnabled)
                {
                    auto bias = base.GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
                    GpuFsaDepthwiseConvolution2dCreateOp(preCompiledBlobPtr,
                                                         input,
                                                         *desc,
                                                         weights,
                                                         bias);
                }
                else
                {
                    GpuFsaDepthwiseConvolution2dCreateOp(preCompiledBlobPtr,
                                                         input,
                                                         *desc,
                                                         weights,
                                                         EmptyOptional());
                }
                break;
            }
            case LayerType::ElementwiseBinary:
            {
                auto desc = PolymorphicDowncast<const ElementwiseBinaryDescriptor *>(&base.GetParameters());
                auto input0 = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto input1 = base.GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo();
                GpuFsaElementwiseBinaryCreateOp(preCompiledBlobPtr, input0, input1, *desc);
                break;
            }
            case (LayerType::Pooling2d):
            {
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto desc = PolymorphicDowncast<const Pooling2dDescriptor*>(&base.GetParameters());
                GpuFsaPooling2dCreateOp(preCompiledBlobPtr, input, *desc);
                break;
            }
            case LayerType::Reshape:
            {
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto desc = PolymorphicDowncast<const ReshapeDescriptor*>(&base.GetParameters());
                GpuFsaReshapeCreateOp(preCompiledBlobPtr, input, *desc);
 
                break;
            }
            case (LayerType::Resize):
            {
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto desc = PolymorphicDowncast<const ResizeDescriptor*>(&base.GetParameters());
                GpuFsaResizeCreateOp(preCompiledBlobPtr, input, *desc);
                break;
            }
            case (LayerType::Softmax):
            {
                auto input = base.GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo();
                auto output = base.GetOutputSlot(0).GetTensorInfo();
 
                auto desc = PolymorphicDowncast<const SoftmaxDescriptor*>(&base.GetParameters());
                GpuFsaSoftmaxCreateOp(preCompiledBlobPtr,
                                      input,
                                      output,
                                      *desc);
                break;
            }
            default:
                // unsupported layer for GpuFsa backend
                continue;
        }
 
        auto compiledBlob =
            std::make_unique<PreCompiledObjectPtr>(preCompiledBlobPtr, DeleteAsType<GpuFsaPreCompiledBlob>);
 
        IConnectableLayer* preCompiledLayer = optimizationViews.GetINetwork()->AddPrecompiledLayer(
            PreCompiledDescriptor(base.GetNumInputSlots(), base.GetNumOutputSlots()),
            std::move(*compiledBlob),
            armnn::Optional<BackendId>(GetId()),
            "GpuFsa_Pre_Compiled_Layer");
 
        // Copy the output tensor infos from sub-graph
        for (unsigned int i = 0; i < subgraph.GetNumOutputSlots(); i++)
        {
            preCompiledLayer->GetOutputSlot(i).SetTensorInfo(base.GetOutputSlot(i).GetTensorInfo());
        }
 
        SubgraphView::SubgraphViewPtr substituteSubgraph =
            CreateSubgraphViewFrom(CreateInputsFrom(&base),
                                   CreateOutputsFrom(&base),
                                   {&base});
 
        optimizationViews.AddSubstitution({ std::move(*substituteSubgraph), SubgraphView(preCompiledLayer) });
 
        untouched.erase(base.GetGuid());
    }
 
    if (optimizationViews.GetSubstitutions().empty())
    {
        optimizationViews.AddUntouchedSubgraph(SubgraphView(subgraph));
    }
    else
    {
        ReportUntouchedLayers(optimizationViews, untouched);
    }
 
 
    return optimizationViews;
}

References armnn::Activation, INetwork::AddPrecompiledLayer(), OptimizationViews::AddSubstitution(), OptimizationViews::AddUntouchedSubgraph(), armnn::BatchMatMul, SubgraphView::begin(), armnn::Cast, armnn::Convolution2d, armnn::CreateInputsFrom(), armnn::CreateOutputsFrom(), armnn::CreateSubgraphViewFrom(), armnn::DepthwiseConvolution2d, armnn::ElementwiseBinary, SubgraphView::end(), InputSlot::GetConnectedOutputSlot(), Layer::GetGuid(), GpuFsaBackend::GetId(), OptimizationViews::GetINetwork(), Layer::GetInputSlot(), Layer::GetNumInputSlots(), SubgraphView::GetNumOutputSlots(), Layer::GetNumOutputSlots(), IConnectableLayer::GetOutputSlot(), Layer::GetOutputSlot(), Layer::GetParameters(), OptimizationViews::GetSubstitutions(), OutputSlot::GetTensorInfo(), Layer::GetType(), armnn::GpuFsaActivationCreateOp(), armnn::GpuFsaBatchMatMulCreateOp(), armnn::GpuFsaCastCreateOp(), armnn::GpuFsaConvolution2dCreateOp(), armnn::GpuFsaDepthwiseConvolution2dCreateOp(), armnn::GpuFsaElementwiseBinaryCreateOp(), armnn::GpuFsaPooling2dCreateOp(), armnn::GpuFsaReshapeCreateOp(), armnn::GpuFsaResizeCreateOp(), armnn::GpuFsaSoftmaxCreateOp(), armnn::Pooling2d, armnn::ReportUntouchedLayers(), armnn::Reshape, armnn::Resize, IOutputSlot::SetTensorInfo(), GpuFsaPreCompiledBlob::sketch, armnn::Softmax, and GpuFsaPreCompiledBlob::workloadContext.

RegisterTensorHandleFactories() [1/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry &  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 151 of file GpuFsaBackend.cpp.

{
    std::shared_ptr<GpuFsaMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<GpuFsaTensorHandleFactory>(memoryManager);
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
 
}

References GpuFsaBackend::m_CustomAllocator, GpuFsaBackend::m_UsingCustomAllocator, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

RegisterTensorHandleFactories() [2/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry &  registry, MemorySourceFlags  inputFlags, MemorySourceFlags  outputFlags  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 169 of file GpuFsaBackend.cpp.

{
    // To allow force import if inputFlags/outputFlags are Undefined, set it as Malloc
    if (inputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        inputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    if (outputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        outputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
 
    std::shared_ptr<GpuFsaMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<GpuFsaMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }
 
    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<GpuFsaTensorHandleFactory>(memoryManager);
    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
}

References GpuFsaBackend::m_CustomAllocator, GpuFsaBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterFactory(), TensorHandleFactoryRegistry::RegisterMemoryManager(), and armnn::Undefined.

UseCustomMemoryAllocator()

virtual bool UseCustomMemoryAllocator ( std::shared_ptr< ICustomAllocator >  allocator, armnn::Optional< std::string & >  errMsg  )

inlineoverridevirtual

Signals the backend to use a custom memory allocator provided by the user.

Parameters

allocator	- a pointer to the provided ICustomAllocator to use with this backend
errMsg	- Optional string variable to return error messages

Returns

- Returns true if switching to custom allocator was successful

Reimplemented from IBackendInternal.

Definition at line 103 of file GpuFsaBackend.hpp.

    {
        ARMNN_LOG(info) << "Using Custom Allocator for GpuFsaBackend";
 
        // Set flag to signal the backend to use a custom memory allocator
        m_CustomAllocator = std::make_shared<GpuFsaBackendCustomAllocatorWrapper>(std::move(allocator));
        m_UsingCustomAllocator = true;
        return m_UsingCustomAllocator;
    }

References ARMNN_LOG, armnn::info, GpuFsaBackend::m_CustomAllocator, and GpuFsaBackend::m_UsingCustomAllocator.

Referenced by GpuFsaBackend::GpuFsaBackend().

Member Data Documentation

m_CustomAllocator

std::shared_ptr<GpuFsaBackendCustomAllocatorWrapper> m_CustomAllocator

Definition at line 301 of file GpuFsaBackend.hpp.

Referenced by GpuFsaBackend::CreateMemoryManager(), GpuFsaBackend::CreateWorkloadFactory(), GpuFsaBackend::RegisterTensorHandleFactories(), and GpuFsaBackend::UseCustomMemoryAllocator().

m_UsingCustomAllocator

bool m_UsingCustomAllocator = false

Definition at line 302 of file GpuFsaBackend.hpp.

Referenced by GpuFsaBackend::CreateMemoryManager(), GpuFsaBackend::CreateWorkloadFactory(), GpuFsaBackend::RegisterTensorHandleFactories(), and GpuFsaBackend::UseCustomMemoryAllocator().

---

The documentation for this class was generated from the following files:

• src/backends/gpuFsa/GpuFsaBackend.hpp
• src/backends/gpuFsa/GpuFsaBackend.cpp