TosaRefBackend Class Reference

#include <TosaRefBackend.hpp>

Inheritance diagram for TosaRefBackend:

Collaboration diagram for TosaRefBackend:

Public Member Functions
	TosaRefBackend ()=default
	~TosaRefBackend ()=default
const BackendId &	GetId () const override
IBackendInternal::IMemoryManagerUniquePtr	CreateMemoryManager () const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry) const override
IBackendInternal::IBackendContextPtr	CreateBackendContext (const IRuntime::CreationOptions &) const override
	Create the runtime context of the backend. More...
IBackendInternal::IBackendProfilingContextPtr	CreateBackendProfilingContext (const IRuntime::CreationOptions &creationOptions, IBackendProfilingPtr &backendProfiling) override
	Create context specifically used for profiling interaction from backends. More...
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport () const override
OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
std::vector< ITensorHandleFactory::FactoryId >	GetHandleFactoryPreferences () const override
	(Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
void	RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
std::unique_ptr< ICustomAllocator >	GetDefaultAllocator () const override
	Returns the default memory allocator for the backend. More...
BackendCapabilities	GetCapabilities () const override
	Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
Public Member Functions inherited from IBackendInternal
	~IBackendInternal () override=default
	Allow backends created by the factory function to be destroyed through IBackendInternal. More...
virtual IWorkloadFactoryPtr	CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const
virtual IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const
virtual IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const
virtual IBackendSpecificModelContextPtr	CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const
virtual ILayerSupportSharedPtr	GetLayerSupport (const ModelOptions &modelOptions) const
virtual OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph) const
bool	SupportsTensorAllocatorAPI () const
ITensorHandleFactory::FactoryId	GetBackwardCompatibleFavoriteHandleFactory ()
virtual void	RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags)
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
virtual bool	UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & > errMsg)
	Signals the backend to use a custom memory allocator provided by the user. More...
virtual unsigned int	GetNumberOfCacheFiles () const
	Returns the number of files cached if backend supports caching. More...

Static Public Member Functions
static const BackendId &	GetIdStatic ()
Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion	GetApiVersion ()
	Returns the version of the Backend API. More...

Additional Inherited Members
Public Types inherited from IBackendInternal
using	IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
using	IBackendContextPtr = std::unique_ptr< IBackendContext >
using	IBackendProfilingContextPtr = std::shared_ptr< arm::pipe::IBackendProfilingContext >
	This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
using	IBackendProfilingPtr = std::unique_ptr< arm::pipe::IBackendProfiling >
using	ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
using	IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
using	IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
using	IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
Protected Member Functions inherited from IBackendInternal
	IBackendInternal ()=default
	Creation must be done through a specific backend interface. More...
Protected Member Functions inherited from IBackend
	IBackend ()
virtual	~IBackend ()

Detailed Description

Definition at line 12 of file TosaRefBackend.hpp.

Constructor & Destructor Documentation

TosaRefBackend()

TosaRefBackend ( )

default

~TosaRefBackend()

~TosaRefBackend ( )

default

Member Function Documentation

CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions &  ) const

overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 60 of file TosaRefBackend.cpp.

{
    return IBackendContextPtr{};
}

CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions &  creationOptions, IBackendProfilingPtr &  backendProfiling  )

overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 65 of file TosaRefBackend.cpp.

{
    return IBackendProfilingContextPtr{};
}

CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 71 of file TosaRefBackend.cpp.

{
    return std::make_unique<TosaRefMemoryManager>();
}

CreateWorkloadFactory() [1/2]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 44 of file TosaRefBackend.cpp.

{
    auto memoryManager = std::make_shared<TosaRefMemoryManager>();
 
    tensorHandleFactoryRegistry.RegisterMemoryManager(memoryManager);
 
    auto factory = std::make_unique<TosaRefTensorHandleFactory>(memoryManager);
    // Register copy and import factory pair
    tensorHandleFactoryRegistry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
    // Register the factory
    tensorHandleFactoryRegistry.RegisterFactory(std::move(factory));
 
    return std::make_unique<TosaRefWorkloadFactory>(PolymorphicPointerDowncast<TosaRefMemoryManager>(memoryManager));
}

References TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

CreateWorkloadFactory() [2/2]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr &  memoryManager = nullptr ) const

overridevirtual

Implements IBackendInternal.

Definition at line 38 of file TosaRefBackend.cpp.

{
    return std::make_unique<TosaRefWorkloadFactory>(PolymorphicPointerDowncast<TosaRefMemoryManager>(memoryManager));
}

GetCapabilities()

BackendCapabilities GetCapabilities ( ) const

overridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 213 of file TosaRefBackend.cpp.

{
    return BackendCapabilities ("TosaRef",
                                {
                                        {"NonConstWeights", true},
                                        {"ConstantTensorsAsInputs", true}
                                });
}

GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const

overridevirtual

Returns the default memory allocator for the backend.

Returns

- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 208 of file TosaRefBackend.cpp.

{
    return std::make_unique<DefaultAllocator>();
}

GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const

overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 189 of file TosaRefBackend.cpp.

{
    return std::vector<ITensorHandleFactory::FactoryId> { TosaRefTensorHandleFactory::GetIdStatic() };
}

References TosaRefTensorHandleFactory::GetIdStatic().

GetId()

const BackendId& GetId ( ) const

inlineoverridevirtual

Implements IBackend.

Definition at line 19 of file TosaRefBackend.hpp.

    {
        return GetIdStatic();
    }

References TosaRefBackend::GetIdStatic().

Referenced by TosaRefBackend::OptimizeSubgraphView().

GetIdStatic()

const BackendId & GetIdStatic ( )

static

Definition at line 32 of file TosaRefBackend.cpp.

{
    static const BackendId s_Id{TosaRefBackendId()};
    return s_Id;
}

References armnn::TosaRefBackendId().

Referenced by TosaRefBackend::GetId().

GetLayerSupport()

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const

overridevirtual

Implements IBackendInternal.

Definition at line 76 of file TosaRefBackend.cpp.

{
    static ILayerSupportSharedPtr layerSupport{new TosaRefLayerSupport};
    return layerSupport;
}

OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView &  subgraph, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 82 of file TosaRefBackend.cpp.

{
    OptimizationViews optimizationViews(modelOptions);
 
    auto handler = std::make_unique<TosaSerializationHandler>();
 
    std::vector<std::string> graphInputs;
    std::vector<std::string> graphOutputs;
    std::vector<TosaSerializationOperator*> operators;
    std::vector<TosaSerializationTensor*> tensors;
    OpBlockSequencer<Layer, TosaSerializationBasicBlock> sequencer;
 
    // These sets are created to check the duplication of tensor and input
    std::unordered_set<std::string> graphInputsSet;
    std::unordered_set<std::string> uniqueTensorNamesSet;
 
    auto it = subgraph.begin();
    while (it != subgraph.end())
    {
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));
        ++it;
 
        if (base.GetType() == LayerType::Input ||
            base.GetType() == LayerType::Output)
        {
            continue;
        }
 
        TosaSerializationBasicBlock* mappings = GetTosaMappingFromLayer(&base);
 
        // Loop through inputs to see if there are any graph inputs, if so save them.
        // If it's an input to the graph "input" can be found in the string.
        for (const std::string& blockInputName : mappings->GetInputs())
        {
            if ((blockInputName.find("input") != std::string::npos) && !graphInputsSet.count(blockInputName))
            {
                graphInputs.push_back(blockInputName);
                graphInputsSet.insert(blockInputName);
            }
        }
 
        // Loop through outputs to see if there are any graph outputs, if so save them.
        // If it's an output to the graph "output" can be found in the string.
        for (const std::string& blockOutputName : mappings->GetOutputs())
        {
            if (blockOutputName.find("output") != std::string::npos)
            {
                graphOutputs.push_back(blockOutputName);
            }
        }
 
        sequencer.Add(&base, mappings);
    }
 
    for (auto & pair : sequencer.Finish())
    {
        auto blockOperators = pair.block->GetOperators();
        operators.insert(operators.end(), blockOperators.begin(), blockOperators.end());
 
        auto blockTensors = pair.block->GetTensors();
 
        // Checking for duplicate tensors, to handle special cases where same input is
        // added to different layer in the architecture.
        for (const auto& tensor : blockTensors)
        {
            const std::string& name = tensor->GetName();
 
            // If the input tensor is already present, we don't need to add to the list
            if (!uniqueTensorNamesSet.count(name))
            {
                uniqueTensorNamesSet.insert(name);
                tensors.push_back(tensor);
            }
        }
    }
 
    // Add all mappings to main block.
    auto* block = new TosaSerializationBasicBlock("main", "main", operators, tensors, graphInputs, graphOutputs);
 
    std::vector<TosaSerializationBasicBlock*> blocks;
    blocks.emplace_back(block);
 
    // Add blocks to the main region.
    auto* region = new TosaSerializationRegion("main", blocks);
    handler->GetRegions().emplace_back(region);
 
    auto compiledBlob =
            std::make_unique<PreCompiledObjectPtr>(handler.release(), DeleteAsType<TosaSerializationHandler>);
 
    IConnectableLayer* preCompiledLayer = optimizationViews.GetINetwork()->AddPrecompiledLayer(
            PreCompiledDescriptor(subgraph.GetNumInputSlots(), subgraph.GetNumOutputSlots()),
            std::move(*compiledBlob),
            armnn::Optional<BackendId>(GetId()),
            "TOSA_Pre_Compiled_Layer");
 
    // Copy the output tensor infos from sub-graph
    for (unsigned int i = 0; i < subgraph.GetNumOutputSlots(); i++)
    {
        preCompiledLayer->GetOutputSlot(i).SetTensorInfo(subgraph.GetIOutputSlot(i)->GetTensorInfo());
    }
 
    optimizationViews.AddSubstitution({ std::move(subgraph), SubgraphView(preCompiledLayer) });
    return optimizationViews;
}

References OpBlockSequencer< LayerT, BlockT >::Add(), INetwork::AddPrecompiledLayer(), OptimizationViews::AddSubstitution(), SubgraphView::begin(), SubgraphView::end(), OpBlockSequencer< LayerT, BlockT >::Finish(), TosaRefBackend::GetId(), OptimizationViews::GetINetwork(), SubgraphView::GetIOutputSlot(), SubgraphView::GetNumInputSlots(), SubgraphView::GetNumOutputSlots(), IConnectableLayer::GetOutputSlot(), IOutputSlot::GetTensorInfo(), GetTosaMappingFromLayer(), Layer::GetType(), armnn::Input, armnn::Output, and IOutputSlot::SetTensorInfo().

RegisterTensorHandleFactories()

void RegisterTensorHandleFactories ( class TensorHandleFactoryRegistry &  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 194 of file TosaRefBackend.cpp.

{
    auto memoryManager = std::make_shared<TosaRefMemoryManager>();
 
    registry.RegisterMemoryManager(memoryManager);
 
    auto factory = std::make_unique<TosaRefTensorHandleFactory>(memoryManager);
 
    // Register copy and import factory pair
    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
    // Register the factory
    registry.RegisterFactory(std::move(factory));
}

References TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

---

The documentation for this class was generated from the following files:

• src/backends/tosaReference/TosaRefBackend.hpp
• src/backends/tosaReference/TosaRefBackend.cpp