TensorAllocator Class Reference

Basic implementation of a CPU memory tensor allocator. More...

#include <TensorAllocator.h>

Collaboration diagram for TensorAllocator:

Public Member Functions
	TensorAllocator (IMemoryManageable *owner)
	Default constructor. More...
	~TensorAllocator ()
	Default destructor. More...
	TensorAllocator (const TensorAllocator &)=delete
	Prevent instances of this class from being copied (As this class contains pointers) More...
TensorAllocator &	operator= (const TensorAllocator &)=delete
	Prevent instances of this class from being copy assigned (As this class contains pointers) More...
	TensorAllocator (TensorAllocator &&) noexcept
	Allow instances of this class to be moved. More...
TensorAllocator &	operator= (TensorAllocator &&) noexcept
	Allow instances of this class to be moved. More...
void	init (const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
	Shares the same backing memory with another tensor allocator, while the tensor info might be different. More...
uint8_t *	data () const
	Returns the pointer to the allocated data. More...
void	allocate () override
	Allocate size specified by TensorInfo of CPU memory. More...
void	free () override
	Free allocated CPU memory. More...
Status	import_memory (void *memory)
	Import an existing memory as a tensor's backing memory. More...
void	set_associated_memory_group (IMemoryGroup *associated_memory_group)
	Associates the tensor with a memory group. More...
void	init (const TensorInfo &input, size_t alignment=0)
	Make ITensorAllocator's init methods available. More...
Public Member Functions inherited from ITensorAllocator
	ITensorAllocator ()=default
	Default constructor. More...
	ITensorAllocator (const ITensorAllocator &)=default
	Allow instances of this class to be copy constructed. More...
ITensorAllocator &	operator= (const ITensorAllocator &)=default
	Allow instances of this class to be copied. More...
	ITensorAllocator (ITensorAllocator &&)=default
	Allow instances of this class to be move constructed. More...
ITensorAllocator &	operator= (ITensorAllocator &&)=default
	Allow instances of this class to be moved. More...
virtual	~ITensorAllocator ()=default
	Default virtual destructor. More...
void	init (const TensorInfo &input, size_t alignment=0)
	Initialize a tensor based on the passed TensorInfo. More...
void	soft_init (TensorInfo &input, size_t alignment=0)
	Initialize a tensor based with a reference TensorInfo. More...
TensorInfo &	info ()
	Return a reference to the tensor's metadata. More...
const TensorInfo &	info () const
	Return a constant reference to the tensor's metadata. More...
size_t	alignment () const
	Return underlying's tensor buffer alignment. More...

Detailed Description

Basic implementation of a CPU memory tensor allocator.

Definition at line 42 of file TensorAllocator.h.

Constructor & Destructor Documentation

TensorAllocator() [1/3]

TensorAllocator

(

IMemoryManageable *

owner

)

Default constructor.

Parameters

[in]

owner

Memory manageable owner

Definition at line 68 of file TensorAllocator.cpp.

    : _owner(owner), _associated_memory_group(nullptr), _memory()
{
}

~TensorAllocator()

~TensorAllocator

(

)

Default destructor.

Definition at line 73 of file TensorAllocator.cpp.

{
    info().set_is_resizable(true);
}

References ITensorAllocator::info(), and TensorInfo::set_is_resizable().

TensorAllocator() [2/3]

TensorAllocator ( const TensorAllocator &  )

delete

Prevent instances of this class from being copied (As this class contains pointers)

TensorAllocator() [3/3]

TensorAllocator ( TensorAllocator &&  o )

noexcept

Allow instances of this class to be moved.

Definition at line 78 of file TensorAllocator.cpp.

    : ITensorAllocator(std::move(o)),
      _owner(o._owner),
      _associated_memory_group(o._associated_memory_group),
      _memory(std::move(o._memory))
{
    o._owner                   = nullptr;
    o._associated_memory_group = nullptr;
    o._memory                  = Memory();
}

Member Function Documentation

allocate()

void allocate ( )

overridevirtual

Allocate size specified by TensorInfo of CPU memory.

Note

The tensor must not already be allocated when calling this function.

Implements ITensorAllocator.

Definition at line 132 of file TensorAllocator.cpp.

{
    // Align to 64-byte boundaries by default if alignment is not specified
    const size_t alignment_to_use = (alignment() != 0) ? alignment() : 64;
    if(_associated_memory_group == nullptr)
    {
        _memory.set_owned_region(std::make_unique<MemoryRegion>(info().total_size(), alignment_to_use));
    }
    else
    {
        _associated_memory_group->finalize_memory(_owner, _memory, info().total_size(), alignment_to_use);
    }
    info().set_is_resizable(false);
}

References ITensorAllocator::alignment(), IMemoryGroup::finalize_memory(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), Memory::set_owned_region(), and TensorInfo::total_size().

Referenced by NETensorHandle::allocate(), CPPBoxWithNonMaximaSuppressionLimit::configure(), CPPDetectionPostProcessLayer::configure(), NEFFT2D::configure(), NERNNLayer::configure(), NEL2NormalizeLayer::configure(), NEFFT1D::configure(), NEReductionOperation::configure(), NEInstanceNormalizationLayer::configure(), NEDetectionPostProcessLayer::configure(), NENormalizationLayer::configure(), NEFFTConvolutionLayer::configure(), NEGenerateProposalsLayer::configure(), NELSTMLayerQuantized::configure(), NELSTMLayer::configure(), NEDeconvolutionLayer::configure(), NEQLSTMLayer::configure(), CpuAuxTensorHandler::CpuAuxTensorHandler(), main(), NumPyAccessor::NumPyAccessor(), NEFFTConvolutionLayer::prepare(), NEDeconvolutionLayer::prepare(), NELSTMLayerQuantized::prepare(), NEQLSTMLayer::prepare(), NEFullyConnectedLayerReshapeWeightsManaged::run(), and arm_compute::test::validation::TEST_CASE().

data()

uint8_t * data

(

)

const

Returns the pointer to the allocated data.

Returns

a pointer to the allocated data.

Definition at line 127 of file TensorAllocator.cpp.

{
    return (_memory.region() == nullptr) ? nullptr : reinterpret_cast<uint8_t *>(_memory.region()->buffer());
}

References IMemoryRegion::buffer(), and Memory::region().

Referenced by Tensor::buffer().

free()

void free ( )

overridevirtual

Free allocated CPU memory.

Note

The tensor must have been allocated when calling this function.

Implements ITensorAllocator.

Definition at line 147 of file TensorAllocator.cpp.

{
    _memory.set_region(nullptr);
    info().set_is_resizable(true);
}

References ITensorAllocator::info(), TensorInfo::set_is_resizable(), and Memory::set_region().

Referenced by CpuDirectConv2d::configure(), CpuDirectConv3d::configure(), NETensorHandle::free(), NEFFTConvolutionLayer::prepare(), NELSTMLayerQuantized::prepare(), NEFullyConnectedLayerReshapeWeightsManaged::release(), NETensorHandle::release_if_unused(), and arm_compute::test::validation::TEST_CASE().

import_memory()

Status import_memory

(

void *

memory

)

Import an existing memory as a tensor's backing memory.

Warning

size is expected to be compliant with total_size reported by ITensorInfo.

ownership of memory is not transferred.

tensor shouldn't be memory managed.

padding should be accounted by the client code.

memory must be writable in case of in-place operations

Note

buffer alignment will be checked to be compliant with alignment reported by ITensorInfo.

Parameters

[in]

memory

Raw memory pointer to be used as backing memory

Returns

An error status

Definition at line 153 of file TensorAllocator.cpp.

{
    ARM_COMPUTE_RETURN_ERROR_ON(memory == nullptr);
    ARM_COMPUTE_RETURN_ERROR_ON(_associated_memory_group != nullptr);
    ARM_COMPUTE_RETURN_ERROR_ON(alignment() != 0 && !arm_compute::utility::check_aligned(memory, alignment()));
 
    _memory.set_owned_region(std::make_unique<MemoryRegion>(memory, info().total_size()));
    info().set_is_resizable(false);
 
    return Status{};
}

References ITensorAllocator::alignment(), ARM_COMPUTE_RETURN_ERROR_ON, arm_compute::utility::check_aligned(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), and Memory::set_owned_region().

Referenced by CpuAuxTensorHandler::CpuAuxTensorHandler(), NEFFTConvolutionLayer::run(), CpuGemmConv2d::run(), and arm_compute::test::validation::TEST_CASE().

init() [1/2]

void init	(	const TensorAllocator &	allocator,
		const Coordinates &	coords,
		TensorInfo &	sub_info
	)

Shares the same backing memory with another tensor allocator, while the tensor info might be different.

In other words this can be used to create a sub-tensor from another tensor while sharing the same memory.

Note

TensorAllocator have to be of the same specialized type.

Parameters

[in]	allocator	The allocator that owns the backing memory to be shared. Ownership becomes shared afterwards.
[in]	coords	The starting coordinates of the new tensor inside the parent tensor.
[in]	sub_info	The new tensor information (e.g. shape etc)

Definition at line 107 of file TensorAllocator.cpp.

{
    // Get parent info
    const TensorInfo parent_info = allocator.info();
 
    // Check if coordinates and new shape are within the parent tensor
    ARM_COMPUTE_ERROR_ON(!validate_subtensor_shape(parent_info, sub_info, coords));
    ARM_COMPUTE_UNUSED(validate_subtensor_shape);
 
    // Copy pointer to buffer
    _memory = Memory(allocator._memory.region());
 
    // Init tensor info with new dimensions
    size_t total_size = parent_info.offset_element_in_bytes(coords) + sub_info.total_size() - sub_info.offset_first_element_in_bytes();
    sub_info.init(sub_info.tensor_shape(), sub_info.format(), parent_info.strides_in_bytes(), parent_info.offset_element_in_bytes(coords), total_size);
 
    // Set TensorInfo
    init(sub_info);
}

References arm_compute::test::validation::allocator(), ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_UNUSED, TensorInfo::format(), TensorInfo::init(), TensorInfo::offset_element_in_bytes(), TensorInfo::offset_first_element_in_bytes(), TensorInfo::strides_in_bytes(), TensorInfo::tensor_shape(), and TensorInfo::total_size().

Referenced by CPPBoxWithNonMaximaSuppressionLimit::configure(), NERNNLayer::configure(), NEFFT1D::configure(), NEReductionOperation::configure(), NENormalizationLayer::configure(), NEFFTConvolutionLayer::configure(), NEGenerateProposalsLayer::configure(), NELSTMLayerQuantized::configure(), NELSTMLayer::configure(), NEDeconvolutionLayer::configure(), NEQLSTMLayer::configure(), main(), NETensorHandle::NETensorHandle(), NumPyAccessor::NumPyAccessor(), and arm_compute::test::validation::TEST_CASE().

init() [2/2]

void init

Make ITensorAllocator's init methods available.

Definition at line 33 of file ITensorAllocator.cpp.

{
    _info_owned    = input;
    _info_external = nullptr;
    _alignment     = alignment;
}

operator=() [1/2]

TensorAllocator& operator= ( const TensorAllocator &  )

delete

Prevent instances of this class from being copy assigned (As this class contains pointers)

operator=() [2/2]

TensorAllocator & operator= ( TensorAllocator &&  o )

noexcept

Allow instances of this class to be moved.

Definition at line 89 of file TensorAllocator.cpp.

{
    if(&o != this)
    {
        _owner   = o._owner;
        o._owner = nullptr;
 
        _associated_memory_group   = o._associated_memory_group;
        o._associated_memory_group = nullptr;
 
        _memory   = std::move(o._memory);
        o._memory = Memory();
 
        ITensorAllocator::operator=(std::move(o));
    }
    return *this;
}

References ITensorAllocator::operator=().

set_associated_memory_group()

void set_associated_memory_group

(

IMemoryGroup *

associated_memory_group

)

Associates the tensor with a memory group.

Parameters

[in]

associated_memory_group

Memory group to associate the tensor with

Definition at line 165 of file TensorAllocator.cpp.

{
    ARM_COMPUTE_ERROR_ON(associated_memory_group == nullptr);
    ARM_COMPUTE_ERROR_ON(_associated_memory_group != nullptr && _associated_memory_group != associated_memory_group);
    ARM_COMPUTE_ERROR_ON(_memory.region() != nullptr && _memory.region()->buffer() != nullptr);
 
    _associated_memory_group = associated_memory_group;
}

References ARM_COMPUTE_ERROR_ON, IMemoryRegion::buffer(), and Memory::region().

Referenced by Tensor::associate_memory_group(), and arm_compute::test::validation::TEST_CASE().

---

The documentation for this class was generated from the following files:

• arm_compute/runtime/TensorAllocator.h
• src/runtime/TensorAllocator.cpp