Compute Library
 21.02
TensorAllocator Class Reference

Basic implementation of a CPU memory tensor allocator. More...

#include <TensorAllocator.h>

Collaboration diagram for TensorAllocator:
[legend]

Public Member Functions

 TensorAllocator (IMemoryManageable *owner)
 Default constructor. More...
 
 ~TensorAllocator ()
 Default destructor. More...
 
 TensorAllocator (const TensorAllocator &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
TensorAllocatoroperator= (const TensorAllocator &)=delete
 Prevent instances of this class from being copy assigned (As this class contains pointers) More...
 
 TensorAllocator (TensorAllocator &&) noexcept
 Allow instances of this class to be moved. More...
 
TensorAllocatoroperator= (TensorAllocator &&) noexcept
 Allow instances of this class to be moved. More...
 
void init (const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
 Shares the same backing memory with another tensor allocator, while the tensor info might be different. More...
 
uint8_t * data () const
 Returns the pointer to the allocated data. More...
 
void allocate () override
 Allocate size specified by TensorInfo of CPU memory. More...
 
void free () override
 Free allocated CPU memory. More...
 
Status import_memory (void *memory)
 Import an existing memory as a tensor's backing memory. More...
 
void set_associated_memory_group (IMemoryGroup *associated_memory_group)
 Associates the tensor with a memory group. More...
 
- Public Member Functions inherited from ITensorAllocator
 ITensorAllocator ()
 Default constructor. More...
 
 ITensorAllocator (const ITensorAllocator &)=default
 Allow instances of this class to be copy constructed. More...
 
ITensorAllocatoroperator= (const ITensorAllocator &)=default
 Allow instances of this class to be copied. More...
 
 ITensorAllocator (ITensorAllocator &&)=default
 Allow instances of this class to be move constructed. More...
 
ITensorAllocatoroperator= (ITensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
virtual ~ITensorAllocator ()=default
 Default virtual destructor. More...
 
void init (const TensorInfo &input, size_t alignment=0)
 Initialize a tensor based on the passed TensorInfo. More...
 
TensorInfoinfo ()
 Return a reference to the tensor's metadata. More...
 
const TensorInfoinfo () const
 Return a constant reference to the tensor's metadata. More...
 
size_t alignment () const
 Return underlying's tensor buffer alignment. More...
 

Detailed Description

Basic implementation of a CPU memory tensor allocator.

Definition at line 42 of file TensorAllocator.h.

Constructor & Destructor Documentation

◆ TensorAllocator() [1/3]

Default constructor.

Parameters
[in]ownerMemory manageable owner

Definition at line 68 of file TensorAllocator.cpp.

69  : _owner(owner), _associated_memory_group(nullptr), _memory()
70 {
71 }

◆ ~TensorAllocator()

Default destructor.

Definition at line 73 of file TensorAllocator.cpp.

References ITensorAllocator::info(), and TensorInfo::set_is_resizable().

74 {
75  info().set_is_resizable(true);
76 }
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:298
TensorInfo & info()
Return a reference to the tensor&#39;s metadata.

◆ TensorAllocator() [2/3]

TensorAllocator ( const TensorAllocator )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ TensorAllocator() [3/3]

TensorAllocator ( TensorAllocator &&  o)
noexcept

Allow instances of this class to be moved.

Definition at line 78 of file TensorAllocator.cpp.

79  : ITensorAllocator(std::move(o)),
80  _owner(o._owner),
81  _associated_memory_group(o._associated_memory_group),
82  _memory(std::move(o._memory))
83 {
84  o._owner = nullptr;
85  o._associated_memory_group = nullptr;
86  o._memory = Memory();
87 }
CPU implementation of memory object.
Definition: Memory.h:37
ITensorAllocator()
Default constructor.

Member Function Documentation

◆ allocate()

void allocate ( )
overridevirtual

Allocate size specified by TensorInfo of CPU memory.

Note
The tensor must not already be allocated when calling this function.

Implements ITensorAllocator.

Definition at line 132 of file TensorAllocator.cpp.

References ITensorAllocator::alignment(), IMemoryGroup::finalize_memory(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), and Memory::set_owned_region().

Referenced by NETensorHandle::allocate(), NEScale::configure(), CpuPoolingAssemblyDispatch::configure(), NEReduceMean::configure(), CPPBoxWithNonMaximaSuppressionLimit::configure(), NERNNLayer::configure(), NEFFT2D::configure(), NEReductionOperation::configure(), CPPDetectionPostProcessLayer::configure(), NEL2NormalizeLayer::configure(), NEInstanceNormalizationLayer::configure(), NENormalizationLayer::configure(), NEDetectionPostProcessLayer::configure(), NEFFT1D::configure(), NEHOGDescriptor::configure(), NEGaussian5x5::configure(), NEHOGGradient::configure(), NESobel5x5::configure(), NESobel7x7::configure(), NEFastCorners::configure(), NEWinogradConvolutionLayer::configure(), NECannyEdge::configure(), NEGenerateProposalsLayer::configure(), NEFFTConvolutionLayer::configure(), NEGEMM::configure(), NEHOGMultiDetection::configure(), NEHarrisCorners::configure(), NEGEMMLowpMatrixMultiplyCore::configure(), NELSTMLayerQuantized::configure(), NELSTMLayer::configure(), NEDeconvolutionLayer::configure(), NEConvolutionSquare< matrix_size >::configure(), NEQLSTMLayer::configure(), NEGEMMConvolutionLayer::configure(), main(), NEQLSTMLayer::NEQLSTMLayer(), NumPyAccessor::NumPyAccessor(), NEWinogradConvolutionLayer::prepare(), NEGEMMConv2d::prepare(), NEGEMM::prepare(), NEFFTConvolutionLayer::prepare(), NEGEMMLowpMatrixMultiplyCore::prepare(), NEDeconvolutionLayer::prepare(), NELSTMLayerQuantized::prepare(), NEFullyConnectedLayer::prepare(), NEQLSTMLayer::prepare(), NEGEMMConvolutionLayer::prepare(), and arm_compute::test::validation::TEST_CASE().

133 {
134  // Align to 64-byte boundaries by default if alignment is not specified
135  const size_t alignment_to_use = (alignment() != 0) ? alignment() : 64;
136  if(_associated_memory_group == nullptr)
137  {
138  _memory.set_owned_region(std::make_unique<MemoryRegion>(info().total_size(), alignment_to_use));
139  }
140  else
141  {
142  _associated_memory_group->finalize_memory(_owner, _memory, info().total_size(), alignment_to_use);
143  }
144  info().set_is_resizable(false);
145 }
void set_owned_region(std::unique_ptr< IMemoryRegion > region) final
Sets a memory region.
Definition: Memory.cpp:64
virtual void finalize_memory(IMemoryManageable *obj, IMemory &obj_memory, size_t size, size_t alignment)=0
Finalizes memory for a given object.
size_t alignment() const
Return underlying&#39;s tensor buffer alignment.
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:298
TensorInfo & info()
Return a reference to the tensor&#39;s metadata.

◆ data()

uint8_t * data ( ) const

Returns the pointer to the allocated data.

Returns
a pointer to the allocated data.

Definition at line 127 of file TensorAllocator.cpp.

References IMemoryRegion::buffer(), and Memory::region().

Referenced by Tensor::buffer().

128 {
129  return (_memory.region() == nullptr) ? nullptr : reinterpret_cast<uint8_t *>(_memory.region()->buffer());
130 }
IMemoryRegion * region() final
Region accessor.
Definition: Memory.cpp:48
virtual void * buffer()=0
Returns the pointer to the allocated data.

◆ free()

void free ( )
overridevirtual

Free allocated CPU memory.

Note
The tensor must have been allocated when calling this function.

Implements ITensorAllocator.

Definition at line 147 of file TensorAllocator.cpp.

References ITensorAllocator::info(), TensorInfo::set_is_resizable(), and Memory::set_region().

Referenced by NEDirectConvolutionLayer::configure(), NETensorHandle::free(), NEWinogradConvolutionLayer::prepare(), NEFFTConvolutionLayer::prepare(), NELSTMLayerQuantized::prepare(), NEGEMMConvolutionLayer::prepare(), NETensorHandle::release_if_unused(), NEGEMMConvolutionLayer::run(), and arm_compute::test::validation::TEST_CASE().

148 {
149  _memory.set_region(nullptr);
150  info().set_is_resizable(true);
151 }
void set_region(IMemoryRegion *region) final
Sets a memory region.
Definition: Memory.cpp:58
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:298
TensorInfo & info()
Return a reference to the tensor&#39;s metadata.

◆ import_memory()

Status import_memory ( void *  memory)

Import an existing memory as a tensor's backing memory.

Warning
size is expected to be compliant with total_size reported by ITensorInfo.
ownership of memory is not transferred.
tensor shouldn't be memory managed.
padding should be accounted by the client code.
memory must be writable in case of in-place operations
Note
buffer alignment will be checked to be compliant with alignment reported by ITensorInfo.
Parameters
[in]memoryRaw memory pointer to be used as backing memory
Returns
An error status

Definition at line 153 of file TensorAllocator.cpp.

References ITensorAllocator::alignment(), ARM_COMPUTE_RETURN_ERROR_ON, arm_compute::utility::check_aligned(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), and Memory::set_owned_region().

Referenced by NEFFTConvolutionLayer::run(), NEGEMMConvolutionLayer::run(), and arm_compute::test::validation::TEST_CASE().

154 {
155  ARM_COMPUTE_RETURN_ERROR_ON(memory == nullptr);
156  ARM_COMPUTE_RETURN_ERROR_ON(_associated_memory_group != nullptr);
158 
159  _memory.set_owned_region(std::make_unique<MemoryRegion>(memory, info().total_size()));
160  info().set_is_resizable(false);
161 
162  return Status{};
163 }
Status class.
Definition: Error.h:52
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:296
void set_owned_region(std::unique_ptr< IMemoryRegion > region) final
Sets a memory region.
Definition: Memory.cpp:64
size_t alignment() const
Return underlying&#39;s tensor buffer alignment.
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:298
TensorInfo & info()
Return a reference to the tensor&#39;s metadata.
bool check_aligned(void *ptr, const size_t alignment)
Checks if a pointer complies with a given alignment.
Definition: Utility.h:192

◆ init()

void init ( const TensorAllocator allocator,
const Coordinates coords,
TensorInfo sub_info 
)

Shares the same backing memory with another tensor allocator, while the tensor info might be different.

In other words this can be used to create a sub-tensor from another tensor while sharing the same memory.

Note
TensorAllocator have to be of the same specialized type.
Parameters
[in]allocatorThe allocator that owns the backing memory to be shared. Ownership becomes shared afterwards.
[in]coordsThe starting coordinates of the new tensor inside the parent tensor.
[in]sub_infoThe new tensor information (e.g. shape etc)

Definition at line 107 of file TensorAllocator.cpp.

References ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_UNUSED, TensorInfo::format(), ITensorAllocator::info(), TensorInfo::init(), TensorInfo::offset_element_in_bytes(), TensorInfo::offset_first_element_in_bytes(), Memory::region(), TensorInfo::strides_in_bytes(), TensorInfo::tensor_shape(), and TensorInfo::total_size().

Referenced by NEScale::configure(), CpuPoolingAssemblyDispatch::configure(), CPPBoxWithNonMaximaSuppressionLimit::configure(), NERNNLayer::configure(), NEReductionOperation::configure(), NEFFT1D::configure(), NENormalizationLayer::configure(), NEHOGDescriptor::configure(), NEHOGGradient::configure(), NEGaussian5x5::configure(), NESobel5x5::configure(), NESobel7x7::configure(), NEFastCorners::configure(), NEWinogradConvolutionLayer::configure(), NECannyEdge::configure(), NEGenerateProposalsLayer::configure(), NEFFTConvolutionLayer::configure(), NEGEMM::configure(), NEHOGMultiDetection::configure(), NEHarrisCorners::configure(), NEGEMMLowpMatrixMultiplyCore::configure(), NELSTMLayerQuantized::configure(), NELSTMLayer::configure(), NEDeconvolutionLayer::configure(), NEConvolutionSquare< matrix_size >::configure(), NEQLSTMLayer::configure(), NEGEMMConvolutionLayer::configure(), main(), NEQLSTMLayer::NEQLSTMLayer(), NETensorHandle::NETensorHandle(), NumPyAccessor::NumPyAccessor(), and arm_compute::test::validation::TEST_CASE().

108 {
109  // Get parent info
110  const TensorInfo parent_info = allocator.info();
111 
112  // Check if coordinates and new shape are within the parent tensor
113  ARM_COMPUTE_ERROR_ON(!validate_subtensor_shape(parent_info, sub_info, coords));
114  ARM_COMPUTE_UNUSED(validate_subtensor_shape);
115 
116  // Copy pointer to buffer
117  _memory = Memory(allocator._memory.region());
118 
119  // Init tensor info with new dimensions
120  size_t total_size = parent_info.offset_element_in_bytes(coords) + sub_info.total_size() - sub_info.offset_first_element_in_bytes();
121  sub_info.init(sub_info.tensor_shape(), sub_info.format(), parent_info.strides_in_bytes(), parent_info.offset_element_in_bytes(coords), total_size);
122 
123  // Set TensorInfo
124  init(sub_info);
125 }
CPU implementation of memory object.
Definition: Memory.h:37
void init(const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
Shares the same backing memory with another tensor allocator, while the tensor info might be differen...
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
IMemoryRegion * region() final
Region accessor.
Definition: Memory.cpp:48
const Strides & strides_in_bytes() const override
The strides in bytes for accessing each dimension of the tensor.
Definition: TensorInfo.h:241
Format format() const override
Colour format of the image.
Definition: TensorInfo.h:274
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:152
size_t total_size() const override
Returns the total size of the tensor in bytes.
Definition: TensorInfo.h:278
TensorInfo & info()
Return a reference to the tensor&#39;s metadata.
void init(Format format)
Initialize the tensor info with just a format.
Definition: TensorInfo.cpp:109
int32_t offset_element_in_bytes(const Coordinates &pos) const override
The offset in bytes from the beginning of the memory allocation to access the element at position (x...
Definition: TensorInfo.cpp:402
size_t offset_first_element_in_bytes() const override
The offset from the beginning of the memory allocation to the first element of the tensor...
Definition: TensorInfo.h:245
Store the tensor&#39;s metadata.
Definition: TensorInfo.h:45
const TensorShape & tensor_shape() const override
Size for each dimension of the tensor.
Definition: TensorInfo.h:262

◆ operator=() [1/2]

TensorAllocator& operator= ( const TensorAllocator )
delete

Prevent instances of this class from being copy assigned (As this class contains pointers)

◆ operator=() [2/2]

TensorAllocator & operator= ( TensorAllocator &&  o)
noexcept

Allow instances of this class to be moved.

Definition at line 89 of file TensorAllocator.cpp.

References ITensorAllocator::operator=().

90 {
91  if(&o != this)
92  {
93  _owner = o._owner;
94  o._owner = nullptr;
95 
96  _associated_memory_group = o._associated_memory_group;
97  o._associated_memory_group = nullptr;
98 
99  _memory = std::move(o._memory);
100  o._memory = Memory();
101 
102  ITensorAllocator::operator=(std::move(o));
103  }
104  return *this;
105 }
CPU implementation of memory object.
Definition: Memory.h:37
ITensorAllocator & operator=(const ITensorAllocator &)=default
Allow instances of this class to be copied.

◆ set_associated_memory_group()

void set_associated_memory_group ( IMemoryGroup associated_memory_group)

Associates the tensor with a memory group.

Parameters
[in]associated_memory_groupMemory group to associate the tensor with

Definition at line 165 of file TensorAllocator.cpp.

References ARM_COMPUTE_ERROR_ON, IMemoryRegion::buffer(), and Memory::region().

Referenced by Tensor::associate_memory_group().

166 {
167  ARM_COMPUTE_ERROR_ON(associated_memory_group == nullptr);
168  ARM_COMPUTE_ERROR_ON(_associated_memory_group != nullptr && _associated_memory_group != associated_memory_group);
169  ARM_COMPUTE_ERROR_ON(_memory.region() != nullptr && _memory.region()->buffer() != nullptr);
170 
171  _associated_memory_group = associated_memory_group;
172 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
IMemoryRegion * region() final
Region accessor.
Definition: Memory.cpp:48
virtual void * buffer()=0
Returns the pointer to the allocated data.

The documentation for this class was generated from the following files: