Compute Library
 19.08
CLTensorAllocator Class Reference

Basic implementation of a CL memory tensor allocator. More...

#include <CLTensorAllocator.h>

Collaboration diagram for CLTensorAllocator:
[legend]

Public Member Functions

 CLTensorAllocator (CLTensor *owner=nullptr)
 Default constructor. More...
 
 CLTensorAllocator (const CLTensorAllocator &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CLTensorAllocatoroperator= (const CLTensorAllocator &)=delete
 Prevent instances of this class from being copy assigned (As this class contains pointers) More...
 
 CLTensorAllocator (CLTensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
CLTensorAllocatoroperator= (CLTensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
uint8_t * data ()
 Interface to be implemented by the child class to return the pointer to the mapped data. More...
 
const cl::Buffer & cl_data () const
 Interface to be implemented by the child class to return the pointer to the CL data. More...
 
CLQuantization quantization () const
 Wrapped quantization info data accessor. More...
 
uint8_t * map (cl::CommandQueue &q, bool blocking)
 Enqueue a map operation of the allocated buffer on the given queue. More...
 
void unmap (cl::CommandQueue &q, uint8_t *mapping)
 Enqueue an unmap operation of the allocated buffer on the given queue. More...
 
void allocate () override
 Allocate size specified by TensorInfo of OpenCL memory. More...
 
void free () override
 Free allocated OpenCL memory. More...
 
Status import_memory (cl::Buffer buffer)
 Import an existing memory as a tensor's backing memory. More...
 
void set_associated_memory_group (CLMemoryGroup *associated_memory_group)
 Associates the tensor with a memory group. More...
 
- Public Member Functions inherited from ITensorAllocator
 ITensorAllocator ()
 Default constructor. More...
 
 ITensorAllocator (const ITensorAllocator &)=default
 Allow instances of this class to be copy constructed. More...
 
ITensorAllocatoroperator= (const ITensorAllocator &)=default
 Allow instances of this class to be copied. More...
 
 ITensorAllocator (ITensorAllocator &&)=default
 Allow instances of this class to be move constructed. More...
 
ITensorAllocatoroperator= (ITensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
virtual ~ITensorAllocator ()=default
 Default virtual destructor. More...
 
void init (const TensorInfo &input, size_t alignment=0)
 Initialize a tensor based on the passed TensorInfo. More...
 
TensorInfoinfo ()
 Return a reference to the tensor's metadata. More...
 
const TensorInfoinfo () const
 Return a constant reference to the tensor's metadata. More...
 
size_t alignment () const
 Return underlying's tensor buffer alignment. More...
 

Detailed Description

Basic implementation of a CL memory tensor allocator.

Definition at line 45 of file CLTensorAllocator.h.

Constructor & Destructor Documentation

◆ CLTensorAllocator() [1/3]

CLTensorAllocator ( CLTensor owner = nullptr)

Default constructor.

Parameters
[in]owner(Optional) Owner of the allocator.

Definition at line 99 of file CLTensorAllocator.cpp.

100  : _associated_memory_group(nullptr), _memory(), _mapping(nullptr), _owner(owner), _scale(), _offset()
101 {
102 }

◆ CLTensorAllocator() [2/3]

CLTensorAllocator ( const CLTensorAllocator )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ CLTensorAllocator() [3/3]

Allow instances of this class to be moved.

Member Function Documentation

◆ allocate()

void allocate ( )
overridevirtual

Allocate size specified by TensorInfo of OpenCL memory.

Note
: The tensor must not already be allocated when calling this function.

Implements ITensorAllocator.

Definition at line 119 of file CLTensorAllocator.cpp.

120 {
121  // Allocate tensor backing memory
122  if(_associated_memory_group == nullptr)
123  {
124  if(_memory.region() != nullptr && _memory.cl_region()->cl_data().get() != nullptr)
125  {
126  // Memory is already allocated. Reuse it if big enough, otherwise fire an assertion
127  ARM_COMPUTE_ERROR_ON_MSG(info().total_size() > _memory.region()->size(),
128  "Reallocation of a bigger memory region is not allowed!");
129  }
130  else
131  {
132  // Perform memory allocation
133  _memory.set_owned_region(allocate_region(CLScheduler::get().context(), info().total_size(), 0));
134  }
135  }
136  else
137  {
138  _associated_memory_group->finalize_memory(_owner, _memory, info().total_size());
139  }
140 
141  // Allocate and fill the quantization parameter arrays
143  {
144  const size_t pad_size = 0;
145  populate_quantization_info(_scale, _offset, info().quantization_info(), pad_size);
146  }
147 
148  // Lock allocator
149  info().set_is_resizable(false);
150 }
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:41
const cl::Buffer & cl_data() const
Returns the underlying CL buffer.
void set_owned_region(std::unique_ptr< IMemoryRegion > region) final
Sets a memory region.
Definition: CLMemory.cpp:76
void finalize_memory(TensorType *obj, IMemory &obj_memory, size_t size, size_t alignment=0)
Finalizes memory for a given object.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:280
TensorInfo & info()
Return a reference to the tensor's metadata.
quantized, symmetric per channel fixed-point 8-bit number
size_t size() const
Memory region size accessor.
Definition: IMemoryRegion.h:73
#define ARM_COMPUTE_ERROR_ON_MSG(cond,...)
Definition: Error.h:328

References ARM_COMPUTE_ERROR_ON_MSG, ICLMemoryRegion::cl_data(), CLMemory::cl_region(), arm_compute::test::validation::data_type, MemoryGroupBase< TensorType >::finalize_memory(), CLScheduler::get(), ITensorAllocator::info(), arm_compute::QSYMM8_PER_CHANNEL, CLMemory::region(), TensorInfo::set_is_resizable(), CLMemory::set_owned_region(), and IMemoryRegion::size().

Referenced by CLTensorHandle::allocate(), CLRNNLayer::configure(), CLFFT2D::configure(), CLFFT1D::configure(), CLMeanStdDev::configure(), CLHOGDescriptor::configure(), CLL2NormalizeLayer::configure(), CLHOGGradient::configure(), CLGaussian5x5::configure(), CLSoftmaxLayer::configure(), CLSobel5x5::configure(), CLSobel7x7::configure(), CLCannyEdge::configure(), CLFastCorners::configure(), CLLocallyConnectedLayer::configure(), CLWinogradConvolutionLayer::configure(), CLDepthwiseConvolutionLayer3x3::configure(), CLHarrisCorners::configure(), CLGEMMLowpMatrixMultiplyCore::configure(), CLHOGMultiDetection::configure(), CLFFTConvolutionLayer::configure(), CLGenerateProposalsLayer::configure(), CLLSTMLayerQuantized::configure(), CLLSTMLayer::configure(), CLDirectDeconvolutionLayer::configure(), CLGEMMDeconvolutionLayer::configure(), CLFullyConnectedLayer::configure(), CLGEMMConvolutionLayer::configure(), CLDepthwiseConvolutionLayer::configure(), CLLocallyConnectedLayer::prepare(), CLGEMMLowpMatrixMultiplyCore::prepare(), CLWinogradConvolutionLayer::prepare(), CLDepthwiseConvolutionLayer3x3::prepare(), CLGEMM::prepare(), CLFFTConvolutionLayer::prepare(), CLGEMMDeconvolutionLayer::prepare(), CLDirectDeconvolutionLayer::prepare(), CLFullyConnectedLayer::prepare(), CLLSTMLayerQuantized::prepare(), CLGEMMConvolutionLayer::prepare(), CLDepthwiseConvolutionLayer::prepare(), and arm_compute::test::validation::TEST_CASE().

◆ cl_data()

const cl::Buffer & cl_data ( ) const

Interface to be implemented by the child class to return the pointer to the CL data.

Returns
pointer to the CL data.

Definition at line 114 of file CLTensorAllocator.cpp.

115 {
116  return _memory.region() == nullptr ? _empty_buffer : _memory.cl_region()->cl_data();
117 }
const cl::Buffer & cl_data() const
Returns the underlying CL buffer.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49

References ICLMemoryRegion::cl_data(), CLMemory::cl_region(), and CLMemory::region().

Referenced by CLTensor::cl_buffer().

◆ data()

uint8_t * data ( )

Interface to be implemented by the child class to return the pointer to the mapped data.

Returns
pointer to the mapped data.

Definition at line 109 of file CLTensorAllocator.cpp.

110 {
111  return _mapping;
112 }

◆ free()

void free ( )
overridevirtual

Free allocated OpenCL memory.

Note
The tensor must have been allocated when calling this function.

Implements ITensorAllocator.

Definition at line 152 of file CLTensorAllocator.cpp.

153 {
154  _mapping = nullptr;
155  _memory.set_region(nullptr);
156  clear_quantization_arrays(_scale, _offset);
157  info().set_is_resizable(true);
158 }
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:280
TensorInfo & info()
Return a reference to the tensor's metadata.
void set_region(IMemoryRegion *region) final
Sets a memory region.
Definition: CLMemory.cpp:69

References ITensorAllocator::info(), TensorInfo::set_is_resizable(), and CLMemory::set_region().

Referenced by CLTensorHandle::free(), CLWinogradConvolutionLayer::prepare(), CLFFTConvolutionLayer::prepare(), CLGEMMDeconvolutionLayer::prepare(), CLDirectDeconvolutionLayer::prepare(), CLLSTMLayerQuantized::prepare(), CLGEMMConvolutionLayer::prepare(), CLTensorHandle::release_if_unused(), and arm_compute::test::validation::TEST_CASE().

◆ import_memory()

Status import_memory ( cl::Buffer  buffer)

Import an existing memory as a tensor's backing memory.

Warning
memory should have been created under the same context that ACL uses.
memory is expected to be aligned with the device requirements.
tensor shouldn't be memory managed.
ownership of memory is not transferred.
memory must be writable in case of in-place operations
padding should be accounted by the client code.
Note
buffer size will be checked to be compliant with total_size reported by ITensorInfo.
Parameters
[in]bufferBuffer to be used as backing memory
Returns
An error status

Definition at line 160 of file CLTensorAllocator.cpp.

161 {
162  ARM_COMPUTE_RETURN_ERROR_ON(buffer.get() == nullptr);
163  ARM_COMPUTE_RETURN_ERROR_ON(buffer.getInfo<CL_MEM_SIZE>() < info().total_size());
164  ARM_COMPUTE_RETURN_ERROR_ON(buffer.getInfo<CL_MEM_CONTEXT>().get() != CLScheduler::get().context().get());
165  ARM_COMPUTE_RETURN_ERROR_ON(_associated_memory_group != nullptr);
166 
167  _memory.set_owned_region(support::cpp14::make_unique<CLBufferMemoryRegion>(buffer));
168  info().set_is_resizable(false);
169 
170  return Status{};
171 }
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:41
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:244
void set_owned_region(std::unique_ptr< IMemoryRegion > region) final
Sets a memory region.
Definition: CLMemory.cpp:76
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:280
TensorInfo & info()
Return a reference to the tensor's metadata.
cl::Context & context()
Accessor for the associated CL context.
Definition: CLScheduler.h:91

References ARM_COMPUTE_RETURN_ERROR_ON, CLScheduler::context(), CLScheduler::get(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), and CLMemory::set_owned_region().

Referenced by CLFFTConvolutionLayer::run(), and arm_compute::test::validation::TEST_CASE().

◆ map()

uint8_t * map ( cl::CommandQueue &  q,
bool  blocking 
)

Enqueue a map operation of the allocated buffer on the given queue.

Parameters
[in,out]qThe CL command queue to use for the mapping operation.
[in]blockingIf true, then the mapping will be ready to use by the time this method returns, else it is the caller's responsibility to flush the queue and wait for the mapping operation to have completed before using the returned mapping pointer.
Returns
The mapping address.

Definition at line 193 of file CLTensorAllocator.cpp.

194 {
195  ARM_COMPUTE_ERROR_ON(_mapping != nullptr);
196  ARM_COMPUTE_ERROR_ON(_memory.region() == nullptr);
197  ARM_COMPUTE_ERROR_ON(_memory.region()->buffer() != nullptr);
198 
199  _mapping = reinterpret_cast<uint8_t *>(_memory.cl_region()->map(q, blocking));
200  return _mapping;
201 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:337
virtual void * buffer()=0
Returns the pointer to the allocated data.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49
virtual void * map(cl::CommandQueue &q, bool blocking)=0
Enqueue a map operation of the allocated buffer on the given queue.

References ARM_COMPUTE_ERROR_ON, IMemoryRegion::buffer(), CLMemory::cl_region(), ICLMemoryRegion::map(), and CLMemory::region().

◆ operator=() [1/2]

CLTensorAllocator& operator= ( const CLTensorAllocator )
delete

Prevent instances of this class from being copy assigned (As this class contains pointers)

◆ operator=() [2/2]

CLTensorAllocator& operator= ( CLTensorAllocator &&  )
default

Allow instances of this class to be moved.

◆ quantization()

CLQuantization quantization ( ) const

Wrapped quantization info data accessor.

Returns
A wrapped quantization info object.

Definition at line 104 of file CLTensorAllocator.cpp.

105 {
106  return { &_scale, &_offset };
107 }

Referenced by CLTensor::quantization().

◆ set_associated_memory_group()

void set_associated_memory_group ( CLMemoryGroup associated_memory_group)

Associates the tensor with a memory group.

Parameters
[in]associated_memory_groupMemory group to associate the tensor with

Definition at line 173 of file CLTensorAllocator.cpp.

174 {
175  ARM_COMPUTE_ERROR_ON(associated_memory_group == nullptr);
176  ARM_COMPUTE_ERROR_ON(_associated_memory_group != nullptr);
177  ARM_COMPUTE_ERROR_ON(_memory.region() != nullptr && _memory.cl_region()->cl_data().get() != nullptr);
178 
179  _associated_memory_group = associated_memory_group;
180 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:337
const cl::Buffer & cl_data() const
Returns the underlying CL buffer.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49

References ARM_COMPUTE_ERROR_ON, ICLMemoryRegion::cl_data(), CLMemory::cl_region(), and CLMemory::region().

◆ unmap()

void unmap ( cl::CommandQueue &  q,
uint8_t *  mapping 
)

Enqueue an unmap operation of the allocated buffer on the given queue.

Note
This method simply enqueue the unmap operation, it is the caller's responsibility to flush the queue and make sure the unmap is finished before the memory is accessed by the device.
Parameters
[in,out]qThe CL command queue to use for the mapping operation.
[in]mappingThe cpu mapping to unmap.

Definition at line 203 of file CLTensorAllocator.cpp.

204 {
205  ARM_COMPUTE_ERROR_ON(_mapping == nullptr);
206  ARM_COMPUTE_ERROR_ON(_mapping != mapping);
207  ARM_COMPUTE_ERROR_ON(_memory.region() == nullptr);
208  ARM_COMPUTE_ERROR_ON(_memory.region()->buffer() == nullptr);
209  ARM_COMPUTE_UNUSED(mapping);
210 
211  _memory.cl_region()->unmap(q);
212  _mapping = nullptr;
213 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:337
virtual void * buffer()=0
Returns the pointer to the allocated data.
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:160
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49
virtual void unmap(cl::CommandQueue &q)=0
Enqueue an unmap operation of the allocated buffer on the given queue.

References ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_UNUSED, IMemoryRegion::buffer(), CLMemory::cl_region(), CLMemory::region(), and ICLMemoryRegion::unmap().


The documentation for this class was generated from the following files: