Compute Library
 20.02.1
CLTensorAllocator Class Reference

Basic implementation of a CL memory tensor allocator. More...

#include <CLTensorAllocator.h>

Collaboration diagram for CLTensorAllocator:
[legend]

Public Member Functions

 CLTensorAllocator (IMemoryManageable *owner=nullptr, CLRuntimeContext *ctx=nullptr)
 Default constructor. More...
 
 CLTensorAllocator (const CLTensorAllocator &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CLTensorAllocatoroperator= (const CLTensorAllocator &)=delete
 Prevent instances of this class from being copy assigned (As this class contains pointers) More...
 
 CLTensorAllocator (CLTensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
CLTensorAllocatoroperator= (CLTensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
uint8_t * data ()
 Interface to be implemented by the child class to return the pointer to the mapped data. More...
 
const cl::Buffer & cl_data () const
 Interface to be implemented by the child class to return the pointer to the CL data. More...
 
CLQuantization quantization () const
 Wrapped quantization info data accessor. More...
 
uint8_t * map (cl::CommandQueue &q, bool blocking)
 Enqueue a map operation of the allocated buffer on the given queue. More...
 
void unmap (cl::CommandQueue &q, uint8_t *mapping)
 Enqueue an unmap operation of the allocated buffer on the given queue. More...
 
void allocate () override
 Allocate size specified by TensorInfo of OpenCL memory. More...
 
void free () override
 Free allocated OpenCL memory. More...
 
Status import_memory (cl::Buffer buffer)
 Import an existing memory as a tensor's backing memory. More...
 
void set_associated_memory_group (IMemoryGroup *associated_memory_group)
 Associates the tensor with a memory group. More...
 
- Public Member Functions inherited from ITensorAllocator
 ITensorAllocator ()
 Default constructor. More...
 
 ITensorAllocator (const ITensorAllocator &)=default
 Allow instances of this class to be copy constructed. More...
 
ITensorAllocatoroperator= (const ITensorAllocator &)=default
 Allow instances of this class to be copied. More...
 
 ITensorAllocator (ITensorAllocator &&)=default
 Allow instances of this class to be move constructed. More...
 
ITensorAllocatoroperator= (ITensorAllocator &&)=default
 Allow instances of this class to be moved. More...
 
virtual ~ITensorAllocator ()=default
 Default virtual destructor. More...
 
void init (const TensorInfo &input, size_t alignment=0)
 Initialize a tensor based on the passed TensorInfo. More...
 
TensorInfoinfo ()
 Return a reference to the tensor's metadata. More...
 
const TensorInfoinfo () const
 Return a constant reference to the tensor's metadata. More...
 
size_t alignment () const
 Return underlying's tensor buffer alignment. More...
 

Detailed Description

Basic implementation of a CL memory tensor allocator.

Definition at line 42 of file CLTensorAllocator.h.

Constructor & Destructor Documentation

◆ CLTensorAllocator() [1/3]

CLTensorAllocator ( IMemoryManageable owner = nullptr,
CLRuntimeContext ctx = nullptr 
)

Default constructor.

Parameters
[in]owner(Optional) Owner of the allocator.
[in]ctx(Optional) Runtime context.

Definition at line 107 of file CLTensorAllocator.cpp.

108  : _ctx(ctx), _owner(owner), _associated_memory_group(nullptr), _memory(), _mapping(nullptr), _scale(), _offset()
109 {
110 }

◆ CLTensorAllocator() [2/3]

CLTensorAllocator ( const CLTensorAllocator )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ CLTensorAllocator() [3/3]

Allow instances of this class to be moved.

Member Function Documentation

◆ allocate()

void allocate ( )
overridevirtual

Allocate size specified by TensorInfo of OpenCL memory.

Note
: The tensor must not already be allocated when calling this function.

Implements ITensorAllocator.

Definition at line 127 of file CLTensorAllocator.cpp.

128 {
129  // Allocate tensor backing memory
130  if(_associated_memory_group == nullptr)
131  {
132  // Perform memory allocation
133  if(_ctx == nullptr)
134  {
135  auto legacy_ctx = CLCoreRuntimeContext(nullptr, CLScheduler::get().context(), CLScheduler::get().queue());
136  _memory.set_owned_region(allocate_region(&legacy_ctx, info().total_size(), 0));
137  }
138  else
139  {
140  _memory.set_owned_region(allocate_region(_ctx->core_runtime_context(), info().total_size(), 0));
141  }
142  }
143  else
144  {
145  _associated_memory_group->finalize_memory(_owner, _memory, info().total_size(), alignment());
146  }
147 
148  // Allocate and fill the quantization parameter arrays
150  {
151  const size_t pad_size = 0;
152  populate_quantization_info(_scale, _offset, info().quantization_info(), pad_size);
153  }
154 
155  // Lock allocator
156  info().set_is_resizable(false);
157 }
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:99
CLCoreRuntimeContext * core_runtime_context()
virtual void finalize_memory(IMemoryManageable *obj, IMemory &obj_memory, size_t size, size_t alignment)=0
Finalizes memory for a given object.
bool is_data_type_quantized_per_channel(DataType dt)
Check if a given data type is of per channel type.
Definition: Utils.h:1194
void set_owned_region(std::unique_ptr< IMemoryRegion > region) final
Sets a memory region.
Definition: CLMemory.cpp:76
size_t total_size() const override
Returns the total size of the tensor in bytes.
Definition: TensorInfo.h:273
size_t alignment() const
Return underlying's tensor buffer alignment.
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:293
TensorInfo & info()
Return a reference to the tensor's metadata.

References ITensorAllocator::alignment(), CLRuntimeContext::core_runtime_context(), arm_compute::test::validation::data_type, IMemoryGroup::finalize_memory(), CLScheduler::get(), ITensorAllocator::info(), arm_compute::is_data_type_quantized_per_channel(), TensorInfo::set_is_resizable(), CLMemory::set_owned_region(), and TensorInfo::total_size().

Referenced by CLTensorHandle::allocate(), CLRNNLayer::configure(), CLFFT2D::configure(), CLFFT1D::configure(), CLMeanStdDev::configure(), CLHOGDescriptor::configure(), CLL2NormalizeLayer::configure(), CLArgMinMaxLayer::configure(), CLHOGGradient::configure(), CLGaussian5x5::configure(), CLSobel5x5::configure(), CLSobel7x7::configure(), CLCannyEdge::configure(), CLFastCorners::configure(), CLLocallyConnectedLayer::configure(), CLWinogradConvolutionLayer::configure(), CLHarrisCorners::configure(), CLHOGMultiDetection::configure(), CLGEMMLowpMatrixMultiplyCore::configure(), CLFFTConvolutionLayer::configure(), CLGenerateProposalsLayer::configure(), CLLSTMLayerQuantized::configure(), CLLSTMLayer::configure(), CLDirectDeconvolutionLayer::configure(), CLGEMMDeconvolutionLayer::configure(), CLGEMMConvolutionLayer::configure(), CLLocallyConnectedLayer::prepare(), CLWinogradConvolutionLayer::prepare(), CLGEMMLowpMatrixMultiplyCore::prepare(), CLFFTConvolutionLayer::prepare(), CLDirectDeconvolutionLayer::prepare(), CLGEMMDeconvolutionLayer::prepare(), CLLSTMLayerQuantized::prepare(), CLGEMM::prepare(), CLFullyConnectedLayer::prepare(), CLGEMMConvolutionLayer::prepare(), CLGEMMReshapeRHSMatrixKernelManaged::run(), CLConvertFullyConnectedWeightsManaged::run(), CLFullyConnectedLayerReshapeWeightsManaged::run(), CLConvolutionLayerReshapeWeightsTransform::run(), and arm_compute::test::validation::TEST_CASE().

◆ cl_data()

const cl::Buffer & cl_data ( ) const

Interface to be implemented by the child class to return the pointer to the CL data.

Returns
pointer to the CL data.

Definition at line 122 of file CLTensorAllocator.cpp.

123 {
124  return _memory.region() == nullptr ? _empty_buffer : _memory.cl_region()->cl_data();
125 }
const cl::Buffer & cl_data() const
Returns the underlying CL buffer.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49

References ICLMemoryRegion::cl_data(), CLMemory::cl_region(), and CLMemory::region().

Referenced by CLTensor::cl_buffer().

◆ data()

uint8_t * data ( )

Interface to be implemented by the child class to return the pointer to the mapped data.

Returns
pointer to the mapped data.

Definition at line 117 of file CLTensorAllocator.cpp.

118 {
119  return _mapping;
120 }

◆ free()

void free ( )
overridevirtual

Free allocated OpenCL memory.

Note
The tensor must have been allocated when calling this function.

Implements ITensorAllocator.

Definition at line 159 of file CLTensorAllocator.cpp.

160 {
161  _mapping = nullptr;
162  _memory.set_region(nullptr);
163  clear_quantization_arrays(_scale, _offset);
164  info().set_is_resizable(true);
165 }
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:293
TensorInfo & info()
Return a reference to the tensor's metadata.
void set_region(IMemoryRegion *region) final
Sets a memory region.
Definition: CLMemory.cpp:69

References ITensorAllocator::info(), TensorInfo::set_is_resizable(), and CLMemory::set_region().

Referenced by CLTensorHandle::free(), CLWinogradConvolutionLayer::prepare(), CLFFTConvolutionLayer::prepare(), CLGEMMDeconvolutionLayer::prepare(), CLDirectDeconvolutionLayer::prepare(), CLLSTMLayerQuantized::prepare(), CLGEMMConvolutionLayer::prepare(), CLGEMMReshapeRHSMatrixKernelManaged::release(), CLConvertFullyConnectedWeightsManaged::release(), CLFullyConnectedLayerReshapeWeightsManaged::release(), CLConvolutionLayerReshapeWeightsTransform::release(), CLTensorHandle::release_if_unused(), and arm_compute::test::validation::TEST_CASE().

◆ import_memory()

Status import_memory ( cl::Buffer  buffer)

Import an existing memory as a tensor's backing memory.

Warning
memory should have been created under the same context that ACL uses.
memory is expected to be aligned with the device requirements.
tensor shouldn't be memory managed.
ownership of memory is not transferred.
memory must be writable in case of in-place operations
padding should be accounted by the client code.
Note
buffer size will be checked to be compliant with total_size reported by ITensorInfo.
Parameters
[in]bufferBuffer to be used as backing memory
Returns
An error status

Definition at line 167 of file CLTensorAllocator.cpp.

168 {
169  ARM_COMPUTE_RETURN_ERROR_ON(buffer.get() == nullptr);
170  ARM_COMPUTE_RETURN_ERROR_ON(buffer.getInfo<CL_MEM_SIZE>() < info().total_size());
171  ARM_COMPUTE_RETURN_ERROR_ON(buffer.getInfo<CL_MEM_CONTEXT>().get() != CLScheduler::get().context().get());
172  ARM_COMPUTE_RETURN_ERROR_ON(_associated_memory_group != nullptr);
173 
174  if(_ctx == nullptr)
175  {
176  auto legacy_ctx = CLCoreRuntimeContext(nullptr, CLScheduler::get().context(), CLScheduler::get().queue());
177  _memory.set_owned_region(support::cpp14::make_unique<CLBufferMemoryRegion>(buffer, &legacy_ctx));
178  }
179  else
180  {
181  _memory.set_owned_region(support::cpp14::make_unique<CLBufferMemoryRegion>(buffer, _ctx->core_runtime_context()));
182  }
183 
184  info().set_is_resizable(false);
185  return Status{};
186 }
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:99
CLCoreRuntimeContext * core_runtime_context()
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:296
cl::Context & context()
Accessor for the associated CL context.
Definition: CLScheduler.cpp:34
void set_owned_region(std::unique_ptr< IMemoryRegion > region) final
Sets a memory region.
Definition: CLMemory.cpp:76
ITensorInfo & set_is_resizable(bool is_resizable) override
Set the flag whether the tensor size can be changed.
Definition: TensorInfo.h:293
TensorInfo & info()
Return a reference to the tensor's metadata.

References ARM_COMPUTE_RETURN_ERROR_ON, CLScheduler::context(), CLRuntimeContext::core_runtime_context(), CLScheduler::get(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), and CLMemory::set_owned_region().

Referenced by CLFFTConvolutionLayer::run(), and arm_compute::test::validation::TEST_CASE().

◆ map()

uint8_t * map ( cl::CommandQueue &  q,
bool  blocking 
)

Enqueue a map operation of the allocated buffer on the given queue.

Parameters
[in,out]qThe CL command queue to use for the mapping operation.
[in]blockingIf true, then the mapping will be ready to use by the time this method returns, else it is the caller's responsibility to flush the queue and wait for the mapping operation to have completed before using the returned mapping pointer.
Returns
The mapping address.

Definition at line 223 of file CLTensorAllocator.cpp.

224 {
225  ARM_COMPUTE_ERROR_ON(_mapping != nullptr);
226  ARM_COMPUTE_ERROR_ON(_memory.region() == nullptr);
227  ARM_COMPUTE_ERROR_ON(_memory.region()->buffer() != nullptr);
228 
229  _mapping = reinterpret_cast<uint8_t *>(_memory.cl_region()->map(q, blocking));
230  return _mapping;
231 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
virtual void * buffer()=0
Returns the pointer to the allocated data.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49
virtual void * map(cl::CommandQueue &q, bool blocking)=0
Enqueue a map operation of the allocated buffer on the given queue.

References ARM_COMPUTE_ERROR_ON, IMemoryRegion::buffer(), CLMemory::cl_region(), ICLMemoryRegion::map(), and CLMemory::region().

◆ operator=() [1/2]

CLTensorAllocator& operator= ( const CLTensorAllocator )
delete

Prevent instances of this class from being copy assigned (As this class contains pointers)

◆ operator=() [2/2]

CLTensorAllocator& operator= ( CLTensorAllocator &&  )
default

Allow instances of this class to be moved.

◆ quantization()

CLQuantization quantization ( ) const

Wrapped quantization info data accessor.

Returns
A wrapped quantization info object.

Definition at line 112 of file CLTensorAllocator.cpp.

113 {
114  return { &_scale, &_offset };
115 }

Referenced by CLTensor::quantization().

◆ set_associated_memory_group()

void set_associated_memory_group ( IMemoryGroup associated_memory_group)

Associates the tensor with a memory group.

Parameters
[in]associated_memory_groupMemory group to associate the tensor with

Definition at line 188 of file CLTensorAllocator.cpp.

189 {
190  ARM_COMPUTE_ERROR_ON(associated_memory_group == nullptr);
191  ARM_COMPUTE_ERROR_ON(_associated_memory_group != nullptr && _associated_memory_group != associated_memory_group);
192  ARM_COMPUTE_ERROR_ON(_memory.region() != nullptr && _memory.cl_region()->cl_data().get() != nullptr);
193 
194  _associated_memory_group = associated_memory_group;
195 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
const cl::Buffer & cl_data() const
Returns the underlying CL buffer.
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49

References ARM_COMPUTE_ERROR_ON, ICLMemoryRegion::cl_data(), CLMemory::cl_region(), and CLMemory::region().

Referenced by CLTensor::associate_memory_group().

◆ unmap()

void unmap ( cl::CommandQueue &  q,
uint8_t *  mapping 
)

Enqueue an unmap operation of the allocated buffer on the given queue.

Note
This method simply enqueue the unmap operation, it is the caller's responsibility to flush the queue and make sure the unmap is finished before the memory is accessed by the device.
Parameters
[in,out]qThe CL command queue to use for the mapping operation.
[in]mappingThe cpu mapping to unmap.

Definition at line 233 of file CLTensorAllocator.cpp.

234 {
235  ARM_COMPUTE_ERROR_ON(_mapping == nullptr);
236  ARM_COMPUTE_ERROR_ON(_mapping != mapping);
237  ARM_COMPUTE_ERROR_ON(_memory.region() == nullptr);
238  ARM_COMPUTE_ERROR_ON(_memory.region()->buffer() == nullptr);
239  ARM_COMPUTE_UNUSED(mapping);
240 
241  _memory.cl_region()->unmap(q);
242  _mapping = nullptr;
243 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
virtual void * buffer()=0
Returns the pointer to the allocated data.
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:152
IMemoryRegion * region() final
Region accessor.
Definition: CLMemory.cpp:59
ICLMemoryRegion * cl_region()
OpenCL Region accessor.
Definition: CLMemory.cpp:49
virtual void unmap(cl::CommandQueue &q)=0
Enqueue an unmap operation of the allocated buffer on the given queue.

References ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_UNUSED, IMemoryRegion::buffer(), CLMemory::cl_region(), CLMemory::region(), and ICLMemoryRegion::unmap().


The documentation for this class was generated from the following files: