CLTensorAllocator Class Reference

Basic implementation of a CL memory tensor allocator. More...

#include <CLTensorAllocator.h>

Collaboration diagram for CLTensorAllocator:

Public Member Functions
	CLTensorAllocator (IMemoryManageable *owner=nullptr, CLRuntimeContext *ctx=nullptr)
	Default constructor. More...
	CLTensorAllocator (const CLTensorAllocator &)=delete
	Prevent instances of this class from being copied (As this class contains pointers) More...
CLTensorAllocator &	operator= (const CLTensorAllocator &)=delete
	Prevent instances of this class from being copy assigned (As this class contains pointers) More...
	CLTensorAllocator (CLTensorAllocator &&)=default
	Allow instances of this class to be moved. More...
CLTensorAllocator &	operator= (CLTensorAllocator &&)=default
	Allow instances of this class to be moved. More...
uint8_t *	data ()
	Interface to be implemented by the child class to return the pointer to the mapped data. More...
const cl::Buffer &	cl_data () const
	Interface to be implemented by the child class to return the pointer to the CL data. More...
CLQuantization	quantization () const
	Wrapped quantization info data accessor. More...
uint8_t *	map (cl::CommandQueue &q, bool blocking)
	Enqueue a map operation of the allocated buffer on the given queue. More...
void	unmap (cl::CommandQueue &q, uint8_t *mapping)
	Enqueue an unmap operation of the allocated buffer on the given queue. More...
void	allocate () override
	Allocate size specified by TensorInfo of OpenCL memory. More...
void	free () override
	Free allocated OpenCL memory. More...
Status	import_memory (cl::Buffer buffer)
	Import an existing memory as a tensor's backing memory. More...
void	set_associated_memory_group (IMemoryGroup *associated_memory_group)
	Associates the tensor with a memory group. More...
Public Member Functions inherited from ITensorAllocator
	ITensorAllocator ()=default
	Default constructor. More...
	ITensorAllocator (const ITensorAllocator &)=default
	Allow instances of this class to be copy constructed. More...
ITensorAllocator &	operator= (const ITensorAllocator &)=default
	Allow instances of this class to be copied. More...
	ITensorAllocator (ITensorAllocator &&)=default
	Allow instances of this class to be move constructed. More...
ITensorAllocator &	operator= (ITensorAllocator &&)=default
	Allow instances of this class to be moved. More...
virtual	~ITensorAllocator ()=default
	Default virtual destructor. More...
void	init (const TensorInfo &input, size_t alignment=0)
	Initialize a tensor based on the passed TensorInfo. More...
void	soft_init (TensorInfo &input, size_t alignment=0)
	Initialize a tensor based with a reference TensorInfo. More...
TensorInfo &	info ()
	Return a reference to the tensor's metadata. More...
const TensorInfo &	info () const
	Return a constant reference to the tensor's metadata. More...
size_t	alignment () const
	Return underlying's tensor buffer alignment. More...

Static Public Member Functions
static void	set_global_allocator (IAllocator *allocator)
	Sets global allocator that will be used by all CLTensor objects. More...

Detailed Description

Basic implementation of a CL memory tensor allocator.

Definition at line 42 of file CLTensorAllocator.h.

Constructor & Destructor Documentation

CLTensorAllocator() [1/3]

CLTensorAllocator	(	IMemoryManageable *	owner = nullptr,
		CLRuntimeContext *	ctx = nullptr
	)

Default constructor.

Parameters

[in]	owner	(Optional) Owner of the allocator.
[in]	ctx	(Optional) Runtime context.

Definition at line 111 of file CLTensorAllocator.cpp.

    : _ctx(ctx), _owner(owner), _associated_memory_group(nullptr), _memory(), _mapping(nullptr), _scale(), _offset()
{
}

CLTensorAllocator() [2/3]

CLTensorAllocator ( const CLTensorAllocator &  )

delete

Prevent instances of this class from being copied (As this class contains pointers)

CLTensorAllocator() [3/3]

CLTensorAllocator ( CLTensorAllocator &&  )

default

Allow instances of this class to be moved.

Member Function Documentation

allocate()

void allocate ( )

overridevirtual

Allocate size specified by TensorInfo of OpenCL memory.

Note

: The tensor must not already be allocated when calling this function.

Implements ITensorAllocator.

Definition at line 131 of file CLTensorAllocator.cpp.

{
    // Allocate tensor backing memory
    if (_associated_memory_group == nullptr)
    {
        // Perform memory allocation
        if (static_global_cl_allocator != nullptr)
        {
            _memory.set_owned_region(static_global_cl_allocator->make_region(info().total_size(), 0));
        }
        else
        {
            _memory.set_owned_region(allocate_region(info().total_size(), 0));
        }
    }
    else
    {
        // Finalize memory management instead
        _associated_memory_group->finalize_memory(_owner, _memory, info().total_size(), alignment());
    }
 
    // Allocate and fill the quantization parameter arrays
    if (is_data_type_quantized_per_channel(info().data_type()))
    {
        const size_t pad_size = 0;
        populate_quantization_info(_scale, _offset, info().quantization_info(), pad_size);
    }
 
    // Lock allocator
    info().set_is_resizable(false);
}

References ITensorAllocator::alignment(), arm_compute::test::validation::data_type, IMemoryGroup::finalize_memory(), ITensorAllocator::info(), arm_compute::is_data_type_quantized_per_channel(), TensorInfo::set_is_resizable(), and CLMemory::set_owned_region().

Referenced by CLTensorHandle::allocate(), CLAuxTensorHandler::CLAuxTensorHandler(), arm_compute::test::validation::compute_float_target_in_place(), CLReduceMean::configure(), CLFFT2D::configure(), CLFFT1D::configure(), CLRNNLayer::configure(), CLDepthwiseConvolutionLayer::configure(), CLL2NormalizeLayer::configure(), CLArgMinMaxLayer::configure(), CLReductionOperation::configure(), CLInstanceNormalizationLayer::configure(), CLFFTConvolutionLayer::configure(), CLGenerateProposalsLayer::configure(), CLGEMMDeconvolutionLayer::configure(), CLDirectDeconvolutionLayer::configure(), CLLSTMLayerQuantized::configure(), CLLSTMLayer::configure(), CLQLSTMLayer::configure(), arm_compute::manage_workspace(), CLDepthwiseConvolutionLayer::prepare(), CLGEMMDeconvolutionLayer::prepare(), CLFFTConvolutionLayer::prepare(), CLDirectDeconvolutionLayer::prepare(), CLLSTMLayerQuantized::prepare(), CLQLSTMLayer::prepare(), CLConvertFullyConnectedWeightsManaged::run(), and arm_compute::test::validation::TEST_CASE().

cl_data()

const cl::Buffer & cl_data

(

)

const

Interface to be implemented by the child class to return the pointer to the CL data.

Returns

pointer to the CL data.

Definition at line 126 of file CLTensorAllocator.cpp.

{
    return _memory.region() == nullptr ? _empty_buffer : _memory.cl_region()->cl_data();
}

References ICLMemoryRegion::cl_data(), CLMemory::cl_region(), and CLMemory::region().

Referenced by CLTensor::cl_buffer().

data()

uint8_t * data

(

)

Interface to be implemented by the child class to return the pointer to the mapped data.

Returns

pointer to the mapped data.

Definition at line 121 of file CLTensorAllocator.cpp.

{
    return _mapping;
}

free()

void free ( )

overridevirtual

Free allocated OpenCL memory.

Note

The tensor must have been allocated when calling this function.

Implements ITensorAllocator.

Definition at line 163 of file CLTensorAllocator.cpp.

{
    _mapping = nullptr;
    _memory.set_region(nullptr);
    clear_quantization_arrays(_scale, _offset);
    info().set_is_resizable(true);
}

References ITensorAllocator::info(), TensorInfo::set_is_resizable(), and CLMemory::set_region().

Referenced by CLTensorHandle::free(), CLFFTConvolutionLayer::prepare(), CLGEMMDeconvolutionLayer::prepare(), CLDirectDeconvolutionLayer::prepare(), CLLSTMLayerQuantized::prepare(), CLConvertFullyConnectedWeightsManaged::release(), CLTensorHandle::release_if_unused(), and arm_compute::test::validation::TEST_CASE().

import_memory()

Status import_memory

(

cl::Buffer

buffer

)

Import an existing memory as a tensor's backing memory.

Warning

memory should have been created under the same context that Compute Library uses.

memory is expected to be aligned with the device requirements.

tensor shouldn't be memory managed.

ownership of memory is not transferred.

memory must be writable in case of in-place operations

padding should be accounted by the client code.

Note

buffer size will be checked to be compliant with total_size reported by ITensorInfo.

Parameters

[in]

buffer

Buffer to be used as backing memory

Returns

An error status

Definition at line 171 of file CLTensorAllocator.cpp.

{
    ARM_COMPUTE_RETURN_ERROR_ON(buffer.get() == nullptr);
    ARM_COMPUTE_RETURN_ERROR_ON(buffer.getInfo<CL_MEM_SIZE>() < info().total_size());
    ARM_COMPUTE_RETURN_ERROR_ON(buffer.getInfo<CL_MEM_CONTEXT>().get() != CLScheduler::get().context().get());
    ARM_COMPUTE_RETURN_ERROR_ON(_associated_memory_group != nullptr);
 
    _memory.set_owned_region(std::make_unique<CLBufferMemoryRegion>(buffer));
 
    info().set_is_resizable(false);
    return Status{};
}

References ARM_COMPUTE_RETURN_ERROR_ON, CLScheduler::context(), CLScheduler::get(), ITensorAllocator::info(), TensorInfo::set_is_resizable(), and CLMemory::set_owned_region().

Referenced by CLAuxTensorHandler::CLAuxTensorHandler(), CLFFTConvolutionLayer::run(), and arm_compute::test::validation::TEST_CASE().

map()

uint8_t * map	(	cl::CommandQueue &	q,
		bool	blocking
	)

Enqueue a map operation of the allocated buffer on the given queue.

Parameters

[in,out]	q	The CL command queue to use for the mapping operation.
[in]	blocking	If true, then the mapping will be ready to use by the time this method returns, else it is the caller's responsibility to flush the queue and wait for the mapping operation to have completed before using the returned mapping pointer.

Returns

The mapping address.

Definition at line 224 of file CLTensorAllocator.cpp.

{
    ARM_COMPUTE_ERROR_ON(_mapping != nullptr);
    ARM_COMPUTE_ERROR_ON(_memory.region() == nullptr);
    ARM_COMPUTE_ERROR_ON(_memory.region()->buffer() != nullptr);
 
    _mapping = reinterpret_cast<uint8_t *>(_memory.cl_region()->map(q, blocking));
    return _mapping;
}

References ARM_COMPUTE_ERROR_ON, IMemoryRegion::buffer(), CLMemory::cl_region(), ICLMemoryRegion::map(), and CLMemory::region().

operator=() [1/2]

CLTensorAllocator& operator= ( CLTensorAllocator &&  )

default

Allow instances of this class to be moved.

operator=() [2/2]

CLTensorAllocator& operator= ( const CLTensorAllocator &  )

delete

Prevent instances of this class from being copy assigned (As this class contains pointers)

quantization()

CLQuantization quantization

(

)

const

Wrapped quantization info data accessor.

Returns

A wrapped quantization info object.

Definition at line 116 of file CLTensorAllocator.cpp.

{
    return {&_scale, &_offset};
}

Referenced by CLTensor::quantization().

set_associated_memory_group()

void set_associated_memory_group

(

IMemoryGroup *

associated_memory_group

)

Associates the tensor with a memory group.

Parameters

[in]

associated_memory_group

Memory group to associate the tensor with

Definition at line 184 of file CLTensorAllocator.cpp.

{
    ARM_COMPUTE_ERROR_ON(associated_memory_group == nullptr);
    ARM_COMPUTE_ERROR_ON(_associated_memory_group != nullptr && _associated_memory_group != associated_memory_group);
    ARM_COMPUTE_ERROR_ON(_memory.region() != nullptr && _memory.cl_region()->cl_data().get() != nullptr);
 
    _associated_memory_group = associated_memory_group;
}

References ARM_COMPUTE_ERROR_ON, ICLMemoryRegion::cl_data(), CLMemory::cl_region(), and CLMemory::region().

Referenced by CLTensor::associate_memory_group(), and arm_compute::test::validation::TEST_CASE().

set_global_allocator()

void set_global_allocator ( IAllocator *  allocator )

static

Sets global allocator that will be used by all CLTensor objects.

Parameters

[in]

allocator

Allocator to be used as a global allocator

Definition at line 193 of file CLTensorAllocator.cpp.

{
    static_global_cl_allocator = allocator;
}

References arm_compute::test::validation::allocator().

Referenced by arm_compute::test::validation::TEST_CASE().

unmap()

void unmap	(	cl::CommandQueue &	q,
		uint8_t *	mapping
	)

Enqueue an unmap operation of the allocated buffer on the given queue.

Note

This method simply enqueue the unmap operation, it is the caller's responsibility to flush the queue and make sure the unmap is finished before the memory is accessed by the device.

Parameters

[in,out]	q	The CL command queue to use for the mapping operation.
[in]	mapping	The cpu mapping to unmap.

Definition at line 234 of file CLTensorAllocator.cpp.

{
    ARM_COMPUTE_ERROR_ON(_mapping == nullptr);
    ARM_COMPUTE_ERROR_ON(_mapping != mapping);
    ARM_COMPUTE_ERROR_ON(_memory.region() == nullptr);
    ARM_COMPUTE_ERROR_ON(_memory.region()->buffer() == nullptr);
    ARM_COMPUTE_UNUSED(mapping);
 
    _memory.cl_region()->unmap(q);
    _mapping = nullptr;
}

References ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_UNUSED, IMemoryRegion::buffer(), CLMemory::cl_region(), CLMemory::region(), and ICLMemoryRegion::unmap().

---

The documentation for this class was generated from the following files:

• arm_compute/runtime/CL/CLTensorAllocator.h
• src/runtime/CL/CLTensorAllocator.cpp