24.02.1
|
Tensor handler to wrap and handle tensor allocations on workspace buffers. More...
#include <CpuAuxTensorHandler.h>
Public Member Functions | |
CpuAuxTensorHandler (int slot_id, TensorInfo &info, ITensorPack &pack, bool pack_inject=false, bool bypass_alloc=false, bool bypass_import=false) | |
Create a temporary tensor handle, by either important an existing tensor from a tensor pack, or allocating a new one. More... | |
CpuAuxTensorHandler (TensorInfo &info, const ITensor &tensor, bool bypass_import=false) | |
Create a temporary handle to the original tensor with a new TensorInfo This is useful if we want to change a tensor's tensor info at run time without modifying the original tensor. More... | |
CpuAuxTensorHandler (const CpuAuxTensorHandler &)=delete | |
CpuAuxTensorHandler & | operator= (const CpuAuxTensorHandler)=delete |
~CpuAuxTensorHandler () | |
ITensor * | get () |
ITensor * | operator() () |
Tensor handler to wrap and handle tensor allocations on workspace buffers.
If not handled correctly, these two flags can lead to performance issues (not bypass when needed to), or memory bugs (bypass when should not to).
Make sure:
The aux tensor handlers must always be declared at the root level, or the same level as the run/prepare methods that potentially use them.
Once the handler is destroyed (e.g. when going out of scope), the memory it owns (returned by the get() method) will also be destroyed.
Thus it's important to ensure the handler is always in-scope when it is being used by a operator / kernel.
The handler's bypass_alloc and bypass_import flags should always be inverse of whether the handler is used in its surrounding scope by run/prepare. (This usually means being added to some tensor pack)
This ensures we only bypass if and only if the aux tensor is not used by the op / kernel later.
So the general usage pattern goes like this:
bool use_aux_tensor = some_condition_about_when_to_use_the_aux_tensor CpuAuxTensorHandler aux_handler {..., !use_aux_tensor || bypass_alloc / bypass_import ||}; if (use_aux_tensor) { tensor_pack.add_tensor(aux_handler.get()); } op.run(tensor_pack);
Definition at line 78 of file CpuAuxTensorHandler.h.
|
inline |
Create a temporary tensor handle, by either important an existing tensor from a tensor pack, or allocating a new one.
[in] | slot_id | Slot id of the tensor to be retrieved in the tensor pack If no such tensor exists in the tensor pack, a new tensor will be allocated. |
[in] | info | Tensor info containing requested size of the new tensor. If requested size is larger than the tensor retrieved from the tensor pack, a new tensor will be allocated. |
[in,out] | pack | Tensor pack to retrieve the old tensor. When pack_inject is true, the new tensor will also be added here. |
[in] | pack_inject | In case of a newly allocated tensor, whether to add this tensor back to the pack |
[in] | bypass_alloc | Bypass allocation in case of a new tensor This is to prevent unnecessary memory operations when the handler object is not used |
[in] | bypass_import | Bypass importation in case of a retrieved tensor This is to prevent unnecessary memory operations when the handler object is not used |
Definition at line 100 of file CpuAuxTensorHandler.h.
References ITensorPack::add_tensor(), TensorAllocator::allocate(), Tensor::allocator(), ARM_COMPUTE_LOG_INFO_WITH_FUNCNAME_ACL, ITensor::buffer(), ITensorPack::get_tensor(), TensorAllocator::import_memory(), ITensor::info(), arm_compute::test::validation::info, arm_compute::test::validation::pack, ITensorAllocator::soft_init(), and ITensorInfo::total_size().
|
inline |
Create a temporary handle to the original tensor with a new TensorInfo This is useful if we want to change a tensor's tensor info at run time without modifying the original tensor.
[in] | info | New tensor info to "assign" to tensor |
[in] | tensor | Tensor to be assigned a new TensorInfo |
[in] | bypass_import | Bypass importing tensor's memory into the handler. This is to prevent unnecessary memory operations when the handler object is not used |
Definition at line 147 of file CpuAuxTensorHandler.h.
References Tensor::allocator(), ARM_COMPUTE_ERROR_ON, TensorAllocator::import_memory(), arm_compute::test::validation::info, ITensorAllocator::soft_init(), and tensor.
|
delete |
|
inline |
|
inline |
Definition at line 171 of file CpuAuxTensorHandler.h.
Referenced by CpuGemmDirectConv2d::prepare(), CpuWinogradConv2d::prepare(), CpuFullyConnected::prepare(), CpuGemmLowpMatrixMultiplyCore::prepare(), CpuGemmConv2d::prepare(), CpuAddMulAdd::run(), CpuSoftmaxGeneric::run(), CpuMatMul::run(), CpuWinogradConv2d::run(), CpuFullyConnected::run(), CpuGemmLowpMatrixMultiplyCore::run(), CpuGemm::run(), and CpuGemmConv2d::run().
|
inline |
Definition at line 176 of file CpuAuxTensorHandler.h.
|
delete |