Compute Library
 21.11
NEGEMMLowpMatrixMultiplyCore Class Reference

Function to run Gemm on quantized types. More...

#include <NEGEMMLowpMatrixMultiplyCore.h>

Collaboration diagram for NEGEMMLowpMatrixMultiplyCore:
[legend]

Public Member Functions

 NEGEMMLowpMatrixMultiplyCore (std::shared_ptr< IMemoryManager > memory_manager=nullptr, IWeightsManager *weights_manager=nullptr)
 Constructor. More...
 
 NEGEMMLowpMatrixMultiplyCore (const NEGEMMLowpMatrixMultiplyCore &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 NEGEMMLowpMatrixMultiplyCore (NEGEMMLowpMatrixMultiplyCore &&)=default
 Default move constructor. More...
 
NEGEMMLowpMatrixMultiplyCoreoperator= (const NEGEMMLowpMatrixMultiplyCore &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NEGEMMLowpMatrixMultiplyCoreoperator= (NEGEMMLowpMatrixMultiplyCore &&)=default
 Default move assignment operator. More...
 
 ~NEGEMMLowpMatrixMultiplyCore ()
 Default destructor. More...
 
void configure (const ITensor *a, const ITensor *b, const ITensor *c, ITensor *output, const GEMMInfo &gemm_info=GEMMInfo())
 Initialise the kernel's inputs, output. More...
 
void run () override
 Run the kernels contained in the function. More...
 
void prepare () override
 Prepare the function for executing. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *a, const ITensorInfo *b, const ITensorInfo *c, const ITensorInfo *output, const GEMMInfo &gemm_info=GEMMInfo())
 Static function to check if given info will lead to a valid configuration of NEGEMMLowpMatrixMultiplyCore. More...
 

Detailed Description

Function to run Gemm on quantized types.

This function calls the following:

  1. cpu::CpuGemmLowpMatrixMultiplyCore

Definition at line 45 of file NEGEMMLowpMatrixMultiplyCore.h.

Constructor & Destructor Documentation

◆ NEGEMMLowpMatrixMultiplyCore() [1/3]

NEGEMMLowpMatrixMultiplyCore ( std::shared_ptr< IMemoryManager memory_manager = nullptr,
IWeightsManager weights_manager = nullptr 
)

Constructor.

Definition at line 53 of file NEGEMMLowpMatrixMultiplyCore.cpp.

References NEGEMMLowpMatrixMultiplyCore::~NEGEMMLowpMatrixMultiplyCore().

54  : _impl(std::make_unique<Impl>())
55 {
56  _impl->weights_manager = weights_manager;
57  _impl->memory_group = MemoryGroup(memory_manager);
58 }

◆ NEGEMMLowpMatrixMultiplyCore() [2/3]

Prevent instances of this class from being copied (As this class contains pointers)

◆ NEGEMMLowpMatrixMultiplyCore() [3/3]

Default move constructor.

◆ ~NEGEMMLowpMatrixMultiplyCore()

Member Function Documentation

◆ configure()

void configure ( const ITensor a,
const ITensor b,
const ITensor c,
ITensor output,
const GEMMInfo gemm_info = GEMMInfo() 
)

Initialise the kernel's inputs, output.

Valid data layouts:

  • NHWC
  • NCHW

Valid data type configurations:

src0 src1 src2 dst
QASYMM8 QASYMM8 S32 QASYMM8
QASYMM8 QSYMM8_PER_CHANNEL S32 QASYMM8
QASYMM8 QSYMM8 S32 QASYMM8
QASYMM8 QASYMM8 S32 S32
QASYMM8 QSYMM8_PER_CHANNEL S32 S32
QASYMM8 QSYMM8 S32 S32
QASYMM8_SIGNED QASYMM8_SIGNED S32 QASYMM8_SIGNED
QASYMM8_SIGNED QSYMM8_PER_CHANNEL S32 QASYMM8_SIGNED
QASYMM8_SIGNED QSYMM8 S32 QASYMM8_SIGNED
QASYMM8_SIGNED QASYMM8_SIGNED S32 S32
QASYMM8_SIGNED QSYMM8_PER_CHANNEL S32 S32
QASYMM8_SIGNED QSYMM8 S32 S32
Note
GEMM_LOWP: low precision GEMM kernel This kernel performs the following computations:
  1. Convert a values from QASYMM8 to int32 and add a_offset to each of them.
  2. Convert b values from QASYMM8 to int32 add b_offset to each of them.
  3. Compute the matrix product of the resulting a * b in int32.
Note
The output type is S32 if gemm_info.type == GEMMLowpOutputStageType::NONE. It is QASYMM8/QASYMM8_SIGNED otherwise
Parameters
[in]aFirst input tensor (Matrix A). Data type supported: QASYMM8/QASYMM8_SIGNED.
[in]bSecond input tensor (Matrix B). Data type supported: QASYMM8/QASYMM8_SIGNED/QSYMM8/QSYMM8_PER_CHANNEL.
[in]cThird input tensor (Matrix C). It can be a nullptr. Data type supported: S32
[out]outputOutput tensor. Data type supported: Data type supported: S32/QASYMM8/QASYMM8_SIGNED
[in]gemm_info(Optional) Specifies if the matrix A and/or matrix B have been reshaped and if the reshape of matrix B should be executed only for the first run

Definition at line 61 of file NEGEMMLowpMatrixMultiplyCore.cpp.

References arm_compute::ACL_DST, arm_compute::ACL_SRC_0, arm_compute::ACL_SRC_1, arm_compute::ACL_SRC_2, ARM_COMPUTE_ERROR_ON_NULLPTR, arm_compute::test::validation::b, arm_compute::test::validation::gemm_info, and ITensor::info().

Referenced by NELSTMLayerQuantized::configure(), main(), and NEQLSTMLayer::NEQLSTMLayer().

62 {
63  ARM_COMPUTE_ERROR_ON_NULLPTR(a, b, output);
64  _impl->b = b;
65  _impl->op = std::make_unique<cpu::CpuGemmLowpMatrixMultiplyCore>();
66  _impl->op->configure(a->info(), b->info(), (c != nullptr ? c->info() : nullptr), output->info(), gemm_info);
67  _impl->run_pack =
68  {
69  { TensorType::ACL_SRC_0, a },
71  { TensorType::ACL_SRC_2, c },
72  { TensorType::ACL_DST, output }
73  };
74  _impl->prep_pack =
75  {
78  };
79  _impl->aux_mem_req = _impl->op->workspace();
80  _impl->workspace_tensors = manage_workspace<Tensor>(_impl->aux_mem_req, _impl->memory_group, _impl->run_pack, _impl->prep_pack);
81 }
SimpleTensor< float > b
Definition: DFT.cpp:157
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:157

◆ operator=() [1/2]

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

Default move assignment operator.

◆ prepare()

void prepare ( )
overridevirtual

Prepare the function for executing.

Any one off pre-processing step required by the function is handled here

Note
Prepare stage might not need all the function's buffers' backing memory to be available in order to execute

Reimplemented from IFunction.

Definition at line 95 of file NEGEMMLowpMatrixMultiplyCore.cpp.

References arm_compute::mlgo::parser::end(), and arm_compute::test::validation::m.

Referenced by NEGEMMLowpMatrixMultiplyCore::run().

96 {
97  if(!_impl->is_prepared)
98  {
99  _impl->op->prepare(_impl->prep_pack);
100 
101  auto has_reshape = std::find_if(_impl->aux_mem_req.begin(),
102  _impl->aux_mem_req.end(),
103  [](const MemoryInfo & m) -> bool { return m.lifetime == MemoryLifetime::Persistent; });
104 
105  if(has_reshape != std::end(_impl->aux_mem_req))
106  {
107  _impl->b->mark_as_unused();
108  }
109 
110  // Release temporary tensors that are only used in prepare stage
111  release_temporaries<Tensor>(_impl->aux_mem_req, _impl->workspace_tensors);
112  _impl->is_prepared = true;
113  }
114 }
void end(TokenStream &in, bool &valid)
Definition: MLGOParser.cpp:290

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For CPU kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 88 of file NEGEMMLowpMatrixMultiplyCore.cpp.

References NEGEMMLowpMatrixMultiplyCore::prepare().

Referenced by main(), NELSTMLayerQuantized::run(), and NEQLSTMLayer::run().

89 {
90  prepare();
91  MemoryGroupResourceScope scope_mg(_impl->memory_group);
92  _impl->op->run(_impl->run_pack);
93 }
void prepare() override
Prepare the function for executing.

◆ validate()

Status validate ( const ITensorInfo a,
const ITensorInfo b,
const ITensorInfo c,
const ITensorInfo output,
const GEMMInfo gemm_info = GEMMInfo() 
)
static

Static function to check if given info will lead to a valid configuration of NEGEMMLowpMatrixMultiplyCore.

Similar to NEGEMMLowpMatrixMultiplyCore::configure()

Returns
a status

Definition at line 83 of file NEGEMMLowpMatrixMultiplyCore.cpp.

References CpuGemmLowpMatrixMultiplyCore::validate().

Referenced by arm_compute::test::validation::DATA_TEST_CASE(), and NELSTMLayerQuantized::validate().

84 {
86 }
SimpleTensor< float > b
Definition: DFT.cpp:157
static Status validate(const ITensorInfo *a, const ITensorInfo *b, const ITensorInfo *c, const ITensorInfo *dst, const GEMMInfo &gemm_info=GEMMInfo())
Static function to check if given info will lead to a valid configuration.

The documentation for this class was generated from the following files: