Compute Library
 21.05
NEGEMMConv2d Class Reference

Basic function to compute the convolution layer. More...

#include <NEGEMMConv2d.h>

Collaboration diagram for NEGEMMConv2d:
[legend]

Public Member Functions

 NEGEMMConv2d (const std::shared_ptr< IMemoryManager > &memory_manager=nullptr)
 Constructor. More...
 
 NEGEMMConv2d (const NEGEMMConv2d &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 NEGEMMConv2d (NEGEMMConv2d &&)=default
 Default move constructor. More...
 
NEGEMMConv2doperator= (const NEGEMMConv2d &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NEGEMMConv2doperator= (NEGEMMConv2d &&)=default
 Default move assignment operator. More...
 
 ~NEGEMMConv2d ()
 Destructor. More...
 
void configure (ITensor *input, const ITensor *weights, const ITensor *biases, ITensor *output, const Conv2dInfo &info)
 Set the input and output tensors. More...
 
void run () override
 Run the kernels contained in the function. More...
 
void prepare () override
 Prepare the function for executing. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, const Conv2dInfo &info)
 Static function to check if given info will lead to a valid configuration of NEGEMMConv2d. More...
 

Detailed Description

Basic function to compute the convolution layer.

This function calls the following kernels/functions:

Supports only NHWC data layout

  1. NEGEMMAssemblyDispatch
  2. NEActivationLayer, in case activation cannot be fused in the assembly dispatch

Weights are transformed from OHWI to HWIO format using the following kernels:

  1. NEPermute

Definition at line 51 of file NEGEMMConv2d.h.

Constructor & Destructor Documentation

◆ NEGEMMConv2d() [1/3]

NEGEMMConv2d ( const std::shared_ptr< IMemoryManager > &  memory_manager = nullptr)

Constructor.

Definition at line 85 of file NEGEMMConv2d.cpp.

86  : _gemm_asm_func(std::make_unique<NEGEMMAssemblyDispatch>(memory_manager)), _activation_func(), _weights_permute_func(), _original_weights(nullptr), _permuted_weights(), _is_prepared(false),
87  _run_activation(false)
88 {
89 }

◆ NEGEMMConv2d() [2/3]

NEGEMMConv2d ( const NEGEMMConv2d )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ NEGEMMConv2d() [3/3]

NEGEMMConv2d ( NEGEMMConv2d &&  )
default

Default move constructor.

◆ ~NEGEMMConv2d()

~NEGEMMConv2d ( )
default

Destructor.

Member Function Documentation

◆ configure()

void configure ( ITensor input,
const ITensor weights,
const ITensor biases,
ITensor output,
const Conv2dInfo info 
)

Set the input and output tensors.

Valid data layouts:

  • All

Valid data type configurations:

src0 src1 src2 dst
QASYMM8 QASYMM8 S32 QASYMM8
QASYMM8_SIGNED QASYMM8_SIGNED S32 QASYMM8_SIGNED
F16 F16 F16 F16
F32 F32 F32 F32
BFLOAT16 BFLOAT16 BFLOAT16 BFLOAT16
Parameters
[in]inputSource tensor. 3 lower dimensions represent a single input [width, height, IFM], while every optional dimension from 4 and above represent a batch of inputs. Data types supported: QASYMM8/QASYMM8_SIGNED/BFLOAT16/F16/F32.
[in]weightsWeights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM]. Data type supported: QASYMM8/QASYMM8_SIGNED/QSYMM8_PER_CHANNEL/BFLOAT16/F16/F32.
[in]biasesBiases tensor. Shared biases supported. Biases are 1D tensor with dimensions [OFM]. Data type supported: Should match input data type, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
[out]outputDestination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs. Data types supported: Same as input.
[in]infoConvolution layer descriptor

Definition at line 93 of file NEGEMMConv2d.cpp.

94 {
95  ARM_COMPUTE_ERROR_ON_NULLPTR(input, weights, output);
97  weights->info(),
98  biases != nullptr ? biases->info() : nullptr,
99  output->info(),
100  info));
101  _original_weights = weights;
102  _weights_permute_func.configure(weights, &_permuted_weights, PermutationVector{ 3, 0, 1, 2 });
103 
104  // Configure assembly dispatch
105  AsmGemmInfo asm_info = init_assembly_metadata(info, false);
106  if(is_data_type_quantized(input->info()->data_type()))
107  {
108  asm_info.output_stage = calculate_output_stage_metadata(input->info(), weights->info(), output->info(), info.act_info);
109  }
110  _gemm_asm_func->configure(input, &_permuted_weights, biases, output, asm_info);
111 
112  // Configure activation
113  if(info.act_info.enabled() && !_gemm_asm_func->is_activation_supported(info.act_info))
114  {
115  _activation_func.configure(output, nullptr, info.act_info);
116  _run_activation = true;
117  }
118 }
bool is_data_type_quantized(DataType dt)
Check if a given data type is of quantized type.
Definition: Utils.h:967
Strides PermutationVector
Permutation vector.
Definition: Types.h:49
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:455
static Status validate(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, const Conv2dInfo &info)
Static function to check if given info will lead to a valid configuration of NEGEMMConv2d.
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
void configure(ITensor *input, ITensor *output, ActivationLayerInfo activation_info)
[NEActivationLayer snippet]
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:157
void configure(const ITensor *input, ITensor *output, const PermutationVector &perm)
Configure the permute function.
Definition: NEPermute.cpp:45

References ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, NEPermute::configure(), NEActivationLayer::configure(), ITensor::info(), arm_compute::test::validation::info, arm_compute::test::validation::input, arm_compute::is_data_type_quantized(), AsmGemmInfo::output_stage, and NEGEMMConv2d::validate().

◆ operator=() [1/2]

NEGEMMConv2d& operator= ( const NEGEMMConv2d )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

NEGEMMConv2d& operator= ( NEGEMMConv2d &&  )
default

Default move assignment operator.

◆ prepare()

void prepare ( )
overridevirtual

Prepare the function for executing.

Any one off pre-processing step required by the function is handled here

Note
Prepare stage might not need all the function's buffers' backing memory to be available in order to execute

Reimplemented from IFunction.

Definition at line 166 of file NEGEMMConv2d.cpp.

167 {
168  if(!_is_prepared)
169  {
170  _permuted_weights.allocator()->allocate();
171  _weights_permute_func.run();
172  _original_weights->mark_as_unused();
173  _is_prepared = true;
174  }
175 }
TensorAllocator * allocator()
Return a pointer to the tensor's allocator.
Definition: Tensor.cpp:48
void mark_as_unused() const
Marks a tensor as unused.
Definition: ITensor.cpp:168
void allocate() override
Allocate size specified by TensorInfo of CPU memory.
void run() override
Run the kernels contained in the function.
Definition: NEPermute.cpp:63

References TensorAllocator::allocate(), Tensor::allocator(), ITensor::mark_as_unused(), and NEPermute::run().

Referenced by NEGEMMConv2d::run().

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For CPU kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 156 of file NEGEMMConv2d.cpp.

157 {
158  prepare();
159 
160  _gemm_asm_func->run();
161  if(_run_activation)
162  {
163  _activation_func.run();
164  }
165 }
void run() override
Run the kernels contained in the function.
void prepare() override
Prepare the function for executing.

References NEGEMMConv2d::prepare(), and NEActivationLayer::run().

◆ validate()

Status validate ( const ITensorInfo input,
const ITensorInfo weights,
const ITensorInfo biases,
const ITensorInfo output,
const Conv2dInfo info 
)
static

Static function to check if given info will lead to a valid configuration of NEGEMMConv2d.

Parameters
[in]inputSource tensor info. 3 lower dimensions represent a single input [width, height, IFM], while every optional dimension from 4 and above represent a batch of inputs. Data types supported: QASYMM8/QASYMM8_SIGNED/BFLOAT16/F16/F32.
[in]weightsWeights tensor info. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM]. Data type supported: QASYMM8/QASYMM8_SIGNED/QSYMM8_PER_CHANNEL/BFLOAT16/F16/F32.
[in]biasesBiases tensor info. Shared biases supported. Biases are 1D tensor with dimensions [OFM]. Data type supported: Should match input data type, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
[in]outputDestination tensor info. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs. Data types supported: Same as input.
[in]infoContains padding and stride information described in PadStrideInfo.
Returns
a status

Definition at line 119 of file NEGEMMConv2d.cpp.

120 {
121  ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(input, weights, output);
125  ARM_COMPUTE_RETURN_ERROR_ON_MSG(info.num_groups > 1, "Grouping (num_groups != 1) is not supported on Neon");
126  ARM_COMPUTE_RETURN_ERROR_ON_MSG(input->data_layout() != DataLayout::NHWC, "Data layout supported is NHWC");
127  const DataType data_type = input->data_type();
128  const TensorShape i_shape = input->tensor_shape();
129  const TensorShape w_shape = weights->tensor_shape();
130  ARM_COMPUTE_RETURN_ERROR_ON(w_shape[0] != i_shape[0]);
131  ARM_COMPUTE_RETURN_ERROR_ON(info.dilation != Size2D(1U, 1U));
132  ARM_COMPUTE_RETURN_ERROR_ON(weights->num_dimensions() > 4);
133  // Validate biases
134  if(biases != nullptr)
135  {
137  {
139  }
140  else if(data_type == DataType::BFLOAT16)
141  {
143  }
144  else
145  {
147  }
148  ARM_COMPUTE_RETURN_ERROR_ON(biases->dimension(0) != weights->dimension(3));
149  ARM_COMPUTE_RETURN_ERROR_ON(biases->num_dimensions() > 1);
150  }
151 
152  AsmGemmInfo asm_info = init_assembly_metadata(info, false);
153  ARM_COMPUTE_RETURN_ON_ERROR(NEGEMMAssemblyDispatch::validate(input, weights, biases, output, asm_info));
154  return Status{};
155 }
#define ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_LAYOUT(...)
Definition: Validate.h:490
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:204
1 channel, 1 F32 per channel
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:296
1 channel, 1 F16 per channel
#define ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(...)
Definition: Validate.h:159
1 channel, 1 S32 per channel
16-bit brain floating-point number
const DataType data_type
Definition: Im2Col.cpp:150
quantized, asymmetric fixed-point 8-bit number unsigned
static Status validate(const ITensorInfo *a, const ITensorInfo *b, const ITensorInfo *c, const ITensorInfo *d, const AsmGemmInfo &info)
Indicates whether or not this function can be used to process the given parameters.
bool is_data_type_quantized_asymmetric(DataType dt)
Check if a given data type is of asymmetric quantized type.
Definition: Utils.h:989
quantized, symmetric per channel fixed-point 8-bit number
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
#define ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_TYPES(...)
Definition: Validate.h:541
Num samples, height, width, channels.
#define ARM_COMPUTE_RETURN_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN(t, c,...)
Definition: Validate.h:788
#define ARM_COMPUTE_RETURN_ERROR_ON_MSG(cond, msg)
If the condition is true, an error is returned.
Definition: Error.h:244
quantized, asymmetric fixed-point 8-bit number signed
DataType
Available data types.
Definition: Types.h:77

References ARM_COMPUTE_RETURN_ERROR_ON, ARM_COMPUTE_RETURN_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN, ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_LAYOUT, ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_TYPES, ARM_COMPUTE_RETURN_ERROR_ON_MSG, ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR, ARM_COMPUTE_RETURN_ON_ERROR, arm_compute::BFLOAT16, arm_compute::test::validation::data_type, ITensorInfo::dimension(), arm_compute::F16, arm_compute::F32, arm_compute::test::validation::info, arm_compute::test::validation::input, arm_compute::is_data_type_quantized_asymmetric(), arm_compute::NHWC, ITensorInfo::num_dimensions(), arm_compute::QASYMM8, arm_compute::QASYMM8_SIGNED, arm_compute::QSYMM8_PER_CHANNEL, arm_compute::S32, ITensorInfo::tensor_shape(), arm_compute::U, and NEGEMMAssemblyDispatch::validate().

Referenced by NEGEMMConv2d::configure(), NEConvolutionLayer::get_convolution_method(), and NEConvolutionLayer::validate().


The documentation for this class was generated from the following files: