Compute Library
 22.11
CLConvolutionLayer.h
Go to the documentation of this file.
1 /*
2  * Copyright (c) 2017-2021 Arm Limited.
3  *
4  * SPDX-License-Identifier: MIT
5  *
6  * Permission is hereby granted, free of charge, to any person obtaining a copy
7  * of this software and associated documentation files (the "Software"), to
8  * deal in the Software without restriction, including without limitation the
9  * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
10  * sell copies of the Software, and to permit persons to whom the Software is
11  * furnished to do so, subject to the following conditions:
12  *
13  * The above copyright notice and this permission notice shall be included in all
14  * copies or substantial portions of the Software.
15  *
16  * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17  * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18  * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19  * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
22  * SOFTWARE.
23  */
24 #ifndef ARM_COMPUTE_CLCONVOLUTIONLAYER_H
25 #define ARM_COMPUTE_CLCONVOLUTIONLAYER_H
26 
28 #include "arm_compute/core/Types.h"
33 
34 #include <memory>
35 
36 namespace arm_compute
37 {
38 class CLCompileContext;
39 class ICLTensor;
40 class ITensorInfo;
41 
42 /** Basic function to compute the convolution layer. This function calls the following OpenCL kernels/functions:
43  *
44  * -# @ref opencl::ClGemmConv2d
45  * -# @ref opencl::ClWinogradConv2d
46  * -# @ref opencl::ClDirectConv2d
47  * -# @ref CLFFTConvolutionLayer
48  *
49  * The function selects one of the algorithms mentioned above based on:
50  * - The size of the kernel
51  * - Number of input/output feature maps
52  * - Amount of memory needed
53  *
54  * Generally GEMM-based convolution is executed when neither Winograd nor FFT nor Direct convolution can be performed.
55  *
56  * FP32 Algorithm| Filter Size | Input/Output feature maps |
57  * --------------|-------------------------------------------------------------|-------------------------------------------|
58  * Winograd | 3x3 1x3 3x1 5x1 1x5 5x5(fast maths) 7x1 1x7 | Input channels is greater than 3 |
59  * FFT | Squared kernels and greater than 9x9 | Input feature maps > Output feature maps |
60  * DirectConv | 9x9 | |
61  * GEMM | Any size | |
62  *
63  * Winograd 5x5 requires fast maths enabled.
64  *
65  * FP16 Algorithm| Filter Size | Input/Output feature maps |
66  * --------------|----------------------------|-------------------------------------------|
67  * Winograd | 3x3 1x3 3x1 5x1 1x5 5x5 | Input channels is greater than 3 |
68  * FFT | Not supported | |
69  * DirectConv | 9x9 | |
70  * GEMM | Any size | |
71  *
72  * Winograd FP16 requires fast maths enabled.
73  *
74  */
76 {
77 public:
78  /** Default constructor */
79  CLConvolutionLayer(std::shared_ptr<IMemoryManager> memory_manager = nullptr);
80  /** Default Destructor */
82  /** Prevent instances of this class from being copied (As this class contains pointers) */
83  CLConvolutionLayer(const CLConvolutionLayer &) = delete;
84  /** Default move constructor */
86  /** Prevent instances of this class from being copied (As this class contains pointers) */
88  /** Default move assignment operator */
90  /** Set the input and output tensors.
91  *
92  * Valid data layouts:
93  * - NHWC
94  * - NCHW
95  *
96  * Valid data type configurations:
97  * |src0 |src1 |src2 |dst |
98  * |:--------------|:------------------|:------|:--------------|
99  * |F16 |F16 |F16 |F16 |
100  * |F32 |F32 |F32 |F32 |
101  * |QASYMM8 |QASYMM8 |S32 |QASYMM8 |
102  * |QASYMM8 |QSYMM8_PER_CHANNEL |S32 |QASYMM8 |
103  * |QASYMM8_SIGNED |QASYMM8_SIGNED |S32 |QASYMM8_SIGNED |
104  * |QASYMM8_SIGNED |QSYMM8_PER_CHANNEL |S32 |QASYMM8_SIGNED |
105  *
106  * @param[in] input Source tensor. 3 lower dimensions represent a single input [width, height, IFM],
107  * while every optional dimension from 4 and above represent a batch of inputs.
108  * Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
109  * @param[in] weights Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM].
110  * Data type supported: Same as @p input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
111  * @param[in] biases Biases tensor. Shared biases supported. Biases are 1D tensor with dimensions [OFM].
112  * Data type supported: Same as @p input, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
113  * @param[out] output Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs.
114  * Data types supported: Same as @p input.
115  * @param[in] conv_info Contains padding and stride information described in @ref PadStrideInfo.
116  * @param[in] weights_info Specifies if the weights tensor has been reshaped with CLWeightsReshapeKernel. Data type supported: Same as @p input.
117  * @param[in] dilation (Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
118  * @param[in] act_info (Optional) Activation layer information in case of a fused activation.
119  * @param[in] enable_fast_math (Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation
120  * available which may introduce a drop of accuracy as well. Default is false
121  * @param[in] num_groups (Optional) Number of groups when performing a grouped convolution. num_groups != 1 is only supported for NCHW data layout
122  * @param[in] post_ops (Optional) A sequence of post operations that are performed after the main operation.
123  */
124  void configure(ICLTensor *input, const ICLTensor *weights, const ICLTensor *biases, ICLTensor *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info = WeightsInfo(),
125  const Size2D &dilation = Size2D(1U, 1U), const ActivationLayerInfo &act_info = ActivationLayerInfo(), bool enable_fast_math = false, unsigned int num_groups = 1,
127  /** Set the input and output tensors.
128  *
129  * @param[in] compile_context The compile context to be used.
130  * @param[in] input Source tensor. 3 lower dimensions represent a single input [width, height, IFM],
131  * while every optional dimension from 4 and above represent a batch of inputs.
132  * Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
133  * @param[in] weights Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM].
134  * Data type supported: Same as @p input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
135  * @param[in] biases Biases tensor. Shared biases supported. Biases are 1D tensor with dimensions [OFM].
136  * Data type supported: Same as @p input, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
137  * @param[out] output Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs.
138  * Data types supported: Same as @p input.
139  * @param[in] conv_info Contains padding and stride information described in @ref PadStrideInfo.
140  * @param[in] weights_info Specifies if the weights tensor has been reshaped with CLWeightsReshapeKernel. Data type supported: Same as @p input.
141  * @param[in] dilation (Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
142  * @param[in] act_info (Optional) Activation layer information in case of a fused activation.
143  * @param[in] enable_fast_math (Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation
144  * available which may introduce a drop of accuracy as well. Default is false
145  * @param[in] num_groups (Optional) Number of groups when performing a grouped convolution. num_groups != 1 is only supported for NCHW data layout
146  * @param[in] post_ops (Optional) A sequence of post operations that are performed after the main operation.
147  */
148  void configure(const CLCompileContext &compile_context, ICLTensor *input, const ICLTensor *weights, const ICLTensor *biases, ICLTensor *output, const PadStrideInfo &conv_info,
149  const WeightsInfo &weights_info = WeightsInfo(), const Size2D &dilation = Size2D(1U, 1U), const ActivationLayerInfo &act_info = ActivationLayerInfo(), bool enable_fast_math = false,
151  /** Static function to check if given info will lead to a valid configuration of @ref CLConvolutionLayer
152  *
153  * @param[in] input Source tensor. 3 lower dimensions represent a single input [width, height, IFM],
154  * while every optional dimension from 4 and above represent a batch of inputs.
155  * Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
156  * @param[in] weights Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM].
157  * Data type supported: Same as @p input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
158  * @param[in] biases Biases tensor. Shared biases supported. Biases are 1D tensor with dimensions [OFM].
159  * Data type supported: Same as @p input, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
160  * @param[in] output Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs.
161  * Data types supported: Same as @p input.
162  * @param[in] conv_info Contains padding and stride information described in @ref PadStrideInfo.
163  * @param[in] weights_info Specifies if the weights tensor has been reshaped with CLWeightsReshapeKernel.
164  * @param[in] dilation (Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
165  * @param[in] act_info (Optional) Activation layer information in case of a fused activation.
166  * @param[in] enable_fast_math (Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation
167  * available which may introduce a drop of accuracy as well. Default is false
168  * @param[in] num_groups (Optional) Number of groups when performing a grouped convolution. num_groups != 1 is only supported for NCHW data layout
169  * @param[in] post_ops (Optional) A sequence of post operations that are performed after the main operation.
170  *
171  * @return a status
172  */
173  static Status validate(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, const PadStrideInfo &conv_info,
174  const WeightsInfo &weights_info = WeightsInfo(), const Size2D &dilation = Size2D(1U, 1U), const ActivationLayerInfo &act_info = ActivationLayerInfo(), bool enable_fast_math = false,
176  /** Static function to check if given info will return the convolution called by @ref CLConvolutionLayer
177  *
178  * @param[in] input Source tensor. 3 lower dimensions represent a single input [width, height, IFM],
179  * while every optional dimension from 4 and above represent a batch of inputs.
180  * Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
181  * @param[in] weights Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM].
182  * Data type supported: Same as @p input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
183  * @param[in] output Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs.
184  * Data types supported: Same as @p input.
185  * @param[in] conv_info Contains padding and stride information described in @ref PadStrideInfo.
186  * @param[in] weights_info Specifies if the weights tensor has been reshaped with CLWeightsReshapeKernel.
187  * @param[in] act_info (Optional) Activation layer information in case of a fused activation.
188  * @param[in] gpu_target Specifies the @p GPUTarget.
189  * @param[in] dilation (Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
190  * @param[in] enable_fast_math (Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation
191  * available which may introduce a drop of accuracy as well. Default is false
192  *
193  * @return the Convolution Method Hint
194  */
195  static ConvolutionMethod get_convolution_method(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *output, const PadStrideInfo &conv_info,
196  const WeightsInfo &weights_info, const ActivationLayerInfo &act_info, const GPUTarget gpu_target, const Size2D &dilation = Size2D(1U, 1U), bool enable_fast_math = false);
197  // Inherited methods overridden:
198  void run() override;
199  void prepare() override;
200 
201 private:
202  struct Impl;
203  std::unique_ptr<Impl> _impl;
204 };
205 }
206 #endif /* ARM_COMPUTE_CLCONVOLUTIONLAYER_H */
experimental::PostOpList< ITensorInfo * > post_ops
void run() override
Run the kernels contained in the function.
Base class for all functions.
Definition: IFunction.h:30
static Status validate(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info=WeightsInfo(), const Size2D &dilation=Size2D(1U, 1U), const ActivationLayerInfo &act_info=ActivationLayerInfo(), bool enable_fast_math=false, unsigned int num_groups=1, const experimental::PostOpList< ITensorInfo *> &post_ops=experimental::PostOpList< ITensorInfo *> {})
Static function to check if given info will lead to a valid configuration of CLConvolutionLayer.
CLConvolutionLayer(std::shared_ptr< IMemoryManager > memory_manager=nullptr)
Default constructor.
~CLConvolutionLayer()
Default Destructor.
void prepare() override
Prepare the function for executing.
Store the tensor&#39;s metadata.
Definition: ITensorInfo.h:40
Status class.
Definition: Error.h:52
ConvolutionMethod
Available ConvolutionMethod.
Definition: Types.h:134
Activation Layer Information class.
Definition: Types.h:1639
Copyright (c) 2017-2022 Arm Limited.
Convolution Layer Weights Information class.
Definition: Types.h:2073
const unsigned int num_groups
Definition: Im2Col.cpp:153
CLConvolutionLayer & operator=(const CLConvolutionLayer &)=delete
Prevent instances of this class from being copied (As this class contains pointers) ...
Padding and stride information class.
Definition: Types.h:669
Basic function to compute the convolution layer.
CLCompileContext class.
Interface for OpenCL tensor.
Definition: ICLTensor.h:42
GPUTarget
Available GPU Targets.
Definition: GPUTarget.h:34
Class for specifying the size of an image or rectangle.
Definition: Size2D.h:34
static ConvolutionMethod get_convolution_method(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info, const ActivationLayerInfo &act_info, const GPUTarget gpu_target, const Size2D &dilation=Size2D(1U, 1U), bool enable_fast_math=false)
Static function to check if given info will return the convolution called by CLConvolutionLayer.
A sequence of PostOps that can be appended to the end of other operators.
Definition: IPostOp.h:119
void configure(ICLTensor *input, const ICLTensor *weights, const ICLTensor *biases, ICLTensor *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info=WeightsInfo(), const Size2D &dilation=Size2D(1U, 1U), const ActivationLayerInfo &act_info=ActivationLayerInfo(), bool enable_fast_math=false, unsigned int num_groups=1, const experimental::PostOpList< ICLTensor *> &post_ops=experimental::PostOpList< ICLTensor *> {})
Set the input and output tensors.