Compute Library
 21.11
CLGenerateProposalsLayer.h
Go to the documentation of this file.
1 /*
2  * Copyright (c) 2019-2021 Arm Limited.
3  *
4  * SPDX-License-Identifier: MIT
5  *
6  * Permission is hereby granted, free of charge, to any person obtaining a copy
7  * of this software and associated documentation files (the "Software"), to
8  * deal in the Software without restriction, including without limitation the
9  * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
10  * sell copies of the Software, and to permit persons to whom the Software is
11  * furnished to do so, subject to the following conditions:
12  *
13  * The above copyright notice and this permission notice shall be included in all
14  * copies or substantial portions of the Software.
15  *
16  * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17  * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18  * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19  * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
22  * SOFTWARE.
23  */
24 #ifndef ARM_COMPUTE_CLGENERATEPROPOSALSLAYER_H
25 #define ARM_COMPUTE_CLGENERATEPROPOSALSLAYER_H
26 
27 #include "arm_compute/core/Types.h"
36 
37 #include <memory>
38 
39 namespace arm_compute
40 {
41 class CLCompileContext;
42 class CLBoundingBoxTransformKernel;
43 class CLDequantizationLayer;
44 class CLComputeAllAnchorsKernel;
45 class CLPadLayerKernel;
46 class CLQuantizationLayer;
47 class ICLTensor;
48 class ITensorInfo;
49 
50 /** Basic function to generate proposals for a RPN (Region Proposal Network)
51  *
52  * This function calls the following OpenCL kernels:
53  * -# @ref CLComputeAllAnchorsKernel
54  * -# @ref CLPermute x 2
55  * -# @ref CLReshapeLayer x 2
56  * -# @ref CLBoundingBoxTransform
57  * -# @ref CLPadLayerKernel
58  * -# @ref CLDequantizationLayer x 2
59  * -# @ref CLQuantizationLayer
60  * And the following CPP functions:
61  * -# @ref CPPBoxWithNonMaximaSuppressionLimit
62  */
64 {
65 public:
66  /** Default constructor
67  *
68  * @param[in] memory_manager (Optional) Memory manager.
69  */
70  CLGenerateProposalsLayer(std::shared_ptr<IMemoryManager> memory_manager = nullptr);
71  /** Prevent instances of this class from being copied (As this class contains pointers) */
73  /** Prevent instances of this class from being copied (As this class contains pointers) */
75  /** Default destructor */
77 
78  /** Set the input and output tensors.
79  *
80  * Valid data layouts:
81  * - All
82  *
83  * Valid data type configurations:
84  * |src0 |src1 |src2 |dst |
85  * |:--------------|:------------------|:--------|:--------------|
86  * |F16 |F16 |F16 |F16 |
87  * |F32 |F32 |F32 |F32 |
88  * |QASYMM8 |QSYMM8 |QSYMM16 |QASYMM8 |
89  *
90  * @param[in] scores Scores from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors.
91  * Data types supported: QASYMM8/F16/F32
92  * @param[in] deltas Bounding box deltas from convolution layer of size (W, H, 4*A). Data types supported: Same as @p scores
93  * @param[in] anchors Anchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if @p scores is QASYMM8, otherwise same as @p scores
94  * @param[out] proposals Box proposals output tensor of size (5, W*H*A).
95  * Data types supported: QASYMM16 with scale of 0.125 and 0 offset if @p scores is QASYMM8, otherwise same as @p scores
96  * @param[out] scores_out Box scores output tensor of size (W*H*A). Data types supported: Same as @p scores
97  * @param[out] num_valid_proposals Scalar output tensor which says which of the first proposals are valid. Data types supported: U32
98  * @param[in] info Contains GenerateProposals operation information described in @ref GenerateProposalsInfo
99  *
100  * @note Only single image prediction is supported. Height and Width (and scale) of the image will be contained in the @ref GenerateProposalsInfo struct.
101  * @note Proposals contains all the proposals. Of those, only the first num_valid_proposals are valid.
102  */
103  void configure(const ICLTensor *scores, const ICLTensor *deltas, const ICLTensor *anchors, ICLTensor *proposals, ICLTensor *scores_out, ICLTensor *num_valid_proposals,
104  const GenerateProposalsInfo &info);
105  /** Set the input and output tensors.
106  *
107  * @param[in] compile_context The compile context to be used.
108  * @param[in] scores Scores from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors.
109  * Data types supported: QASYMM8/F16/F32
110  * @param[in] deltas Bounding box deltas from convolution layer of size (W, H, 4*A). Data types supported: Same as @p scores
111  * @param[in] anchors Anchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if @p scores is QASYMM8, otherwise same as @p scores
112  * @param[out] proposals Box proposals output tensor of size (5, W*H*A).
113  * Data types supported: QASYMM16 with scale of 0.125 and 0 offset if @p scores is QASYMM8, otherwise same as @p scores
114  * @param[out] scores_out Box scores output tensor of size (W*H*A). Data types supported: Same as @p scores
115  * @param[out] num_valid_proposals Scalar output tensor which says which of the first proposals are valid. Data types supported: U32
116  * @param[in] info Contains GenerateProposals operation information described in @ref GenerateProposalsInfo
117  *
118  * @note Only single image prediction is supported. Height and Width (and scale) of the image will be contained in the @ref GenerateProposalsInfo struct.
119  * @note Proposals contains all the proposals. Of those, only the first num_valid_proposals are valid.
120  */
121  void configure(const CLCompileContext &compile_context, const ICLTensor *scores, const ICLTensor *deltas, const ICLTensor *anchors, ICLTensor *proposals, ICLTensor *scores_out,
122  ICLTensor *num_valid_proposals, const GenerateProposalsInfo &info);
123 
124  /** Static function to check if given info will lead to a valid configuration of @ref CLGenerateProposalsLayer
125  *
126  * @param[in] scores Scores info from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors.
127  * Data types supported: QASYMM8/F16/F32
128  * @param[in] deltas Bounding box deltas info from convolution layer of size (W, H, 4*A). Data types supported: Same as @p scores
129  * @param[in] anchors Anchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if @p scores is QASYMM8, otherwise same as @p scores
130  * @param[in] proposals Box proposals info output tensor of size (5, W*H*A).
131  * Data types supported: QASYMM16 with scale of 0.125 and 0 offset if @p scores is QASYMM8, otherwise same as @p scores
132  * @param[in] scores_out Box scores output tensor info of size (W*H*A). Data types supported: Same as @p scores
133  * @param[in] num_valid_proposals Scalar output tensor info which says which of the first proposals are valid. Data types supported: U32
134  * @param[in] info Contains GenerateProposals operation information described in @ref GenerateProposalsInfo
135  *
136  * @return a Status
137  */
138  static Status validate(const ITensorInfo *scores, const ITensorInfo *deltas, const ITensorInfo *anchors, const ITensorInfo *proposals, const ITensorInfo *scores_out,
139  const ITensorInfo *num_valid_proposals,
140  const GenerateProposalsInfo &info);
141 
142  // Inherited methods overridden:
143  void run() override;
144 
145 private:
146  // Memory group manager
147  MemoryGroup _memory_group;
148 
149  // OpenCL kernels
150  CLPermute _permute_deltas;
151  CLReshapeLayer _flatten_deltas;
152  CLPermute _permute_scores;
153  CLReshapeLayer _flatten_scores;
154  std::unique_ptr<CLComputeAllAnchorsKernel> _compute_anchors_kernel;
155  std::unique_ptr<CLBoundingBoxTransformKernel> _bounding_box_kernel;
156  std::unique_ptr<CLPadLayerKernel> _pad_kernel;
157  std::unique_ptr<CLDequantizationLayer> _dequantize_anchors;
158  std::unique_ptr<CLDequantizationLayer> _dequantize_deltas;
159  std::unique_ptr<CLQuantizationLayer> _quantize_all_proposals;
160 
161  // CPP functions
163 
164  bool _is_nhwc;
165  bool _is_qasymm8;
166 
167  // Temporary tensors
168  CLTensor _deltas_permuted;
169  CLTensor _deltas_flattened;
170  CLTensor _deltas_flattened_f32;
171  CLTensor _scores_permuted;
172  CLTensor _scores_flattened;
173  CLTensor _all_anchors;
174  CLTensor _all_anchors_f32;
175  CLTensor _all_proposals;
176  CLTensor _all_proposals_quantized;
177  CLTensor _keeps_nms_unused;
178  CLTensor _classes_nms_unused;
179  CLTensor _proposals_4_roi_values;
180 
181  // Temporary tensor pointers
182  CLTensor *_all_proposals_to_use;
183 
184  // Output tensor pointers
185  ICLTensor *_num_valid_proposals;
186  ICLTensor *_scores_out;
187 
188  /** Internal function to run the CPP BoxWithNMS kernel */
189  void run_cpp_nms_kernel();
190 };
191 } // namespace arm_compute
192 #endif /* ARM_COMPUTE_CLGENERATEPROPOSALSLAYER_H */
void configure(const ICLTensor *scores, const ICLTensor *deltas, const ICLTensor *anchors, ICLTensor *proposals, ICLTensor *scores_out, ICLTensor *num_valid_proposals, const GenerateProposalsInfo &info)
Set the input and output tensors.
Generate Proposals Information class.
Definition: Types.h:1311
Base class for all functions.
Definition: IFunction.h:30
void run() override
Run the kernels contained in the function.
Store the tensor&#39;s metadata.
Definition: ITensorInfo.h:40
Status class.
Definition: Error.h:52
Copyright (c) 2017-2021 Arm Limited.
Interface to enqueue OpenCL kernels and get/set the OpenCL CommandQueue and ICLTuner.
CLGenerateProposalsLayer & operator=(const CLGenerateProposalsLayer &)=delete
Prevent instances of this class from being copied (As this class contains pointers) ...
Basic function to execute an opencl::kernels::ClPermuteKernel.
Definition: CLPermute.h:39
CLCompileContext class.
Basic pool of threads to execute CPP/Neon code on several cores in parallel.
static Status validate(const ITensorInfo *scores, const ITensorInfo *deltas, const ITensorInfo *anchors, const ITensorInfo *proposals, const ITensorInfo *scores_out, const ITensorInfo *num_valid_proposals, const GenerateProposalsInfo &info)
Static function to check if given info will lead to a valid configuration of CLGenerateProposalsLayer...
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
Basic function to run CPPBoxWithNonMaximaSuppressionLimitKernel.
Interface for OpenCL tensor.
Definition: ICLTensor.h:42
CLGenerateProposalsLayer(std::shared_ptr< IMemoryManager > memory_manager=nullptr)
Default constructor.
Basic function to run opencl::kernels::ClReshapeKernel.
Basic function to generate proposals for a RPN (Region Proposal Network)
~CLGenerateProposalsLayer()
Default destructor.
Basic implementation of the OpenCL tensor interface.
Definition: CLTensor.h:41