Compute Library
 21.02
NEGenerateProposalsLayer.h
Go to the documentation of this file.
1 /*
2  * Copyright (c) 2019-2021 Arm Limited.
3  *
4  * SPDX-License-Identifier: MIT
5  *
6  * Permission is hereby granted, free of charge, to any person obtaining a copy
7  * of this software and associated documentation files (the "Software"), to
8  * deal in the Software without restriction, including without limitation the
9  * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
10  * sell copies of the Software, and to permit persons to whom the Software is
11  * furnished to do so, subject to the following conditions:
12  *
13  * The above copyright notice and this permission notice shall be included in all
14  * copies or substantial portions of the Software.
15  *
16  * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17  * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18  * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19  * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
22  * SOFTWARE.
23  */
24 #ifndef ARM_COMPUTE_NEGENERATEPROPOSALSLAYER_H
25 #define ARM_COMPUTE_NEGENERATEPROPOSALSLAYER_H
26 
27 #include "arm_compute/core/Types.h"
39 
40 namespace arm_compute
41 {
42 class ITensor;
43 class NEComputeAllAnchorsKernel;
44 
45 /** Basic function to generate proposals for a RPN (Region Proposal Network)
46  *
47  * This function calls the following Neon kernels:
48  * -# @ref NEComputeAllAnchorsKernel
49  * -# @ref NEPermute x 2
50  * -# @ref NEReshapeLayer x 2
51  * -# @ref NEBoundingBoxTransform
52  * -# @ref NEPadLayerKernel
53  * -# @ref NEDequantizationLayerKernel x 2
54  * -# @ref NEQuantizationLayerKernel
55  * And the following CPP kernels:
56  * -# @ref CPPBoxWithNonMaximaSuppressionLimit
57  */
59 {
60 public:
61  /** Default constructor
62  *
63  * @param[in] memory_manager (Optional) Memory manager.
64  */
65  NEGenerateProposalsLayer(std::shared_ptr<IMemoryManager> memory_manager = nullptr);
66  /** Prevent instances of this class from being copied (As this class contains pointers) */
68  /** Prevent instances of this class from being copied (As this class contains pointers) */
70  /** Default destructor */
72 
73  /** Set the input and output tensors.
74  *
75  * @param[in] scores Scores from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors.
76  * Data types supported: QASYMM8/F16/F32
77  * @param[in] deltas Bounding box deltas from convolution layer of size (W, H, 4*A). Data types supported: Same as @p scores
78  * @param[in] anchors Anchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if @p scores is QASYMM8, otherwise same as @p scores
79  * @param[out] proposals Box proposals output tensor of size (5, W*H*A).
80  * Data types supported: QASYMM16 with scale of 0.125 and 0 offset if @p scores is QASYMM8, otherwise same as @p scores
81  * @param[out] scores_out Box scores output tensor of size (W*H*A). Data types supported: Same as @p input
82  * @param[out] num_valid_proposals Scalar output tensor which says which of the first proposals are valid. Data types supported: U32
83  * @param[in] info Contains GenerateProposals operation information described in @ref GenerateProposalsInfo
84  *
85  * @note Only single image prediction is supported. Height and Width (and scale) of the image will be contained in the @ref GenerateProposalsInfo struct.
86  * @note Proposals contains all the proposals. Of those, only the first num_valid_proposals are valid.
87  */
88  void configure(const ITensor *scores, const ITensor *deltas, const ITensor *anchors, ITensor *proposals, ITensor *scores_out, ITensor *num_valid_proposals,
90 
91  /** Static function to check if given info will lead to a valid configuration of @ref NEGenerateProposalsLayer
92  *
93  * @param[in] scores Scores info from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors.
94  * Data types supported: QASYMM8/F16/F32
95  * @param[in] deltas Bounding box deltas info from convolution layer of size (W, H, 4*A). Data types supported: Same as @p scores
96  * @param[in] anchors Anchors tensor info of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if @p scores is QASYMM8, otherwise same as @p scores
97  * @param[in] proposals Box proposals info output tensor of size (5, W*H*A).
98  * Data types supported: QASYMM16 with scale of 0.125 and 0 offset if @p scores is QASYMM8, otherwise same as @p scores
99  * @param[in] scores_out Box scores output tensor info of size (W*H*A). Data types supported: Same as @p input
100  * @param[in] num_valid_proposals Scalar output tensor info which says which of the first proposals are valid. Data types supported: U32
101  * @param[in] info Contains GenerateProposals operation information described in @ref GenerateProposalsInfo
102  *
103  * @return a Status
104  */
105  static Status validate(const ITensorInfo *scores, const ITensorInfo *deltas, const ITensorInfo *anchors, const ITensorInfo *proposals, const ITensorInfo *scores_out,
106  const ITensorInfo *num_valid_proposals,
107  const GenerateProposalsInfo &info);
108 
109  // Inherited methods overridden:
110  void run() override;
111 
112 private:
113  // Memory group manager
114  MemoryGroup _memory_group;
115 
116  // Neon kernels
117  NEPermute _permute_deltas;
118  NEReshapeLayer _flatten_deltas;
119  NEPermute _permute_scores;
120  NEReshapeLayer _flatten_scores;
121  std::unique_ptr<NEComputeAllAnchorsKernel> _compute_anchors;
122  NEBoundingBoxTransform _bounding_box;
123  NEPadLayer _pad;
124  NEDequantizationLayer _dequantize_anchors;
125  NEDequantizationLayer _dequantize_deltas;
126  NEQuantizationLayer _quantize_all_proposals;
127 
128  // CPP functions
130 
131  bool _is_nhwc;
132  bool _is_qasymm8;
133 
134  // Temporary tensors
135  Tensor _deltas_permuted;
136  Tensor _deltas_flattened;
137  Tensor _deltas_flattened_f32;
138  Tensor _scores_permuted;
139  Tensor _scores_flattened;
140  Tensor _all_anchors;
141  Tensor _all_anchors_f32;
142  Tensor _all_proposals;
143  Tensor _all_proposals_quantized;
144  Tensor _keeps_nms_unused;
145  Tensor _classes_nms_unused;
146  Tensor _proposals_4_roi_values;
147 
148  // Temporary tensor pointers
149  Tensor *_all_proposals_to_use;
150 
151  // Output tensor pointers
152  ITensor *_num_valid_proposals;
153  ITensor *_scores_out;
154 };
155 } // namespace arm_compute
156 #endif /* ARM_COMPUTE_NEGENERATEPROPOSALSLAYER_H */
Generate Proposals Information class.
Definition: Types.h:1352
~NEGenerateProposalsLayer()
Default destructor.
Base class for all functions.
Definition: IFunction.h:30
void run() override
Run the kernels contained in the function.
Basic function to generate proposals for a RPN (Region Proposal Network)
Store the tensor&#39;s metadata.
Definition: ITensorInfo.h:40
Basic function to simulate a quantization layer.
Status class.
Definition: Error.h:52
Basic function to run cpu::kernels::CpuPermuteKernel.
Definition: NEPermute.h:40
Interface for Neon tensor.
Definition: ITensor.h:36
NEGenerateProposalsLayer & operator=(const NEGenerateProposalsLayer &)=delete
Prevent instances of this class from being copied (As this class contains pointers) ...
static Status validate(const ITensorInfo *scores, const ITensorInfo *deltas, const ITensorInfo *anchors, const ITensorInfo *proposals, const ITensorInfo *scores_out, const ITensorInfo *num_valid_proposals, const GenerateProposalsInfo &info)
Static function to check if given info will lead to a valid configuration of NEGenerateProposalsLayer...
Copyright (c) 2017-2021 Arm Limited.
void configure(const ITensor *scores, const ITensor *deltas, const ITensor *anchors, ITensor *proposals, ITensor *scores_out, ITensor *num_valid_proposals, const GenerateProposalsInfo &info)
Set the input and output tensors.
Basic implementation of the tensor interface.
Definition: Tensor.h:37
Basic function to run NEDequantizationLayerKernel that dequantizes an input tensor.
Basic function to run NEBoundingBoxTransformKernel.
NEGenerateProposalsLayer(std::shared_ptr< IMemoryManager > memory_manager=nullptr)
Default constructor.
Basic pool of threads to execute CPP/Neon code on several cores in parallel.
Basic function to run cpu::kernels::CpuReshapeKernel.
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
Basic function to run CPPBoxWithNonMaximaSuppressionLimitKernel.
Basic function to pad a tensor.
Definition: NEPadLayer.h:51