Compute Library
 21.02
CLGenerateProposalsLayer Class Reference

Basic function to generate proposals for a RPN (Region Proposal Network) More...

#include <CLGenerateProposalsLayer.h>

Collaboration diagram for CLGenerateProposalsLayer:
[legend]

Public Member Functions

 CLGenerateProposalsLayer (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 
 CLGenerateProposalsLayer (const CLGenerateProposalsLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CLGenerateProposalsLayeroperator= (const CLGenerateProposalsLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 ~CLGenerateProposalsLayer ()
 Default destructor. More...
 
void configure (const ICLTensor *scores, const ICLTensor *deltas, const ICLTensor *anchors, ICLTensor *proposals, ICLTensor *scores_out, ICLTensor *num_valid_proposals, const GenerateProposalsInfo &info)
 Set the input and output tensors. More...
 
void configure (const CLCompileContext &compile_context, const ICLTensor *scores, const ICLTensor *deltas, const ICLTensor *anchors, ICLTensor *proposals, ICLTensor *scores_out, ICLTensor *num_valid_proposals, const GenerateProposalsInfo &info)
 Set the input and output tensors. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *scores, const ITensorInfo *deltas, const ITensorInfo *anchors, const ITensorInfo *proposals, const ITensorInfo *scores_out, const ITensorInfo *num_valid_proposals, const GenerateProposalsInfo &info)
 Static function to check if given info will lead to a valid configuration of CLGenerateProposalsLayer. More...
 

Detailed Description

Basic function to generate proposals for a RPN (Region Proposal Network)

This function calls the following OpenCL kernels:

  1. CLComputeAllAnchorsKernel
  2. CLPermute x 2
  3. CLReshapeLayer x 2
  4. CLBoundingBoxTransform
  5. CLPadLayerKernel
  6. CLDequantizationLayerKernel x 2
  7. CLQuantizationLayerKernel And the following CPP functions:
  8. CPPBoxWithNonMaximaSuppressionLimit

Definition at line 63 of file CLGenerateProposalsLayer.h.

Constructor & Destructor Documentation

◆ CLGenerateProposalsLayer() [1/2]

CLGenerateProposalsLayer ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Parameters
[in]memory_manager(Optional) Memory manager.

Definition at line 37 of file CLGenerateProposalsLayer.cpp.

References CLGenerateProposalsLayer::~CLGenerateProposalsLayer().

38  : _memory_group(memory_manager),
39  _permute_deltas(),
40  _flatten_deltas(),
41  _permute_scores(),
42  _flatten_scores(),
43  _compute_anchors_kernel(std::make_unique<CLComputeAllAnchorsKernel>()),
44  _bounding_box_kernel(std::make_unique<CLBoundingBoxTransformKernel>()),
45  _pad_kernel(std::make_unique<CLPadLayerKernel>()),
46  _dequantize_anchors(std::make_unique<CLDequantizationLayerKernel>()),
47  _dequantize_deltas(std::make_unique<CLDequantizationLayerKernel>()),
48  _quantize_all_proposals(std::make_unique<CLQuantizationLayerKernel>()),
49  _cpp_nms(memory_manager),
50  _is_nhwc(false),
51  _is_qasymm8(false),
52  _deltas_permuted(),
53  _deltas_flattened(),
54  _deltas_flattened_f32(),
55  _scores_permuted(),
56  _scores_flattened(),
57  _all_anchors(),
58  _all_anchors_f32(),
59  _all_proposals(),
60  _all_proposals_quantized(),
61  _keeps_nms_unused(),
62  _classes_nms_unused(),
63  _proposals_4_roi_values(),
64  _all_proposals_to_use(nullptr),
65  _num_valid_proposals(nullptr),
66  _scores_out(nullptr)
67 {
68 }

◆ CLGenerateProposalsLayer() [2/2]

Prevent instances of this class from being copied (As this class contains pointers)

◆ ~CLGenerateProposalsLayer()

Default destructor.

Referenced by CLGenerateProposalsLayer::CLGenerateProposalsLayer().

Member Function Documentation

◆ configure() [1/2]

void configure ( const ICLTensor scores,
const ICLTensor deltas,
const ICLTensor anchors,
ICLTensor proposals,
ICLTensor scores_out,
ICLTensor num_valid_proposals,
const GenerateProposalsInfo info 
)

Set the input and output tensors.

Parameters
[in]scoresScores from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors. Data types supported: QASYMM8/F16/F32
[in]deltasBounding box deltas from convolution layer of size (W, H, 4*A). Data types supported: Same as scores
[in]anchorsAnchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if scores is QASYMM8, otherwise same as scores
[out]proposalsBox proposals output tensor of size (5, W*H*A). Data types supported: QASYMM16 with scale of 0.125 and 0 offset if scores is QASYMM8, otherwise same as scores
[out]scores_outBox scores output tensor of size (W*H*A). Data types supported: Same as scores
[out]num_valid_proposalsScalar output tensor which says which of the first proposals are valid. Data types supported: U32
[in]infoContains GenerateProposals operation information described in GenerateProposalsInfo
Note
Only single image prediction is supported. Height and Width (and scale) of the image will be contained in the GenerateProposalsInfo struct.
Proposals contains all the proposals. Of those, only the first num_valid_proposals are valid.

Definition at line 72 of file CLGenerateProposalsLayer.cpp.

References CLKernelLibrary::get().

Referenced by arm_compute::test::validation::DATA_TEST_CASE().

74 {
75  configure(CLKernelLibrary::get().get_compile_context(), scores, deltas, anchors, proposals, scores_out, num_valid_proposals, info);
76 }
void configure(const ICLTensor *scores, const ICLTensor *deltas, const ICLTensor *anchors, ICLTensor *proposals, ICLTensor *scores_out, ICLTensor *num_valid_proposals, const GenerateProposalsInfo &info)
Set the input and output tensors.
static CLKernelLibrary & get()
Access the KernelLibrary singleton.
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)

◆ configure() [2/2]

void configure ( const CLCompileContext compile_context,
const ICLTensor scores,
const ICLTensor deltas,
const ICLTensor anchors,
ICLTensor proposals,
ICLTensor scores_out,
ICLTensor num_valid_proposals,
const GenerateProposalsInfo info 
)

Set the input and output tensors.

Parameters
[in]compile_contextThe compile context to be used.
[in]scoresScores from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors. Data types supported: QASYMM8/F16/F32
[in]deltasBounding box deltas from convolution layer of size (W, H, 4*A). Data types supported: Same as scores
[in]anchorsAnchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if scores is QASYMM8, otherwise same as scores
[out]proposalsBox proposals output tensor of size (5, W*H*A). Data types supported: QASYMM16 with scale of 0.125 and 0 offset if scores is QASYMM8, otherwise same as scores
[out]scores_outBox scores output tensor of size (W*H*A). Data types supported: Same as scores
[out]num_valid_proposalsScalar output tensor which says which of the first proposals are valid. Data types supported: U32
[in]infoContains GenerateProposals operation information described in GenerateProposalsInfo
Note
Only single image prediction is supported. Height and Width (and scale) of the image will be contained in the GenerateProposalsInfo struct.
Proposals contains all the proposals. Of those, only the first num_valid_proposals are valid.

Definition at line 78 of file CLGenerateProposalsLayer.cpp.

References CLTensorAllocator::allocate(), CLTensor::allocator(), ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, arm_compute::auto_init_if_empty(), arm_compute::CHANNEL, CLReshapeLayer::configure(), CLPermute::configure(), CPPBoxWithNonMaximaSuppressionLimit::configure(), ITensorInfo::data_layout(), ITensorInfo::data_type(), ITensorInfo::dimension(), arm_compute::F32, arm_compute::get_data_layout_dimension_index(), arm_compute::HEIGHT, GenerateProposalsInfo::im_height(), GenerateProposalsInfo::im_scale(), GenerateProposalsInfo::im_width(), ITensor::info(), CLTensor::info(), arm_compute::test::validation::info, ITensorAllocator::init(), arm_compute::LINEAR, MemoryGroup::manage(), GenerateProposalsInfo::min_size(), arm_compute::NHWC, GenerateProposalsInfo::nms_thres(), GenerateProposalsInfo::post_nms_topN(), GenerateProposalsInfo::pre_nms_topN(), arm_compute::QASYMM16, arm_compute::QASYMM8, ITensorInfo::quantization_info(), GenerateProposalsInfo::spatial_scale(), TensorInfo::tensor_shape(), arm_compute::U32, CLGenerateProposalsLayer::validate(), GenerateProposalsInfo::values_per_roi(), and arm_compute::WIDTH.

81 {
82  ARM_COMPUTE_ERROR_ON_NULLPTR(scores, deltas, anchors, proposals, scores_out, num_valid_proposals);
83  ARM_COMPUTE_ERROR_THROW_ON(CLGenerateProposalsLayer::validate(scores->info(), deltas->info(), anchors->info(), proposals->info(), scores_out->info(), num_valid_proposals->info(), info));
84 
85  _is_nhwc = scores->info()->data_layout() == DataLayout::NHWC;
86  const DataType scores_data_type = scores->info()->data_type();
87  _is_qasymm8 = scores_data_type == DataType::QASYMM8;
88  const int num_anchors = scores->info()->dimension(get_data_layout_dimension_index(scores->info()->data_layout(), DataLayoutDimension::CHANNEL));
89  const int feat_width = scores->info()->dimension(get_data_layout_dimension_index(scores->info()->data_layout(), DataLayoutDimension::WIDTH));
90  const int feat_height = scores->info()->dimension(get_data_layout_dimension_index(scores->info()->data_layout(), DataLayoutDimension::HEIGHT));
91  const int total_num_anchors = num_anchors * feat_width * feat_height;
92  const int pre_nms_topN = info.pre_nms_topN();
93  const int post_nms_topN = info.post_nms_topN();
94  const size_t values_per_roi = info.values_per_roi();
95 
96  const QuantizationInfo scores_qinfo = scores->info()->quantization_info();
97  const DataType rois_data_type = (_is_qasymm8) ? DataType::QASYMM16 : scores_data_type;
98  const QuantizationInfo rois_qinfo = (_is_qasymm8) ? QuantizationInfo(0.125f, 0) : scores->info()->quantization_info();
99 
100  // Compute all the anchors
101  _memory_group.manage(&_all_anchors);
102  _compute_anchors_kernel->configure(compile_context, anchors, &_all_anchors, ComputeAnchorsInfo(feat_width, feat_height, info.spatial_scale()));
103 
104  const TensorShape flatten_shape_deltas(values_per_roi, total_num_anchors);
105  _deltas_flattened.allocator()->init(TensorInfo(flatten_shape_deltas, 1, scores_data_type, deltas->info()->quantization_info()));
106 
107  // Permute and reshape deltas
108  _memory_group.manage(&_deltas_flattened);
109  if(!_is_nhwc)
110  {
111  _memory_group.manage(&_deltas_permuted);
112  _permute_deltas.configure(compile_context, deltas, &_deltas_permuted, PermutationVector{ 2, 0, 1 });
113  _flatten_deltas.configure(compile_context, &_deltas_permuted, &_deltas_flattened);
114  _deltas_permuted.allocator()->allocate();
115  }
116  else
117  {
118  _flatten_deltas.configure(compile_context, deltas, &_deltas_flattened);
119  }
120 
121  const TensorShape flatten_shape_scores(1, total_num_anchors);
122  _scores_flattened.allocator()->init(TensorInfo(flatten_shape_scores, 1, scores_data_type, scores_qinfo));
123 
124  // Permute and reshape scores
125  _memory_group.manage(&_scores_flattened);
126  if(!_is_nhwc)
127  {
128  _memory_group.manage(&_scores_permuted);
129  _permute_scores.configure(compile_context, scores, &_scores_permuted, PermutationVector{ 2, 0, 1 });
130  _flatten_scores.configure(compile_context, &_scores_permuted, &_scores_flattened);
131  _scores_permuted.allocator()->allocate();
132  }
133  else
134  {
135  _flatten_scores.configure(compile_context, scores, &_scores_flattened);
136  }
137 
138  CLTensor *anchors_to_use = &_all_anchors;
139  CLTensor *deltas_to_use = &_deltas_flattened;
140  if(_is_qasymm8)
141  {
142  _all_anchors_f32.allocator()->init(TensorInfo(_all_anchors.info()->tensor_shape(), 1, DataType::F32));
143  _deltas_flattened_f32.allocator()->init(TensorInfo(_deltas_flattened.info()->tensor_shape(), 1, DataType::F32));
144  _memory_group.manage(&_all_anchors_f32);
145  _memory_group.manage(&_deltas_flattened_f32);
146  // Dequantize anchors to float
147  _dequantize_anchors->configure(compile_context, &_all_anchors, &_all_anchors_f32);
148  _all_anchors.allocator()->allocate();
149  anchors_to_use = &_all_anchors_f32;
150  // Dequantize deltas to float
151  _dequantize_deltas->configure(compile_context, &_deltas_flattened, &_deltas_flattened_f32);
152  _deltas_flattened.allocator()->allocate();
153  deltas_to_use = &_deltas_flattened_f32;
154  }
155  // Bounding box transform
156  _memory_group.manage(&_all_proposals);
157  BoundingBoxTransformInfo bbox_info(info.im_width(), info.im_height(), 1.f);
158  _bounding_box_kernel->configure(compile_context, anchors_to_use, &_all_proposals, deltas_to_use, bbox_info);
159  deltas_to_use->allocator()->allocate();
160  anchors_to_use->allocator()->allocate();
161 
162  _all_proposals_to_use = &_all_proposals;
163  if(_is_qasymm8)
164  {
165  _memory_group.manage(&_all_proposals_quantized);
166  // Requantize all_proposals to QASYMM16 with 0.125 scale and 0 offset
167  _all_proposals_quantized.allocator()->init(TensorInfo(_all_proposals.info()->tensor_shape(), 1, DataType::QASYMM16, QuantizationInfo(0.125f, 0)));
168  _quantize_all_proposals->configure(compile_context, &_all_proposals, &_all_proposals_quantized);
169  _all_proposals.allocator()->allocate();
170  _all_proposals_to_use = &_all_proposals_quantized;
171  }
172  // The original layer implementation first selects the best pre_nms_topN anchors (thus having a lightweight sort)
173  // that are then transformed by bbox_transform. The boxes generated are then fed into a non-sorting NMS operation.
174  // Since we are reusing the NMS layer and we don't implement any CL/sort, we let NMS do the sorting (of all the input)
175  // and the filtering
176  const int scores_nms_size = std::min<int>(std::min<int>(post_nms_topN, pre_nms_topN), total_num_anchors);
177  const float min_size_scaled = info.min_size() * info.im_scale();
178  _memory_group.manage(&_classes_nms_unused);
179  _memory_group.manage(&_keeps_nms_unused);
180 
181  // Note that NMS needs outputs preinitialized.
182  auto_init_if_empty(*scores_out->info(), TensorShape(scores_nms_size), 1, scores_data_type, scores_qinfo);
183  auto_init_if_empty(*_proposals_4_roi_values.info(), TensorShape(values_per_roi, scores_nms_size), 1, rois_data_type, rois_qinfo);
184  auto_init_if_empty(*num_valid_proposals->info(), TensorShape(1), 1, DataType::U32);
185 
186  // Initialize temporaries (unused) outputs
187  _classes_nms_unused.allocator()->init(TensorInfo(TensorShape(scores_nms_size), 1, scores_data_type, scores_qinfo));
188  _keeps_nms_unused.allocator()->init(*scores_out->info());
189 
190  // Save the output (to map and unmap them at run)
191  _scores_out = scores_out;
192  _num_valid_proposals = num_valid_proposals;
193 
194  _memory_group.manage(&_proposals_4_roi_values);
195  _cpp_nms.configure(&_scores_flattened, _all_proposals_to_use, nullptr, scores_out, &_proposals_4_roi_values, &_classes_nms_unused, nullptr, &_keeps_nms_unused, num_valid_proposals,
196  BoxNMSLimitInfo(0.0f, info.nms_thres(), scores_nms_size, false, NMSType::LINEAR, 0.5f, 0.001f, true, min_size_scaled, info.im_width(), info.im_height()));
197  _keeps_nms_unused.allocator()->allocate();
198  _classes_nms_unused.allocator()->allocate();
199  _all_proposals_to_use->allocator()->allocate();
200  _scores_flattened.allocator()->allocate();
201 
202  // Add the first column that represents the batch id. This will be all zeros, as we don't support multiple images
203  _pad_kernel->configure(compile_context, &_proposals_4_roi_values, proposals, PaddingList{ { 1, 0 } });
204  _proposals_4_roi_values.allocator()->allocate();
205 }
TensorInfo * info() const override
Interface to be implemented by the child class to return the tensor&#39;s metadata.
Definition: CLTensor.cpp:41
std::vector< PaddingInfo > PaddingList
List of padding information.
Definition: Types.h:481
1 channel, 1 F32 per channel
Strides PermutationVector
Permutation vector.
Definition: Types.h:49
CLTensorAllocator * allocator()
Return a pointer to the tensor&#39;s allocator.
Definition: CLTensor.cpp:61
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:455
quantized, asymmetric fixed-point 16-bit number
void init(const TensorInfo &input, size_t alignment=0)
Initialize a tensor based on the passed TensorInfo.
void manage(IMemoryManageable *obj) override
Sets a object to be managed by the given memory group.
Definition: MemoryGroup.h:79
1 channel, 1 U32 per channel
quantized, asymmetric fixed-point 8-bit number unsigned
bool auto_init_if_empty(ITensorInfo &info, const TensorShape &shape, int num_channels, DataType data_type, QuantizationInfo quantization_info=QuantizationInfo())
Auto initialize the tensor info (shape, number of channels and data type) if the current assignment i...
static Status validate(const ITensorInfo *scores, const ITensorInfo *deltas, const ITensorInfo *anchors, const ITensorInfo *proposals, const ITensorInfo *scores_out, const ITensorInfo *num_valid_proposals, const GenerateProposalsInfo &info)
Static function to check if given info will lead to a valid configuration of CLGenerateProposalsLayer...
void allocate() override
Allocate size specified by TensorInfo of OpenCL memory.
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
Num samples, height, width, channels.
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:161
void configure(const ICLTensor *input, ICLTensor *output, const PermutationVector &perm)
Set the input and output tensors.
Definition: CLPermute.cpp:50
void configure(const ICLTensor *input, ICLTensor *output)
Initialise the kernel&#39;s inputs and outputs.
size_t get_data_layout_dimension_index(const DataLayout data_layout, const DataLayoutDimension data_layout_dimension)
Get the index of the given dimension.
Definition: Helpers.inl:193
const TensorShape & tensor_shape() const override
Size for each dimension of the tensor.
Definition: TensorInfo.h:262
DataType
Available data types.
Definition: Types.h:77
void configure(const ITensor *scores_in, const ITensor *boxes_in, const ITensor *batch_splits_in, ITensor *scores_out, ITensor *boxes_out, ITensor *classes, ITensor *batch_splits_out=nullptr, ITensor *keeps=nullptr, ITensor *keeps_size=nullptr, const BoxNMSLimitInfo info=BoxNMSLimitInfo())
Configure the BoxWithNonMaximaSuppressionLimit CPP kernel.

◆ operator=()

CLGenerateProposalsLayer& operator= ( const CLGenerateProposalsLayer )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For Neon kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 347 of file CLGenerateProposalsLayer.cpp.

References CLScheduler::enqueue(), CLScheduler::get(), CLReshapeLayer::run(), and CLPermute::run().

Referenced by arm_compute::test::validation::DATA_TEST_CASE().

348 {
349  // Acquire all the temporaries
350  MemoryGroupResourceScope scope_mg(_memory_group);
351 
352  // Compute all the anchors
353  CLScheduler::get().enqueue(*_compute_anchors_kernel, false);
354 
355  // Transpose and reshape the inputs
356  if(!_is_nhwc)
357  {
358  _permute_deltas.run();
359  _permute_scores.run();
360  }
361  _flatten_deltas.run();
362  _flatten_scores.run();
363 
364  if(_is_qasymm8)
365  {
366  CLScheduler::get().enqueue(*_dequantize_anchors, false);
367  CLScheduler::get().enqueue(*_dequantize_deltas, false);
368  }
369 
370  // Build the boxes
371  CLScheduler::get().enqueue(*_bounding_box_kernel, false);
372 
373  if(_is_qasymm8)
374  {
375  CLScheduler::get().enqueue(*_quantize_all_proposals, false);
376  }
377 
378  // Non maxima suppression
379  run_cpp_nms_kernel();
380  // Add dummy batch indexes
381  CLScheduler::get().enqueue(*_pad_kernel, true);
382 }
static CLScheduler & get()
Access the scheduler singleton.
void run() override
Run the kernels contained in the function.
Definition: CLPermute.cpp:71
void run() override
Run the kernels contained in the function.
void enqueue(ICLKernel &kernel, bool flush=true)
Schedule the execution of the passed kernel if possible.

◆ validate()

Status validate ( const ITensorInfo scores,
const ITensorInfo deltas,
const ITensorInfo anchors,
const ITensorInfo proposals,
const ITensorInfo scores_out,
const ITensorInfo num_valid_proposals,
const GenerateProposalsInfo info 
)
static

Static function to check if given info will lead to a valid configuration of CLGenerateProposalsLayer.

Parameters
[in]scoresScores info from convolution layer of size (W, H, A), where H and W are the height and width of the feature map, and A is the number of anchors. Data types supported: QASYMM8/F16/F32
[in]deltasBounding box deltas info from convolution layer of size (W, H, 4*A). Data types supported: Same as scores
[in]anchorsAnchors tensor of size (4, A). Data types supported: QSYMM16 with scale of 0.125 if scores is QASYMM8, otherwise same as scores
[in]proposalsBox proposals info output tensor of size (5, W*H*A). Data types supported: QASYMM16 with scale of 0.125 and 0 offset if scores is QASYMM8, otherwise same as scores
[in]scores_outBox scores output tensor info of size (W*H*A). Data types supported: Same as scores
[in]num_valid_proposalsScalar output tensor info which says which of the first proposals are valid. Data types supported: U32
[in]infoContains GenerateProposals operation information described in GenerateProposalsInfo
Returns
a Status

Definition at line 207 of file CLGenerateProposalsLayer.cpp.

References ARM_COMPUTE_RETURN_ERROR_ON, ARM_COMPUTE_RETURN_ERROR_ON_DATA_LAYOUT_NOT_IN, ARM_COMPUTE_RETURN_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN, ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_LAYOUT, ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_TYPES, ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_SHAPES, ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR, ARM_COMPUTE_RETURN_ON_ERROR, arm_compute::CHANNEL, ICloneable< T >::clone(), ITensorInfo::data_layout(), ITensorInfo::data_type(), ITensorInfo::dimension(), arm_compute::F16, arm_compute::F32, CLScheduler::get(), arm_compute::get_data_layout_dimension_index(), arm_compute::HEIGHT, GenerateProposalsInfo::im_height(), GenerateProposalsInfo::im_width(), CLTensor::map(), ICLTensor::map(), arm_compute::NCHW, arm_compute::NHWC, ITensorInfo::num_dimensions(), UniformQuantizationInfo::offset, arm_compute::QASYMM16, arm_compute::QASYMM8, arm_compute::QSYMM16, ITensorInfo::quantization_info(), CPPBoxWithNonMaximaSuppressionLimit::run(), UniformQuantizationInfo::scale, TensorInfo::set_data_type(), ITensorInfo::set_quantization_info(), GenerateProposalsInfo::spatial_scale(), ITensorInfo::total_size(), arm_compute::U32, QuantizationInfo::uniform(), CLTensor::unmap(), ICLTensor::unmap(), CLDequantizationLayerKernel::validate(), CLReshapeLayer::validate(), CLComputeAllAnchorsKernel::validate(), CLQuantizationLayerKernel::validate(), CLPadLayerKernel::validate(), CLPermute::validate(), CLBoundingBoxTransformKernel::validate(), GenerateProposalsInfo::values_per_roi(), and arm_compute::WIDTH.

Referenced by CLGenerateProposalsLayer::configure(), and arm_compute::test::validation::DATA_TEST_CASE().

209 {
210  ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(scores, deltas, anchors, proposals, scores_out, num_valid_proposals);
215 
216  const int num_anchors = scores->dimension(get_data_layout_dimension_index(scores->data_layout(), DataLayoutDimension::CHANNEL));
217  const int feat_width = scores->dimension(get_data_layout_dimension_index(scores->data_layout(), DataLayoutDimension::WIDTH));
218  const int feat_height = scores->dimension(get_data_layout_dimension_index(scores->data_layout(), DataLayoutDimension::HEIGHT));
219  const int num_images = scores->dimension(3);
220  const int total_num_anchors = num_anchors * feat_width * feat_height;
221  const int values_per_roi = info.values_per_roi();
222 
223  const bool is_qasymm8 = scores->data_type() == DataType::QASYMM8;
224 
225  ARM_COMPUTE_RETURN_ERROR_ON(num_images > 1);
226 
227  if(is_qasymm8)
228  {
230  const UniformQuantizationInfo anchors_qinfo = anchors->quantization_info().uniform();
231  ARM_COMPUTE_RETURN_ERROR_ON(anchors_qinfo.scale != 0.125f);
232  }
233 
234  TensorInfo all_anchors_info(anchors->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true));
235  ARM_COMPUTE_RETURN_ON_ERROR(CLComputeAllAnchorsKernel::validate(anchors, &all_anchors_info, ComputeAnchorsInfo(feat_width, feat_height, info.spatial_scale())));
236 
237  TensorInfo deltas_permuted_info = deltas->clone()->set_tensor_shape(TensorShape(values_per_roi * num_anchors, feat_width, feat_height)).set_is_resizable(true);
238  TensorInfo scores_permuted_info = scores->clone()->set_tensor_shape(TensorShape(num_anchors, feat_width, feat_height)).set_is_resizable(true);
239  if(scores->data_layout() == DataLayout::NHWC)
240  {
241  ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_SHAPES(deltas, &deltas_permuted_info);
242  ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_SHAPES(scores, &scores_permuted_info);
243  }
244  else
245  {
246  ARM_COMPUTE_RETURN_ON_ERROR(CLPermute::validate(deltas, &deltas_permuted_info, PermutationVector{ 2, 0, 1 }));
247  ARM_COMPUTE_RETURN_ON_ERROR(CLPermute::validate(scores, &scores_permuted_info, PermutationVector{ 2, 0, 1 }));
248  }
249 
250  TensorInfo deltas_flattened_info(deltas->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true));
251  ARM_COMPUTE_RETURN_ON_ERROR(CLReshapeLayer::validate(&deltas_permuted_info, &deltas_flattened_info));
252 
253  TensorInfo scores_flattened_info(scores->clone()->set_tensor_shape(TensorShape(1, total_num_anchors)).set_is_resizable(true));
254  TensorInfo proposals_4_roi_values(deltas->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true));
255 
256  ARM_COMPUTE_RETURN_ON_ERROR(CLReshapeLayer::validate(&scores_permuted_info, &scores_flattened_info));
257 
258  TensorInfo *proposals_4_roi_values_to_use = &proposals_4_roi_values;
259  TensorInfo proposals_4_roi_values_quantized(deltas->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true));
260  proposals_4_roi_values_quantized.set_data_type(DataType::QASYMM16).set_quantization_info(QuantizationInfo(0.125f, 0));
261  if(is_qasymm8)
262  {
263  TensorInfo all_anchors_f32_info(anchors->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true).set_data_type(DataType::F32));
264  ARM_COMPUTE_RETURN_ON_ERROR(CLDequantizationLayerKernel::validate(&all_anchors_info, &all_anchors_f32_info));
265 
266  TensorInfo deltas_flattened_f32_info(deltas->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true).set_data_type(DataType::F32));
267  ARM_COMPUTE_RETURN_ON_ERROR(CLDequantizationLayerKernel::validate(&deltas_flattened_info, &deltas_flattened_f32_info));
268 
269  TensorInfo proposals_4_roi_values_f32(deltas->clone()->set_tensor_shape(TensorShape(values_per_roi, total_num_anchors)).set_is_resizable(true).set_data_type(DataType::F32));
270  ARM_COMPUTE_RETURN_ON_ERROR(CLBoundingBoxTransformKernel::validate(&all_anchors_f32_info, &proposals_4_roi_values_f32, &deltas_flattened_f32_info,
271  BoundingBoxTransformInfo(info.im_width(), info.im_height(), 1.f)));
272 
273  ARM_COMPUTE_RETURN_ON_ERROR(CLQuantizationLayerKernel::validate(&proposals_4_roi_values_f32, &proposals_4_roi_values_quantized));
274  proposals_4_roi_values_to_use = &proposals_4_roi_values_quantized;
275  }
276  else
277  {
278  ARM_COMPUTE_RETURN_ON_ERROR(CLBoundingBoxTransformKernel::validate(&all_anchors_info, &proposals_4_roi_values, &deltas_flattened_info,
279  BoundingBoxTransformInfo(info.im_width(), info.im_height(), 1.f)));
280  }
281 
282  ARM_COMPUTE_RETURN_ON_ERROR(CLPadLayerKernel::validate(proposals_4_roi_values_to_use, proposals, PaddingList{ { 1, 0 } }));
283 
284  if(num_valid_proposals->total_size() > 0)
285  {
286  ARM_COMPUTE_RETURN_ERROR_ON(num_valid_proposals->num_dimensions() > 1);
287  ARM_COMPUTE_RETURN_ERROR_ON(num_valid_proposals->dimension(0) > 1);
289  }
290 
291  if(proposals->total_size() > 0)
292  {
293  ARM_COMPUTE_RETURN_ERROR_ON(proposals->num_dimensions() > 2);
294  ARM_COMPUTE_RETURN_ERROR_ON(proposals->dimension(0) != size_t(values_per_roi) + 1);
295  ARM_COMPUTE_RETURN_ERROR_ON(proposals->dimension(1) != size_t(total_num_anchors));
296  if(is_qasymm8)
297  {
299  const UniformQuantizationInfo proposals_qinfo = proposals->quantization_info().uniform();
300  ARM_COMPUTE_RETURN_ERROR_ON(proposals_qinfo.scale != 0.125f);
301  ARM_COMPUTE_RETURN_ERROR_ON(proposals_qinfo.offset != 0);
302  }
303  else
304  {
306  }
307  }
308 
309  if(scores_out->total_size() > 0)
310  {
311  ARM_COMPUTE_RETURN_ERROR_ON(scores_out->num_dimensions() > 1);
312  ARM_COMPUTE_RETURN_ERROR_ON(scores_out->dimension(0) != size_t(total_num_anchors));
314  }
315 
316  return Status{};
317 }
static Status validate(const ITensorInfo *input, const ITensorInfo *output, const PaddingList &padding, PixelValue constant_value=PixelValue(), PaddingMode mode=PaddingMode::CONSTANT)
Static function to check if given info will lead to a valid configuration of CLPadLayerKernel.
static Status validate(const ITensorInfo *input, const ITensorInfo *output)
Static function to check if given info will lead to a valid configuration of CLReshapeLayer.
#define ARM_COMPUTE_RETURN_ERROR_ON_DATA_LAYOUT_NOT_IN(t,...)
Definition: Validate.h:746
quantized, symmetric fixed-point 16-bit number
#define ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_LAYOUT(...)
Definition: Validate.h:494
std::vector< PaddingInfo > PaddingList
List of padding information.
Definition: Types.h:481
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:204
1 channel, 1 F32 per channel
Strides PermutationVector
Permutation vector.
Definition: Types.h:49
quantized, asymmetric fixed-point 16-bit number
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:296
static Status validate(const ITensorInfo *input, const ITensorInfo *output)
Static function to check if given info will lead to a valid configuration of CLQuantizationLayerKerne...
1 channel, 1 F16 per channel
#define ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(...)
Definition: Validate.h:163
static Status validate(const ITensorInfo *input, const ITensorInfo *output)
Static function to check if given info will lead to a valid configuration of CLDequantizationLayerKer...
static Status validate(const ITensorInfo *anchors, const ITensorInfo *all_anchors, const ComputeAnchorsInfo &info)
Static function to check if given info will lead to a valid configuration of CLComputeAllAnchorsKerne...
1 channel, 1 U32 per channel
quantized, asymmetric fixed-point 8-bit number unsigned
Num samples, channels, height, width.
static Status validate(const ITensorInfo *boxes, const ITensorInfo *pred_boxes, const ITensorInfo *deltas, const BoundingBoxTransformInfo &info)
Static function to check if given info will lead to a valid configuration of CLBoundingBoxTransform.
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
#define ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_SHAPES(...)
Definition: Validate.h:443
#define ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DATA_TYPES(...)
Definition: Validate.h:545
Num samples, height, width, channels.
#define ARM_COMPUTE_RETURN_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN(t, c,...)
Definition: Validate.h:792
static Status validate(const ITensorInfo *input, const ITensorInfo *output, const PermutationVector &perm)
Static function to check if given info will lead to a valid configuration of CLPermute.
Definition: CLPermute.cpp:66
size_t get_data_layout_dimension_index(const DataLayout data_layout, const DataLayoutDimension data_layout_dimension)
Get the index of the given dimension.
Definition: Helpers.inl:193

The documentation for this class was generated from the following files: