Compute Library
 19.08
CPPDetectionOutputLayer Class Reference

CPP Function to generate the detection output based on location and confidence predictions by doing non maximum suppression. More...

#include <CPPDetectionOutputLayer.h>

Collaboration diagram for CPPDetectionOutputLayer:
[legend]

Public Member Functions

 CPPDetectionOutputLayer ()
 Default constructor. More...
 
void configure (const ITensor *input_loc, const ITensor *input_conf, const ITensor *input_priorbox, ITensor *output, DetectionOutputLayerInfo info=DetectionOutputLayerInfo())
 Configure the detection output layer CPP kernel. More...
 
void run () override
 Run the kernels contained in the function. More...
 
 CPPDetectionOutputLayer (const CPPDetectionOutputLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CPPDetectionOutputLayeroperator= (const CPPDetectionOutputLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *input_loc, const ITensorInfo *input_conf, const ITensorInfo *input_priorbox, const ITensorInfo *output, DetectionOutputLayerInfo info=DetectionOutputLayerInfo())
 Static function to check if given info will lead to a valid configuration of CPPDetectionOutputLayer. More...
 

Detailed Description

CPP Function to generate the detection output based on location and confidence predictions by doing non maximum suppression.

Note
Intended for use with MultiBox detection method.

Definition at line 40 of file CPPDetectionOutputLayer.h.

Constructor & Destructor Documentation

◆ CPPDetectionOutputLayer() [1/2]

Default constructor.

Definition at line 385 of file CPPDetectionOutputLayer.cpp.

386  : _input_loc(nullptr), _input_conf(nullptr), _input_priorbox(nullptr), _output(nullptr), _info(), _num_priors(), _num(), _all_location_predictions(), _all_confidence_scores(), _all_prior_bboxes(),
387  _all_prior_variances(), _all_decode_bboxes(), _all_indices()
388 {
389 }

◆ CPPDetectionOutputLayer() [2/2]

Prevent instances of this class from being copied (As this class contains pointers)

Member Function Documentation

◆ configure()

void configure ( const ITensor input_loc,
const ITensor input_conf,
const ITensor input_priorbox,
ITensor output,
DetectionOutputLayerInfo  info = DetectionOutputLayerInfo() 
)

Configure the detection output layer CPP kernel.

Parameters
[in]input_locThe mbox location input tensor of size [C1, N]. Data types supported: F32.
[in]input_confThe mbox confidence input tensor of size [C2, N]. Data types supported: F32.
[in]input_priorboxThe mbox prior box input tensor of size [C3, 2, N]. Data types supported: F32.
[out]outputThe output tensor of size [7, M]. Data types supported: Same as input
[in]info(Optional) DetectionOutputLayerInfo information.
Note
Output contains all the detections. Of those, only the ones selected by the valid region are valid.

Definition at line 391 of file CPPDetectionOutputLayer.cpp.

392 {
393  ARM_COMPUTE_ERROR_ON_NULLPTR(input_loc, input_conf, input_priorbox, output);
394  // Output auto initialization if not yet initialized
395  // Since the number of bboxes to kept is unknown before nms, the shape is set to the maximum
396  // The maximum is keep_top_k * input_loc_size[1]
397  // Each row is a 7 dimension std::vector, which stores [image_id, label, confidence, xmin, ymin, xmax, ymax]
398  const unsigned int max_size = info.keep_top_k() * (input_loc->info()->num_dimensions() > 1 ? input_loc->info()->dimension(1) : 1);
399  auto_init_if_empty(*output->info(), input_loc->info()->clone()->set_tensor_shape(TensorShape(7U, max_size)));
400 
401  // Perform validation step
402  ARM_COMPUTE_ERROR_THROW_ON(validate_arguments(input_loc->info(), input_conf->info(), input_priorbox->info(), output->info(), info));
403 
404  _input_loc = input_loc;
405  _input_conf = input_conf;
406  _input_priorbox = input_priorbox;
407  _output = output;
408  _info = info;
409  _num_priors = input_priorbox->info()->dimension(0) / 4;
410  _num = (_input_loc->info()->num_dimensions() > 1 ? _input_loc->info()->dimension(1) : 1);
411 
412  _all_location_predictions.resize(_num);
413  _all_confidence_scores.resize(_num);
414  _all_prior_bboxes.resize(_num_priors);
415  _all_prior_variances.resize(_num_priors);
416  _all_decode_bboxes.resize(_num);
417 
418  for(int i = 0; i < _num; ++i)
419  {
420  for(int c = 0; c < _info.num_loc_classes(); ++c)
421  {
422  const int label = _info.share_location() ? -1 : c;
423  if(label == _info.background_label_id())
424  {
425  // Ignore background class.
426  continue;
427  }
428  _all_decode_bboxes[i][label].resize(_num_priors);
429  }
430  }
431  _all_indices.resize(_num);
432 
433  Coordinates coord;
434  coord.set_num_dimensions(output->info()->num_dimensions());
435  output->info()->set_valid_region(ValidRegion(coord, output->info()->tensor_shape()));
436 }
virtual size_t num_dimensions() const =0
The number of dimensions of the tensor (rank)
bool share_location() const
Get share location.
Definition: Types.h:1016
virtual size_t dimension(size_t index) const =0
Return the size of the requested dimension.
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:327
bool auto_init_if_empty(ITensorInfo &info, const TensorShape &shape, int num_channels, DataType data_type, QuantizationInfo quantization_info=QuantizationInfo())
Auto initialize the tensor info (shape, number of channels and data type) if the current assignment i...
Definition: Helpers.inl:201
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:161
int background_label_id() const
Get background label ID.
Definition: Types.h:1046
int num_loc_classes() const
Get number of location classes.
Definition: Types.h:1061

References ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, arm_compute::auto_init_if_empty(), DetectionOutputLayerInfo::background_label_id(), ICloneable< T >::clone(), ITensorInfo::dimension(), ITensor::info(), arm_compute::test::validation::info, ITensorInfo::num_dimensions(), DetectionOutputLayerInfo::num_loc_classes(), Dimensions< T >::set_num_dimensions(), ITensorInfo::set_valid_region(), DetectionOutputLayerInfo::share_location(), ITensorInfo::tensor_shape(), and arm_compute::U.

◆ operator=()

CPPDetectionOutputLayer& operator= ( const CPPDetectionOutputLayer )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For NEON kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 444 of file CPPDetectionOutputLayer.cpp.

445 {
446  // Retrieve all location predictions.
447  retrieve_all_loc_predictions(_input_loc, _num, _num_priors, _info.num_loc_classes(), _info.share_location(), _all_location_predictions);
448 
449  // Retrieve all confidences.
450  retrieve_all_conf_scores(_input_conf, _num, _num_priors, _info.num_classes(), _all_confidence_scores);
451 
452  // Retrieve all prior bboxes.
453  retrieve_all_priorbox(_input_priorbox, _num_priors, _all_prior_bboxes, _all_prior_variances);
454 
455  // Decode all loc predictions to bboxes
456  const bool clip_bbox = false;
457  for(int i = 0; i < _num; ++i)
458  {
459  for(int c = 0; c < _info.num_loc_classes(); ++c)
460  {
461  const int label = _info.share_location() ? -1 : c;
462  if(label == _info.background_label_id())
463  {
464  // Ignore background class.
465  continue;
466  }
467  ARM_COMPUTE_ERROR_ON_MSG(_all_location_predictions[i].find(label) == _all_location_predictions[i].end(), "Could not find location predictions for label %d.", label);
468 
469  const std::vector<BBox> &label_loc_preds = _all_location_predictions[i].find(label)->second;
470 
471  const int num_bboxes = _all_prior_bboxes.size();
472  ARM_COMPUTE_ERROR_ON(_all_prior_variances[i].size() != 4);
473 
474  for(int j = 0; j < num_bboxes; ++j)
475  {
476  DecodeBBox(_all_prior_bboxes[j], _all_prior_variances[j], _info.code_type(), _info.variance_encoded_in_target(), clip_bbox, label_loc_preds[j], _all_decode_bboxes[i][label][j]);
477  }
478  }
479  }
480 
481  int num_kept = 0;
482 
483  for(int i = 0; i < _num; ++i)
484  {
485  const LabelBBox &decode_bboxes = _all_decode_bboxes[i];
486  const std::map<int, std::vector<float>> &conf_scores = _all_confidence_scores[i];
487 
488  std::map<int, std::vector<int>> indices;
489  int num_det = 0;
490  for(int c = 0; c < _info.num_classes(); ++c)
491  {
492  if(c == _info.background_label_id())
493  {
494  // Ignore background class
495  continue;
496  }
497  const int label = _info.share_location() ? -1 : c;
498  if(conf_scores.find(c) == conf_scores.end() || decode_bboxes.find(label) == decode_bboxes.end())
499  {
500  ARM_COMPUTE_ERROR("Could not find predictions for label %d.", label);
501  }
502  const std::vector<float> &scores = conf_scores.find(c)->second;
503  const std::vector<BBox> &bboxes = decode_bboxes.find(label)->second;
504 
505  ApplyNMSFast(bboxes, scores, _info.confidence_threshold(), _info.nms_threshold(), _info.eta(), _info.top_k(), indices[c]);
506 
507  num_det += indices[c].size();
508  }
509 
510  int num_to_add = 0;
511  if(_info.keep_top_k() > -1 && num_det > _info.keep_top_k())
512  {
513  std::vector<std::pair<float, std::pair<int, int>>> score_index_pairs;
514  for(auto const &it : indices)
515  {
516  const int label = it.first;
517  const std::vector<int> &label_indices = it.second;
518 
519  if(conf_scores.find(label) == conf_scores.end())
520  {
521  ARM_COMPUTE_ERROR("Could not find predictions for label %d.", label);
522  }
523 
524  const std::vector<float> &scores = conf_scores.find(label)->second;
525  for(auto idx : label_indices)
526  {
527  ARM_COMPUTE_ERROR_ON(idx > static_cast<int>(scores.size()));
528  score_index_pairs.emplace_back(std::make_pair(scores[idx], std::make_pair(label, idx)));
529  }
530  }
531 
532  // Keep top k results per image.
533  std::sort(score_index_pairs.begin(), score_index_pairs.end(), SortScorePairDescend<std::pair<int, int>>);
534  score_index_pairs.resize(_info.keep_top_k());
535 
536  // Store the new indices.
537 
538  std::map<int, std::vector<int>> new_indices;
539  for(auto score_index_pair : score_index_pairs)
540  {
541  int label = score_index_pair.second.first;
542  int idx = score_index_pair.second.second;
543  new_indices[label].push_back(idx);
544  }
545  _all_indices[i] = new_indices;
546  num_to_add = _info.keep_top_k();
547  }
548  else
549  {
550  _all_indices[i] = indices;
551  num_to_add = num_det;
552  }
553  num_kept += num_to_add;
554  }
555 
556  //Update the valid region of the ouput to mark the exact number of detection
557  _output->info()->set_valid_region(ValidRegion(Coordinates(0, 0), TensorShape(7, num_kept)));
558 
559  int count = 0;
560  for(int i = 0; i < _num; ++i)
561  {
562  const std::map<int, std::vector<float>> &conf_scores = _all_confidence_scores[i];
563  const LabelBBox &decode_bboxes = _all_decode_bboxes[i];
564  for(auto &it : _all_indices[i])
565  {
566  const int label = it.first;
567  const std::vector<float> &scores = conf_scores.find(label)->second;
568  const int loc_label = _info.share_location() ? -1 : label;
569  if(conf_scores.find(label) == conf_scores.end() || decode_bboxes.find(loc_label) == decode_bboxes.end())
570  {
571  // Either if there are no confidence predictions
572  // or there are no location predictions for current label.
573  ARM_COMPUTE_ERROR("Could not find predictions for the label %d.", label);
574  }
575  const std::vector<BBox> &bboxes = decode_bboxes.find(loc_label)->second;
576  const std::vector<int> &indices = it.second;
577 
578  for(auto idx : indices)
579  {
580  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7)))) = i;
581  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7 + 1)))) = label;
582  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7 + 2)))) = scores[idx];
583  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7 + 3)))) = bboxes[idx][0];
584  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7 + 4)))) = bboxes[idx][1];
585  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7 + 5)))) = bboxes[idx][2];
586  *(reinterpret_cast<float *>(_output->ptr_to_element(Coordinates(count * 7 + 6)))) = bboxes[idx][3];
587 
588  ++count;
589  }
590  }
591  }
592 }
#define ARM_COMPUTE_ERROR(...)
Print the given message then throw an std::runtime_error.
Definition: Error.h:261
int num_classes() const
Get num classes.
Definition: Types.h:1011
uint8_t * ptr_to_element(const Coordinates &id) const
Return a pointer to the element at the passed coordinates.
Definition: ITensor.h:63
bool share_location() const
Get share location.
Definition: Types.h:1016
std::map< int, std::vector< BBox > > LabelBBox
Definition: Types.h:950
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:337
float eta() const
Get eta.
Definition: Types.h:1041
int top_k() const
Get top K.
Definition: Types.h:1056
virtual void set_valid_region(const ValidRegion &valid_region)=0
Set the valid region of the tensor.
bool variance_encoded_in_target() const
Get if variance encoded in target.
Definition: Types.h:1026
float nms_threshold() const
Get nms threshold.
Definition: Types.h:1036
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
DetectionOutputLayerCodeType code_type() const
Get detection output code type.
Definition: Types.h:1021
int keep_top_k() const
Get the number of total bounding boxes to be kept per image.
Definition: Types.h:1031
int background_label_id() const
Get background label ID.
Definition: Types.h:1046
float confidence_threshold() const
Get confidence threshold.
Definition: Types.h:1051
int num_loc_classes() const
Get number of location classes.
Definition: Types.h:1061
#define ARM_COMPUTE_ERROR_ON_MSG(cond,...)
Definition: Error.h:328

References ARM_COMPUTE_ERROR, ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_ERROR_ON_MSG, DetectionOutputLayerInfo::background_label_id(), DetectionOutputLayerInfo::code_type(), DetectionOutputLayerInfo::confidence_threshold(), DetectionOutputLayerInfo::eta(), ITensor::info(), DetectionOutputLayerInfo::keep_top_k(), DetectionOutputLayerInfo::nms_threshold(), DetectionOutputLayerInfo::num_classes(), DetectionOutputLayerInfo::num_loc_classes(), ITensor::ptr_to_element(), ITensorInfo::set_valid_region(), DetectionOutputLayerInfo::share_location(), DetectionOutputLayerInfo::top_k(), and DetectionOutputLayerInfo::variance_encoded_in_target().

◆ validate()

Status validate ( const ITensorInfo input_loc,
const ITensorInfo input_conf,
const ITensorInfo input_priorbox,
const ITensorInfo output,
DetectionOutputLayerInfo  info = DetectionOutputLayerInfo() 
)
static

Static function to check if given info will lead to a valid configuration of CPPDetectionOutputLayer.

Parameters
[in]input_locThe mbox location input tensor info. Data types supported: F32.
[in]input_confThe mbox confidence input tensor info. Data types supported: F32.
[in]input_priorboxThe mbox prior box input tensor info. Data types supported: F32.
[in]outputThe output tensor info. Data types supported: Same as input
[in]info(Optional) DetectionOutputLayerInfo information.
Returns
a status

Definition at line 438 of file CPPDetectionOutputLayer.cpp.

439 {
440  ARM_COMPUTE_RETURN_ON_ERROR(validate_arguments(input_loc, input_conf, input_priorbox, output, info));
441  return Status{};
442 }
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:193

References ARM_COMPUTE_RETURN_ON_ERROR, and arm_compute::test::validation::info.


The documentation for this class was generated from the following files: