Compute Library
 21.08
CPPNonMaximumSuppressionKernel Class Reference

CPP Function to perform non maximum suppression on the bounding boxes and scores. More...

#include <CPPNonMaximumSuppressionKernel.h>

Collaboration diagram for CPPNonMaximumSuppressionKernel:
[legend]

Public Member Functions

const char * name () const override
 Name of the kernel. More...
 
 CPPNonMaximumSuppressionKernel ()
 Default constructor. More...
 
 CPPNonMaximumSuppressionKernel (const CPPNonMaximumSuppressionKernel &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CPPNonMaximumSuppressionKerneloperator= (const CPPNonMaximumSuppressionKernel &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 CPPNonMaximumSuppressionKernel (CPPNonMaximumSuppressionKernel &&)=default
 Allow instances of this class to be moved. More...
 
CPPNonMaximumSuppressionKerneloperator= (CPPNonMaximumSuppressionKernel &&)=default
 Allow instances of this class to be moved. More...
 
 ~CPPNonMaximumSuppressionKernel ()=default
 Default destructor. More...
 
void configure (const ITensor *input_bboxes, const ITensor *input_scores, ITensor *output_indices, unsigned int max_output_size, const float score_threshold, const float iou_threshold)
 Configure the kernel to perform non maximal suppression. More...
 
void run (const Window &window, const ThreadInfo &info) override
 Execute the kernel on the passed window. More...
 
- Public Member Functions inherited from ICPPKernel
virtual ~ICPPKernel ()=default
 Default destructor. More...
 
virtual void run_nd (const Window &window, const ThreadInfo &info, const Window &thread_locator)
 legacy compatibility layer for implemantions which do not support thread_locator In these cases we simply narrow the interface down the legacy version More...
 
virtual void run_op (ITensorPack &tensors, const Window &window, const ThreadInfo &info)
 Execute the kernel on the passed window. More...
 
- Public Member Functions inherited from IKernel
 IKernel ()
 Constructor. More...
 
virtual ~IKernel ()=default
 Destructor. More...
 
virtual bool is_parallelisable () const
 Indicates whether or not the kernel is parallelisable. More...
 
virtual BorderSize border_size () const
 The size of the border for that kernel. More...
 
const Windowwindow () const
 The maximum window the kernel can be executed on. More...
 
bool is_window_configured () const
 Function to check if the embedded window of this kernel has been configured. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *input_bboxes, const ITensorInfo *input_scores, const ITensorInfo *output_indices, unsigned int max_output_size, const float score_threshold, const float iou_threshold)
 Static function to check if given arguments will lead to a valid configuration of CPPNonMaximumSuppressionKernel. More...
 

Detailed Description

CPP Function to perform non maximum suppression on the bounding boxes and scores.

Definition at line 38 of file CPPNonMaximumSuppressionKernel.h.

Constructor & Destructor Documentation

◆ CPPNonMaximumSuppressionKernel() [1/3]

Default constructor.

Definition at line 57 of file CPPNonMaximumSuppressionKernel.cpp.

Referenced by CPPNonMaximumSuppressionKernel::name().

58  : _input_bboxes(nullptr), _input_scores(nullptr), _output_indices(nullptr), _max_output_size(0), _score_threshold(0.f), _iou_threshold(0.f), _num_boxes(0)
59 {
60 }

◆ CPPNonMaximumSuppressionKernel() [2/3]

Prevent instances of this class from being copied (As this class contains pointers)

◆ CPPNonMaximumSuppressionKernel() [3/3]

Allow instances of this class to be moved.

◆ ~CPPNonMaximumSuppressionKernel()

Default destructor.

Referenced by CPPNonMaximumSuppressionKernel::name().

Member Function Documentation

◆ configure()

void configure ( const ITensor input_bboxes,
const ITensor input_scores,
ITensor output_indices,
unsigned int  max_output_size,
const float  score_threshold,
const float  iou_threshold 
)

Configure the kernel to perform non maximal suppression.

Parameters
[in]input_bboxesThe input bounding boxes. Data types supported: F32.
[in]input_scoresThe corresponding input confidence. Same as input_bboxes.
[out]output_indicesThe kept indices of bboxes after nms. Data types supported: S32.
[in]max_output_sizeAn integer tensor representing the maximum number of boxes to be selected by non max suppression.
[in]score_thresholdThe threshold used to filter detection results.
[in]iou_thresholdThe threshold used in non maximum suppression.

Definition at line 62 of file CPPNonMaximumSuppressionKernel.cpp.

References ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, arm_compute::auto_init_if_empty(), arm_compute::calculate_max_window(), ITensorInfo::dimension(), ITensor::info(), and arm_compute::U8.

Referenced by CPPNonMaximumSuppressionKernel::name().

64 {
65  ARM_COMPUTE_ERROR_ON_NULLPTR(input_bboxes, input_scores, output_indices);
66  ARM_COMPUTE_ERROR_THROW_ON(validate_arguments(input_bboxes->info(), input_scores->info(), output_indices->info(), max_output_size, score_threshold, iou_threshold));
67 
68  auto_init_if_empty(*output_indices->info(), TensorShape(max_output_size), 1, DataType::U8, QuantizationInfo());
69 
70  _input_bboxes = input_bboxes;
71  _input_scores = input_scores;
72  _output_indices = output_indices;
73  _score_threshold = score_threshold;
74  _iou_threshold = iou_threshold;
75  _max_output_size = max_output_size;
76  _num_boxes = input_scores->info()->dimension(0);
77 
78  // Configure kernel window
79  Window win = calculate_max_window(*output_indices->info(), Steps());
80 
81  // The CPPNonMaximumSuppressionKernel doesn't need padding so update_window_and_padding() can be skipped
82  ICPPKernel::configure(win);
83 }
Window calculate_max_window(const ValidRegion &valid_region, const Steps &steps, bool skip_border, BorderSize border_size)
1 channel, 1 U8 per channel
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:455
bool auto_init_if_empty(ITensorInfo &info, const TensorShape &shape, int num_channels, DataType data_type, QuantizationInfo quantization_info=QuantizationInfo())
Auto initialize the tensor info (shape, number of channels and data type) if the current assignment i...
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:157

◆ name()

◆ operator=() [1/2]

Prevent instances of this class from being copied (As this class contains pointers)

Referenced by CPPNonMaximumSuppressionKernel::name().

◆ operator=() [2/2]

Allow instances of this class to be moved.

◆ run()

void run ( const Window window,
const ThreadInfo info 
)
overridevirtual

Execute the kernel on the passed window.

Warning
If is_parallelisable() returns false then the passed window must be equal to window()
Note
The window has to be a region within the window returned by the window() method
The width of the window has to be a multiple of num_elems_processed_per_iteration().
Parameters
[in]windowRegion on which to execute the kernel. (Must be a region of the window returned by window())
[in]infoInfo about executing thread and CPU.

Reimplemented from ICPPKernel.

Definition at line 92 of file CPPNonMaximumSuppressionKernel.cpp.

References ARM_COMPUTE_ERROR_ON_INVALID_SUBWINDOW, ARM_COMPUTE_ERROR_ON_UNCONFIGURED_KERNEL, ARM_COMPUTE_UNUSED, arm_compute::mlgo::parser::end(), ITensor::ptr_to_element(), and IKernel::window().

Referenced by CPPNonMaximumSuppressionKernel::name().

93 {
98 
99  // Auxiliary tensors
100  std::vector<int> indices_above_thd;
101  std::vector<float> scores_above_thd;
102  for(unsigned int i = 0; i < _num_boxes; ++i)
103  {
104  const float score_i = *(reinterpret_cast<float *>(_input_scores->ptr_to_element(Coordinates(i))));
105  if(score_i >= _score_threshold)
106  {
107  scores_above_thd.emplace_back(score_i);
108  indices_above_thd.emplace_back(i);
109  }
110  }
111 
112  // Sort selected indices based on scores
113  const unsigned int num_above_thd = indices_above_thd.size();
114  std::vector<unsigned int> sorted_indices;
115  sorted_indices.resize(num_above_thd);
116  std::iota(sorted_indices.data(), sorted_indices.data() + num_above_thd, 0);
117  std::sort(std::begin(sorted_indices),
118  std::end(sorted_indices),
119  [&](unsigned int first, unsigned int second)
120  {
121  return scores_above_thd[first] > scores_above_thd[second];
122  });
123 
124  // Number of output is the minimum between max_detection and the scores above the threshold
125  const unsigned int num_output = std::min(_max_output_size, num_above_thd);
126  unsigned int output_idx = 0;
127  std::vector<bool> visited(num_above_thd, false);
128 
129  // Keep only boxes with small IoU
130  for(unsigned int i = 0; i < num_above_thd; ++i)
131  {
132  // Check if the output is full
133  if(output_idx >= num_output)
134  {
135  break;
136  }
137 
138  // Check if it was already visited, if not add it to the output and update the indices counter
139  if(!visited[sorted_indices[i]])
140  {
141  *(reinterpret_cast<int *>(_output_indices->ptr_to_element(Coordinates(output_idx)))) = indices_above_thd[sorted_indices[i]];
142  visited[sorted_indices[i]] = true;
143  ++output_idx;
144  }
145  else
146  {
147  continue;
148  }
149 
150  // Once added one element at the output check if the next ones overlap and can be skipped
151  for(unsigned int j = i + 1; j < num_above_thd; ++j)
152  {
153  if(!visited[sorted_indices[j]])
154  {
155  // Calculate IoU
156  const unsigned int i_index = indices_above_thd[sorted_indices[i]];
157  const unsigned int j_index = indices_above_thd[sorted_indices[j]];
158  // Box-corner format: xmin, ymin, xmax, ymax
159  const auto box_i_xmin = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(0, i_index))));
160  const auto box_i_ymin = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(1, i_index))));
161  const auto box_i_xmax = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(2, i_index))));
162  const auto box_i_ymax = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(3, i_index))));
163 
164  const auto box_j_xmin = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(0, j_index))));
165  const auto box_j_ymin = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(1, j_index))));
166  const auto box_j_xmax = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(2, j_index))));
167  const auto box_j_ymax = *(reinterpret_cast<float *>(_input_bboxes->ptr_to_element(Coordinates(3, j_index))));
168 
169  const float area_i = (box_i_xmax - box_i_xmin) * (box_i_ymax - box_i_ymin);
170  const float area_j = (box_j_xmax - box_j_xmin) * (box_j_ymax - box_j_ymin);
171  float overlap;
172  if(area_i <= 0 || area_j <= 0)
173  {
174  overlap = 0.0f;
175  }
176  else
177  {
178  const auto y_min_intersection = std::max<float>(box_i_ymin, box_j_ymin);
179  const auto x_min_intersection = std::max<float>(box_i_xmin, box_j_xmin);
180  const auto y_max_intersection = std::min<float>(box_i_ymax, box_j_ymax);
181  const auto x_max_intersection = std::min<float>(box_i_xmax, box_j_xmax);
182  const auto area_intersection = std::max<float>(y_max_intersection - y_min_intersection, 0.0f) * std::max<float>(x_max_intersection - x_min_intersection, 0.0f);
183  overlap = area_intersection / (area_i + area_j - area_intersection);
184  }
185 
186  if(overlap > _iou_threshold)
187  {
188  visited[sorted_indices[j]] = true;
189  }
190  }
191  }
192  }
193  // The output could be full but not the output indices tensor
194  // Instead return values not valid we put -1
195  for(; output_idx < _max_output_size; ++output_idx)
196  {
197  *(reinterpret_cast<int *>(_output_indices->ptr_to_element(Coordinates(output_idx)))) = -1;
198  }
199 }
const Window & window() const
The maximum window the kernel can be executed on.
Definition: IKernel.cpp:28
uint8_t * ptr_to_element(const Coordinates &id) const
Return a pointer to the element at the passed coordinates.
Definition: ITensor.h:63
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:152
void end(TokenStream &in, bool &valid)
Definition: MLGOParser.cpp:290
#define ARM_COMPUTE_ERROR_ON_UNCONFIGURED_KERNEL(k)
Definition: Validate.h:915
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
#define ARM_COMPUTE_ERROR_ON_INVALID_SUBWINDOW(f, s)
Definition: Validate.h:201

◆ validate()

Status validate ( const ITensorInfo input_bboxes,
const ITensorInfo input_scores,
const ITensorInfo output_indices,
unsigned int  max_output_size,
const float  score_threshold,
const float  iou_threshold 
)
static

Static function to check if given arguments will lead to a valid configuration of CPPNonMaximumSuppressionKernel.

Parameters
[in]input_bboxesThe input bounding boxes tensor info. Data types supported: F32.
[in]input_scoresThe corresponding input confidence tensor info. Same as input_bboxes.
[out]output_indicesThe kept indices of bboxes after nms tensor info. Data types supported: S32.
[in]max_output_sizeAn integer tensor representing the maximum number of boxes to be selected by non max suppression.
[in]score_thresholdThe threshold used to filter detection results.
[in]iou_thresholdThe threshold used in non maximum suppression.

Definition at line 85 of file CPPNonMaximumSuppressionKernel.cpp.

References ARM_COMPUTE_RETURN_ON_ERROR.

Referenced by CPPNonMaximumSuppressionKernel::name(), and CPPNonMaximumSuppression::validate().

87 {
88  ARM_COMPUTE_RETURN_ON_ERROR(validate_arguments(bboxes, scores, output_indices, max_output_size, score_threshold, iou_threshold));
89  return Status{};
90 }
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:204

The documentation for this class was generated from the following files: