Compute Library
 21.02
NEHOGMultiDetection Class Reference

Basic function to detect multiple objects (or the same object at different scales) on the same input image using HOG. More...

#include <NEHOGMultiDetection.h>

Collaboration diagram for NEHOGMultiDetection:
[legend]

Public Member Functions

 NEHOGMultiDetection (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 
 NEHOGMultiDetection (const NEHOGMultiDetection &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 NEHOGMultiDetection (NEHOGMultiDetection &&)=delete
 Prevent instances of this class from being moved (As this class contains pointers) More...
 
NEHOGMultiDetectionoperator= (const NEHOGMultiDetection &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NEHOGMultiDetectionoperator= (NEHOGMultiDetection &&)=delete
 Prevent instances of this class from being moved (As this class contains pointers) More...
 
 ~NEHOGMultiDetection ()
 Default destructor. More...
 
void configure (ITensor *input, const IMultiHOG *multi_hog, IDetectionWindowArray *detection_windows, const ISize2DArray *detection_window_strides, BorderMode border_mode, uint8_t constant_border_value=0, float threshold=0.0f, bool non_maxima_suppression=false, float min_distance=1.0f)
 Initialise the function's source, destination, detection window strides, border mode, threshold and non-maxima suppression. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Detailed Description

Basic function to detect multiple objects (or the same object at different scales) on the same input image using HOG.

This function calls the following Neon kernels:

  1. NEHOGGradient
  2. NEHOGOrientationBinningKernel
  3. NEHOGBlockNormalizationKernel
  4. NEHOGDetector
  5. CPPDetectionWindowNonMaximaSuppressionKernel (executed if non_maxima_suppression == true)
Note
This implementation works if all the HOG data-objects within the IMultiHOG container have the same:
  1. Phase type
  2. Normalization type
  3. L2 hysteresis threshold if the normalization type is L2HYS_NORM
Deprecated:
This function is deprecated and is intended to be removed in 21.05 release

Definition at line 60 of file NEHOGMultiDetection.h.

Constructor & Destructor Documentation

◆ NEHOGMultiDetection() [1/3]

NEHOGMultiDetection ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Definition at line 39 of file NEHOGMultiDetection.cpp.

40  : _memory_group(std::move(memory_manager)),
41  _gradient_kernel(),
42  _orient_bin_kernel(),
43  _block_norm_kernel(),
44  _hog_detect_kernel(),
45  _non_maxima_kernel(),
46  _hog_space(),
47  _hog_norm_space(),
48  _detection_windows(),
49  _mag(),
50  _phase(),
51  _non_maxima_suppression(false),
52  _num_orient_bin_kernel(0),
53  _num_block_norm_kernel(0),
54  _num_hog_detect_kernel(0)
55 {
56 }

◆ NEHOGMultiDetection() [2/3]

Prevent instances of this class from being copied (As this class contains pointers)

◆ NEHOGMultiDetection() [3/3]

Prevent instances of this class from being moved (As this class contains pointers)

◆ ~NEHOGMultiDetection()

~NEHOGMultiDetection ( )
default

Default destructor.

Member Function Documentation

◆ configure()

void configure ( ITensor input,
const IMultiHOG multi_hog,
IDetectionWindowArray detection_windows,
const ISize2DArray detection_window_strides,
BorderMode  border_mode,
uint8_t  constant_border_value = 0,
float  threshold = 0.0f,
bool  non_maxima_suppression = false,
float  min_distance = 1.0f 
)

Initialise the function's source, destination, detection window strides, border mode, threshold and non-maxima suppression.

Parameters
[in,out]inputInput tensor. Data type supported: U8 (Written to only for border_mode != UNDEFINED)
[in]multi_hogContainer of multiple HOG data object. Each HOG data object describes one HOG model to detect. This container should store the HOG data-objects in descending or ascending cell_size width order. This will help to understand if the HOG descriptor computation can be skipped for some HOG data-objects
[out]detection_windowsArray of DetectionWindow used for locating the detected objects
[in]detection_window_stridesArray of Size2D used to specify the distance in pixels between 2 consecutive detection windows in x and y directions for each HOG data-object The dimension of this array must be the same of multi_hog->num_models() The i-th detection_window_stride of this array must be multiple of the block_stride stored in the i-th multi_hog array
[in]border_modeBorder mode to use.
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT.
[in]threshold(Optional) Threshold for the distance between features and SVM classifying plane
[in]non_maxima_suppression(Optional) Flag to specify whether the non-maxima suppression is required or not. True if the non-maxima suppression stage has to be computed
[in]min_distance(Optional) Radial Euclidean distance to use for the non-maxima suppression stage

Definition at line 58 of file NEHOGMultiDetection.cpp.

References TensorAllocator::allocate(), Tensor::allocator(), ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN, ARM_COMPUTE_ERROR_ON_INVALID_MULTI_HOG, IArray< T >::at(), HOGInfo::block_size(), HOGInfo::block_stride(), HOGInfo::cell_size(), CPPDetectionWindowNonMaximaSuppressionKernel::configure(), NEHOGGradient::configure(), ITensorInfo::dimension(), Window::DimX, Window::DimY, arm_compute::F32, Size2D::height, IHOG::info(), ITensor::info(), TensorAllocator::init(), MemoryGroup::manage(), IMultiHOG::model(), arm_compute::test::validation::reference::non_maxima_suppression(), HOGInfo::num_bins(), IMultiHOG::num_models(), IArray< T >::num_values(), HOGInfo::phase_type(), arm_compute::S16, TensorShape::set(), ITensorInfo::tensor_shape(), arm_compute::test::validation::reference::threshold(), arm_compute::U8, and Size2D::width.

60 {
63  ARM_COMPUTE_ERROR_ON(nullptr == detection_windows);
64  ARM_COMPUTE_ERROR_ON(detection_window_strides->num_values() != multi_hog->num_models());
65 
66  const size_t width = input->info()->dimension(Window::DimX);
67  const size_t height = input->info()->dimension(Window::DimY);
68  const TensorShape &shape_img = input->info()->tensor_shape();
69  const size_t num_models = multi_hog->num_models();
70  PhaseType phase_type = multi_hog->model(0)->info()->phase_type();
71 
72  size_t prev_num_bins = multi_hog->model(0)->info()->num_bins();
73  Size2D prev_cell_size = multi_hog->model(0)->info()->cell_size();
74  Size2D prev_block_size = multi_hog->model(0)->info()->block_size();
75  Size2D prev_block_stride = multi_hog->model(0)->info()->block_stride();
76 
77  /* Check if NEHOGOrientationBinningKernel and NEHOGBlockNormalizationKernel kernels can be skipped for a specific HOG data-object
78  *
79  * 1) NEHOGOrientationBinningKernel and NEHOGBlockNormalizationKernel are skipped if the cell size and the number of bins don't change.
80  * Since "multi_hog" is sorted,it is enough to check the HOG descriptors at level "ith" and level "(i-1)th
81  * 2) NEHOGBlockNormalizationKernel is skipped if the cell size, the number of bins and block size do not change.
82  * Since "multi_hog" is sorted,it is enough to check the HOG descriptors at level "ith" and level "(i-1)th
83  *
84  * @note Since the orientation binning and block normalization kernels can be skipped, we need to keep track of the input to process for each kernel
85  * with "input_orient_bin", "input_hog_detect" and "input_block_norm"
86  */
87  std::vector<size_t> input_orient_bin;
88  std::vector<size_t> input_hog_detect;
89  std::vector<std::pair<size_t, size_t>> input_block_norm;
90 
91  input_orient_bin.push_back(0);
92  input_hog_detect.push_back(0);
93  input_block_norm.emplace_back(0, 0);
94 
95  for(size_t i = 1; i < num_models; ++i)
96  {
97  size_t cur_num_bins = multi_hog->model(i)->info()->num_bins();
98  Size2D cur_cell_size = multi_hog->model(i)->info()->cell_size();
99  Size2D cur_block_size = multi_hog->model(i)->info()->block_size();
100  Size2D cur_block_stride = multi_hog->model(i)->info()->block_stride();
101 
102  if((cur_num_bins != prev_num_bins) || (cur_cell_size.width != prev_cell_size.width) || (cur_cell_size.height != prev_cell_size.height))
103  {
104  prev_num_bins = cur_num_bins;
105  prev_cell_size = cur_cell_size;
106  prev_block_size = cur_block_size;
107  prev_block_stride = cur_block_stride;
108 
109  // Compute orientation binning and block normalization kernels. Update input to process
110  input_orient_bin.push_back(i);
111  input_block_norm.emplace_back(i, input_orient_bin.size() - 1);
112  }
113  else if((cur_block_size.width != prev_block_size.width) || (cur_block_size.height != prev_block_size.height) || (cur_block_stride.width != prev_block_stride.width)
114  || (cur_block_stride.height != prev_block_stride.height))
115  {
116  prev_block_size = cur_block_size;
117  prev_block_stride = cur_block_stride;
118 
119  // Compute block normalization kernel. Update input to process
120  input_block_norm.emplace_back(i, input_orient_bin.size() - 1);
121  }
122 
123  // Update input to process for hog detector kernel
124  input_hog_detect.push_back(input_block_norm.size() - 1);
125  }
126 
127  _detection_windows = detection_windows;
128  _non_maxima_suppression = non_maxima_suppression;
129  _num_orient_bin_kernel = input_orient_bin.size(); // Number of NEHOGOrientationBinningKernel kernels to compute
130  _num_block_norm_kernel = input_block_norm.size(); // Number of NEHOGBlockNormalizationKernel kernels to compute
131  _num_hog_detect_kernel = input_hog_detect.size(); // Number of NEHOGDetector functions to compute
132 
133  _orient_bin_kernel.clear();
134  _block_norm_kernel.clear();
135  _hog_detect_kernel.clear();
136  _hog_space.clear();
137  _hog_norm_space.clear();
138 
139  _orient_bin_kernel.resize(_num_orient_bin_kernel);
140  _block_norm_kernel.resize(_num_block_norm_kernel);
141  _hog_detect_kernel.resize(_num_hog_detect_kernel);
142  _hog_space.resize(_num_orient_bin_kernel);
143  _hog_norm_space.resize(_num_block_norm_kernel);
144  _non_maxima_kernel = CPPDetectionWindowNonMaximaSuppressionKernel();
145 
146  // Allocate tensors for magnitude and phase
147  TensorInfo info_mag(shape_img, Format::S16);
148  _mag.allocator()->init(info_mag);
149 
150  TensorInfo info_phase(shape_img, Format::U8);
151  _phase.allocator()->init(info_phase);
152 
153  // Manage intermediate buffers
154  _memory_group.manage(&_mag);
155  _memory_group.manage(&_phase);
156 
157  // Initialise gradient kernel
158  _gradient_kernel.configure(input, &_mag, &_phase, phase_type, border_mode, constant_border_value);
159 
160  // Configure NETensor for the HOG space and orientation binning kernel
161  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
162  {
163  const size_t idx_multi_hog = input_orient_bin[i];
164 
165  // Get the corresponding cell size and number of bins
166  const Size2D &cell = multi_hog->model(idx_multi_hog)->info()->cell_size();
167  const size_t num_bins = multi_hog->model(idx_multi_hog)->info()->num_bins();
168 
169  // Calculate number of cells along the x and y directions for the hog_space
170  const size_t num_cells_x = width / cell.width;
171  const size_t num_cells_y = height / cell.height;
172 
173  // TensorShape of hog space
174  TensorShape shape_hog_space = input->info()->tensor_shape();
175  shape_hog_space.set(Window::DimX, num_cells_x);
176  shape_hog_space.set(Window::DimY, num_cells_y);
177 
178  // Allocate HOG space
179  TensorInfo info_space(shape_hog_space, num_bins, DataType::F32);
180  _hog_space[i].allocator()->init(info_space);
181 
182  // Manage intermediate buffers
183  _memory_group.manage(&_hog_space[i]);
184 
185  // Initialise orientation binning kernel
186  _orient_bin_kernel[i].configure(&_mag, &_phase, &_hog_space[i], multi_hog->model(idx_multi_hog)->info());
187  }
188 
189  // Allocate intermediate tensors
190  _mag.allocator()->allocate();
191  _phase.allocator()->allocate();
192 
193  // Configure NETensor for the normalized HOG space and block normalization kernel
194  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
195  {
196  const size_t idx_multi_hog = input_block_norm[i].first;
197  const size_t idx_orient_bin = input_block_norm[i].second;
198 
199  // Allocate normalized HOG space
200  TensorInfo tensor_info(*(multi_hog->model(idx_multi_hog)->info()), width, height);
201  _hog_norm_space[i].allocator()->init(tensor_info);
202 
203  // Manage intermediate buffers
204  _memory_group.manage(&_hog_norm_space[i]);
205 
206  // Initialize block normalization kernel
207  _block_norm_kernel[i].configure(&_hog_space[idx_orient_bin], &_hog_norm_space[i], multi_hog->model(idx_multi_hog)->info());
208  }
209 
210  // Allocate intermediate tensors
211  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
212  {
213  _hog_space[i].allocator()->allocate();
214  }
215 
216  // Configure HOG detector kernel
217  for(size_t i = 0; i < _num_hog_detect_kernel; ++i)
218  {
219  const size_t idx_block_norm = input_hog_detect[i];
220 
221  _hog_detect_kernel[i].configure(&_hog_norm_space[idx_block_norm], multi_hog->model(i), detection_windows, detection_window_strides->at(i), threshold, i);
222  }
223 
224  // Configure non maxima suppression kernel
225  _non_maxima_kernel.configure(_detection_windows, min_distance);
226 
227  // Allocate intermediate tensors
228  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
229  {
230  _hog_norm_space[i].allocator()->allocate();
231  }
232 }
void init(const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
Shares the same backing memory with another tensor allocator, while the tensor info might be differen...
PhaseType
Phase calculation type.
Definition: Types.h:432
1 channel, 1 U8 per channel
1 channel, 1 F32 per channel
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
TensorAllocator * allocator()
Return a pointer to the tensor&#39;s allocator.
Definition: Tensor.cpp:48
void configure(ITensor *input, ITensor *output_magnitude, ITensor *output_phase, PhaseType phase_type, BorderMode border_mode, uint8_t constant_border_value=0)
Initialise the function&#39;s source, destinations, phase type and border mode.
void manage(IMemoryManageable *obj) override
Sets a object to be managed by the given memory group.
Definition: MemoryGroup.h:79
static constexpr size_t DimX
Alias for dimension 0 also known as X dimension.
Definition: Window.h:43
#define ARM_COMPUTE_ERROR_ON_INVALID_MULTI_HOG(m)
Definition: Validate.h:925
void allocate() override
Allocate size specified by TensorInfo of CPU memory.
void configure(IDetectionWindowArray *input_output, float min_distance)
Initialise the kernel&#39;s input, output and the euclidean minimum distance.
1 channel, 1 S16 per channel
#define ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN(t, c,...)
Definition: Validate.h:790
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
SimpleTensor< T > non_maxima_suppression(const SimpleTensor< T > &src, BorderMode border_mode, T constant_border_value)
SimpleTensor< T > threshold(const SimpleTensor< T > &src, T threshold, T false_value, T true_value, ThresholdType type, T upper)
Definition: Threshold.cpp:35

◆ operator=() [1/2]

NEHOGMultiDetection& operator= ( const NEHOGMultiDetection )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

NEHOGMultiDetection& operator= ( NEHOGMultiDetection &&  )
delete

Prevent instances of this class from being moved (As this class contains pointers)

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For Neon kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 234 of file NEHOGMultiDetection.cpp.

References ARM_COMPUTE_ERROR_ON_MSG, IArray< T >::clear(), Window::DimY, Scheduler::get(), NEHOGGradient::run(), and IScheduler::schedule().

235 {
236  ARM_COMPUTE_ERROR_ON_MSG(_detection_windows == nullptr, "Unconfigured function");
237 
238  MemoryGroupResourceScope scope_mg(_memory_group);
239 
240  // Reset detection window
241  _detection_windows->clear();
242 
243  // Run gradient
244  _gradient_kernel.run();
245 
246  // Run orientation binning kernel
247  for(auto &kernel : _orient_bin_kernel)
248  {
250  }
251 
252  // Run block normalization kernel
253  for(auto &kernel : _block_norm_kernel)
254  {
256  }
257 
258  // Run HOG detector kernel
259  for(auto &kernel : _hog_detect_kernel)
260  {
261  kernel.run();
262  }
263 
264  // Run non-maxima suppression kernel if enabled
265  if(_non_maxima_suppression)
266  {
267  NEScheduler::get().schedule(&_non_maxima_kernel, Window::DimY);
268  }
269 }
void clear()
Clear all the points from the array.
Definition: IArray.h:91
void run() override
Run the kernels contained in the function.
#define ARM_COMPUTE_ERROR_ON_MSG(cond, msg)
Definition: Error.h:456
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
virtual void schedule(ICPPKernel *kernel, const Hints &hints)=0
Runs the kernel in the same thread as the caller synchronously.
static IScheduler & get()
Access the scheduler singleton.
Definition: Scheduler.cpp:94

The documentation for this class was generated from the following files: