Compute Library
 19.08
NEHOGMultiDetection Class Reference

Basic function to detect multiple objects (or the same object at different scales) on the same input image using HOG. More...

#include <NEHOGMultiDetection.h>

Collaboration diagram for NEHOGMultiDetection:
[legend]

Public Member Functions

 NEHOGMultiDetection (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 
 NEHOGMultiDetection (const NEHOGMultiDetection &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NEHOGMultiDetectionoperator= (const NEHOGMultiDetection &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
void configure (ITensor *input, const IMultiHOG *multi_hog, IDetectionWindowArray *detection_windows, const ISize2DArray *detection_window_strides, BorderMode border_mode, uint8_t constant_border_value=0, float threshold=0.0f, bool non_maxima_suppression=false, float min_distance=1.0f)
 Initialise the function's source, destination, detection window strides, border mode, threshold and non-maxima suppression. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Detailed Description

Basic function to detect multiple objects (or the same object at different scales) on the same input image using HOG.

This function calls the following NEON kernels:

  1. NEHOGGradient
  2. NEHOGOrientationBinningKernel
  3. NEHOGBlockNormalizationKernel
  4. NEHOGDetector
  5. CPPDetectionWindowNonMaximaSuppressionKernel (executed if non_maxima_suppression == true)
Note
This implementation works if all the HOG data-objects within the IMultiHOG container have the same:
  1. Phase type
  2. Normalization type
  3. L2 hysteresis threshold if the normalization type is L2HYS_NORM

Definition at line 56 of file NEHOGMultiDetection.h.

Constructor & Destructor Documentation

◆ NEHOGMultiDetection() [1/2]

NEHOGMultiDetection ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Definition at line 35 of file NEHOGMultiDetection.cpp.

36  : _memory_group(std::move(memory_manager)),
37  _gradient_kernel(),
38  _orient_bin_kernel(),
39  _block_norm_kernel(),
40  _hog_detect_kernel(),
41  _non_maxima_kernel(),
42  _hog_space(),
43  _hog_norm_space(),
44  _detection_windows(),
45  _mag(),
46  _phase(),
47  _non_maxima_suppression(false),
48  _num_orient_bin_kernel(0),
49  _num_block_norm_kernel(0),
50  _num_hog_detect_kernel(0)
51 {
52 }

◆ NEHOGMultiDetection() [2/2]

Prevent instances of this class from being copied (As this class contains pointers)

Member Function Documentation

◆ configure()

void configure ( ITensor input,
const IMultiHOG multi_hog,
IDetectionWindowArray detection_windows,
const ISize2DArray detection_window_strides,
BorderMode  border_mode,
uint8_t  constant_border_value = 0,
float  threshold = 0.0f,
bool  non_maxima_suppression = false,
float  min_distance = 1.0f 
)

Initialise the function's source, destination, detection window strides, border mode, threshold and non-maxima suppression.

Parameters
[in,out]inputInput tensor. Data type supported: U8 (Written to only for border_mode != UNDEFINED)
[in]multi_hogContainer of multiple HOG data object. Each HOG data object describes one HOG model to detect. This container should store the HOG data-objects in descending or ascending cell_size width order. This will help to understand if the HOG descriptor computation can be skipped for some HOG data-objects
[out]detection_windowsArray of DetectionWindow used for locating the detected objects
[in]detection_window_stridesArray of Size2D used to specify the distance in pixels between 2 consecutive detection windows in x and y directions for each HOG data-object The dimension of this array must be the same of multi_hog->num_models() The i-th detection_window_stride of this array must be multiple of the block_stride stored in the i-th multi_hog array
[in]border_modeBorder mode to use.
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT.
[in]threshold(Optional) Threshold for the distance between features and SVM classifying plane
[in]non_maxima_suppression(Optional) Flag to specify whether the non-maxima suppression is required or not. True if the non-maxima suppression stage has to be computed
[in]min_distance(Optional) Radial Euclidean distance to use for the non-maxima suppression stage

Definition at line 54 of file NEHOGMultiDetection.cpp.

56 {
59  ARM_COMPUTE_ERROR_ON(nullptr == detection_windows);
60  ARM_COMPUTE_ERROR_ON(detection_window_strides->num_values() != multi_hog->num_models());
61 
62  const size_t width = input->info()->dimension(Window::DimX);
63  const size_t height = input->info()->dimension(Window::DimY);
64  const TensorShape &shape_img = input->info()->tensor_shape();
65  const size_t num_models = multi_hog->num_models();
66  PhaseType phase_type = multi_hog->model(0)->info()->phase_type();
67 
68  size_t prev_num_bins = multi_hog->model(0)->info()->num_bins();
69  Size2D prev_cell_size = multi_hog->model(0)->info()->cell_size();
70  Size2D prev_block_size = multi_hog->model(0)->info()->block_size();
71  Size2D prev_block_stride = multi_hog->model(0)->info()->block_stride();
72 
73  /* Check if NEHOGOrientationBinningKernel and NEHOGBlockNormalizationKernel kernels can be skipped for a specific HOG data-object
74  *
75  * 1) NEHOGOrientationBinningKernel and NEHOGBlockNormalizationKernel are skipped if the cell size and the number of bins don't change.
76  * Since "multi_hog" is sorted,it is enough to check the HOG descriptors at level "ith" and level "(i-1)th
77  * 2) NEHOGBlockNormalizationKernel is skipped if the cell size, the number of bins and block size do not change.
78  * Since "multi_hog" is sorted,it is enough to check the HOG descriptors at level "ith" and level "(i-1)th
79  *
80  * @note Since the orientation binning and block normalization kernels can be skipped, we need to keep track of the input to process for each kernel
81  * with "input_orient_bin", "input_hog_detect" and "input_block_norm"
82  */
83  std::vector<size_t> input_orient_bin;
84  std::vector<size_t> input_hog_detect;
85  std::vector<std::pair<size_t, size_t>> input_block_norm;
86 
87  input_orient_bin.push_back(0);
88  input_hog_detect.push_back(0);
89  input_block_norm.emplace_back(0, 0);
90 
91  for(size_t i = 1; i < num_models; ++i)
92  {
93  size_t cur_num_bins = multi_hog->model(i)->info()->num_bins();
94  Size2D cur_cell_size = multi_hog->model(i)->info()->cell_size();
95  Size2D cur_block_size = multi_hog->model(i)->info()->block_size();
96  Size2D cur_block_stride = multi_hog->model(i)->info()->block_stride();
97 
98  if((cur_num_bins != prev_num_bins) || (cur_cell_size.width != prev_cell_size.width) || (cur_cell_size.height != prev_cell_size.height))
99  {
100  prev_num_bins = cur_num_bins;
101  prev_cell_size = cur_cell_size;
102  prev_block_size = cur_block_size;
103  prev_block_stride = cur_block_stride;
104 
105  // Compute orientation binning and block normalization kernels. Update input to process
106  input_orient_bin.push_back(i);
107  input_block_norm.emplace_back(i, input_orient_bin.size() - 1);
108  }
109  else if((cur_block_size.width != prev_block_size.width) || (cur_block_size.height != prev_block_size.height) || (cur_block_stride.width != prev_block_stride.width)
110  || (cur_block_stride.height != prev_block_stride.height))
111  {
112  prev_block_size = cur_block_size;
113  prev_block_stride = cur_block_stride;
114 
115  // Compute block normalization kernel. Update input to process
116  input_block_norm.emplace_back(i, input_orient_bin.size() - 1);
117  }
118 
119  // Update input to process for hog detector kernel
120  input_hog_detect.push_back(input_block_norm.size() - 1);
121  }
122 
123  _detection_windows = detection_windows;
124  _non_maxima_suppression = non_maxima_suppression;
125  _num_orient_bin_kernel = input_orient_bin.size(); // Number of NEHOGOrientationBinningKernel kernels to compute
126  _num_block_norm_kernel = input_block_norm.size(); // Number of NEHOGBlockNormalizationKernel kernels to compute
127  _num_hog_detect_kernel = input_hog_detect.size(); // Number of NEHOGDetector functions to compute
128 
129  _orient_bin_kernel.clear();
130  _block_norm_kernel.clear();
131  _hog_detect_kernel.clear();
132  _hog_space.clear();
133  _hog_norm_space.clear();
134 
135  _orient_bin_kernel.resize(_num_orient_bin_kernel);
136  _block_norm_kernel.resize(_num_block_norm_kernel);
137  _hog_detect_kernel.resize(_num_hog_detect_kernel);
138  _hog_space.resize(_num_orient_bin_kernel);
139  _hog_norm_space.resize(_num_block_norm_kernel);
140  _non_maxima_kernel = CPPDetectionWindowNonMaximaSuppressionKernel();
141 
142  // Allocate tensors for magnitude and phase
143  TensorInfo info_mag(shape_img, Format::S16);
144  _mag.allocator()->init(info_mag);
145 
146  TensorInfo info_phase(shape_img, Format::U8);
147  _phase.allocator()->init(info_phase);
148 
149  // Manage intermediate buffers
150  _memory_group.manage(&_mag);
151  _memory_group.manage(&_phase);
152 
153  // Initialise gradient kernel
154  _gradient_kernel.configure(input, &_mag, &_phase, phase_type, border_mode, constant_border_value);
155 
156  // Configure NETensor for the HOG space and orientation binning kernel
157  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
158  {
159  const size_t idx_multi_hog = input_orient_bin[i];
160 
161  // Get the corresponding cell size and number of bins
162  const Size2D &cell = multi_hog->model(idx_multi_hog)->info()->cell_size();
163  const size_t num_bins = multi_hog->model(idx_multi_hog)->info()->num_bins();
164 
165  // Calculate number of cells along the x and y directions for the hog_space
166  const size_t num_cells_x = width / cell.width;
167  const size_t num_cells_y = height / cell.height;
168 
169  // TensorShape of hog space
170  TensorShape shape_hog_space = input->info()->tensor_shape();
171  shape_hog_space.set(Window::DimX, num_cells_x);
172  shape_hog_space.set(Window::DimY, num_cells_y);
173 
174  // Allocate HOG space
175  TensorInfo info_space(shape_hog_space, num_bins, DataType::F32);
176  _hog_space[i].allocator()->init(info_space);
177 
178  // Manage intermediate buffers
179  _memory_group.manage(&_hog_space[i]);
180 
181  // Initialise orientation binning kernel
182  _orient_bin_kernel[i].configure(&_mag, &_phase, &_hog_space[i], multi_hog->model(idx_multi_hog)->info());
183  }
184 
185  // Allocate intermediate tensors
186  _mag.allocator()->allocate();
187  _phase.allocator()->allocate();
188 
189  // Configure NETensor for the normalized HOG space and block normalization kernel
190  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
191  {
192  const size_t idx_multi_hog = input_block_norm[i].first;
193  const size_t idx_orient_bin = input_block_norm[i].second;
194 
195  // Allocate normalized HOG space
196  TensorInfo tensor_info(*(multi_hog->model(idx_multi_hog)->info()), width, height);
197  _hog_norm_space[i].allocator()->init(tensor_info);
198 
199  // Manage intermediate buffers
200  _memory_group.manage(&_hog_norm_space[i]);
201 
202  // Initialize block normalization kernel
203  _block_norm_kernel[i].configure(&_hog_space[idx_orient_bin], &_hog_norm_space[i], multi_hog->model(idx_multi_hog)->info());
204  }
205 
206  // Allocate intermediate tensors
207  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
208  {
209  _hog_space[i].allocator()->allocate();
210  }
211 
212  // Configure HOG detector kernel
213  for(size_t i = 0; i < _num_hog_detect_kernel; ++i)
214  {
215  const size_t idx_block_norm = input_hog_detect[i];
216 
217  _hog_detect_kernel[i].configure(&_hog_norm_space[idx_block_norm], multi_hog->model(i), detection_windows, detection_window_strides->at(i), threshold, i);
218  }
219 
220  // Configure non maxima suppression kernel
221  _non_maxima_kernel.configure(_detection_windows, min_distance);
222 
223  // Allocate intermediate tensors
224  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
225  {
226  _hog_norm_space[i].allocator()->allocate();
227  }
228 }
size_t num_bins() const
The number of histogram bins for each cell.
Definition: HOGInfo.cpp:111
Shape of a tensor.
Definition: TensorShape.h:39
void init(const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
Shares the same backing memory with another tensor allocator, while the tensor info might be differen...
virtual size_t dimension(size_t index) const =0
Return the size of the requested dimension.
const Size2D & cell_size() const
The cell size in pixels.
Definition: HOGInfo.cpp:91
PhaseType
Phase calculation type.
Definition: Types.h:396
1 channel, 1 U8 per channel
1 channel, 1 F32 per channel
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:337
PhaseType phase_type() const
The type of PhaseType.
Definition: HOGInfo.cpp:126
const Size2D & block_stride() const
The block stride in pixels.
Definition: HOGInfo.cpp:106
size_t height
Height of the image region or rectangle.
Definition: Size2D.h:93
#define ARM_COMPUTE_ERROR_ON_INVALID_MULTI_HOG(m)
Definition: Validate.h:924
TensorAllocator * allocator()
Return a pointer to the tensor's allocator.
Definition: Tensor.cpp:48
void configure(ITensor *input, ITensor *output_magnitude, ITensor *output_phase, PhaseType phase_type, BorderMode border_mode, uint8_t constant_border_value=0)
Initialise the function's source, destinations, phase type and border mode.
virtual size_t num_models() const =0
The number of HOG models stored.
static constexpr size_t DimX
Alias for dimension 0 also known as X dimension.
Definition: Window.h:43
size_t num_values() const
Number of values currently stored in the array.
Definition: IArray.h:68
void manage(TensorType *obj)
Sets a object to be managed by the given memory group.
virtual const TensorShape & tensor_shape() const =0
Size for each dimension of the tensor.
const Size2D & block_size() const
The block size in pixels.
Definition: HOGInfo.cpp:96
void allocate() override
Allocate size specified by TensorInfo of CPU memory.
void configure(IDetectionWindowArray *input_output, float min_distance)
Initialise the kernel's input, output and the euclidean minimum distance.
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
1 channel, 1 S16 per channel
CPP kernel to perform in-place computation of euclidean distance on IDetectionWindowArray.
#define ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN(t, c,...)
Definition: Validate.h:789
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
virtual IHOG * model(size_t index)=0
Return a pointer to the requested HOG model.
size_t width
Width of the image region or rectangle.
Definition: Size2D.h:92
SimpleTensor< T > non_maxima_suppression(const SimpleTensor< T > &src, BorderMode border_mode, T constant_border_value)
Class for specifying the size of an image or rectangle.
Definition: Size2D.h:34
TensorShape & set(size_t dimension, size_t value, bool apply_dim_correction=true)
Accessor to set the value of one of the dimensions.
Definition: TensorShape.h:78
Store the tensor's metadata.
Definition: TensorInfo.h:45
virtual const HOGInfo * info() const =0
Interface to be implemented by the child class to return the HOG's metadata.
SimpleTensor< T > threshold(const SimpleTensor< T > &src, T threshold, T false_value, T true_value, ThresholdType type, T upper)
Definition: Threshold.cpp:35
virtual T & at(size_t index) const
Reference to the element of the array located at the given index.
Definition: IArray.h:117

References TensorAllocator::allocate(), Tensor::allocator(), ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN, ARM_COMPUTE_ERROR_ON_INVALID_MULTI_HOG, IArray< T >::at(), HOGInfo::block_size(), HOGInfo::block_stride(), arm_compute::test::validation::border_mode, HOGInfo::cell_size(), NEHOGGradient::configure(), CPPDetectionWindowNonMaximaSuppressionKernel::configure(), ITensorInfo::dimension(), Window::DimX, Window::DimY, arm_compute::F32, Size2D::height, IHOG::info(), ITensor::info(), TensorAllocator::init(), MemoryGroupBase< TensorType >::manage(), IMultiHOG::model(), arm_compute::test::validation::reference::non_maxima_suppression(), HOGInfo::num_bins(), IMultiHOG::num_models(), IArray< T >::num_values(), HOGInfo::phase_type(), arm_compute::S16, TensorShape::set(), ITensorInfo::tensor_shape(), arm_compute::test::validation::reference::threshold(), arm_compute::U8, and Size2D::width.

◆ operator=()

NEHOGMultiDetection& operator= ( const NEHOGMultiDetection )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For NEON kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 230 of file NEHOGMultiDetection.cpp.

231 {
232  ARM_COMPUTE_ERROR_ON_MSG(_detection_windows == nullptr, "Unconfigured function");
233 
234  MemoryGroupResourceScope scope_mg(_memory_group);
235 
236  // Reset detection window
237  _detection_windows->clear();
238 
239  // Run gradient
240  _gradient_kernel.run();
241 
242  // Run orientation binning kernel
243  for(auto &kernel : _orient_bin_kernel)
244  {
246  }
247 
248  // Run block normalization kernel
249  for(auto &kernel : _block_norm_kernel)
250  {
252  }
253 
254  // Run HOG detector kernel
255  for(auto &kernel : _hog_detect_kernel)
256  {
257  kernel.run();
258  }
259 
260  // Run non-maxima suppression kernel if enabled
261  if(_non_maxima_suppression)
262  {
263  NEScheduler::get().schedule(&_non_maxima_kernel, Window::DimY);
264  }
265 }
void clear()
Clear all the points from the array.
Definition: IArray.h:91
void run() override
Run the kernels contained in the function.
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
Memory group resources scope handling class.
Definition: IMemoryGroup.h:46
virtual void schedule(ICPPKernel *kernel, const Hints &hints)=0
Runs the kernel in the same thread as the caller synchronously.
static IScheduler & get()
Access the scheduler singleton.
Definition: Scheduler.cpp:96
#define ARM_COMPUTE_ERROR_ON_MSG(cond,...)
Definition: Error.h:328

References ARM_COMPUTE_ERROR_ON_MSG, IArray< T >::clear(), Window::DimY, Scheduler::get(), NEHOGGradient::run(), and IScheduler::schedule().


The documentation for this class was generated from the following files: