Compute Library
 19.08
CLHOGMultiDetection Class Reference

Basic function to detect multiple objects (or the same object at different scales) on the same input image using HOG. More...

#include <CLHOGMultiDetection.h>

Collaboration diagram for CLHOGMultiDetection:
[legend]

Public Member Functions

 CLHOGMultiDetection (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 
 CLHOGMultiDetection (const CLHOGMultiDetection &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CLHOGMultiDetectionoperator= (const CLHOGMultiDetection &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
void configure (ICLTensor *input, const ICLMultiHOG *multi_hog, ICLDetectionWindowArray *detection_windows, ICLSize2DArray *detection_window_strides, BorderMode border_mode, uint8_t constant_border_value=0, float threshold=0.0f, bool non_maxima_suppression=false, float min_distance=1.0f)
 Initialise the function's source, destination, detection window strides, border mode, threshold and non-maxima suppression. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Detailed Description

Basic function to detect multiple objects (or the same object at different scales) on the same input image using HOG.

This function calls the following kernels:

  1. CLHOGGradient
  2. CLHOGOrientationBinningKernel
  3. CLHOGBlockNormalizationKernel
  4. CLHOGDetector
  5. CPPDetectionWindowNonMaximaSuppressionKernel (executed if non_maxima_suppression == true)
Note
This implementation works if all the HOG data-objects within the IMultiHOG container have the same:
  1. Phase type
  2. Normalization type
  3. L2 hysteresis threshold if the normalization type is L2HYS_NORM

Definition at line 56 of file CLHOGMultiDetection.h.

Constructor & Destructor Documentation

◆ CLHOGMultiDetection() [1/2]

CLHOGMultiDetection ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Definition at line 37 of file CLHOGMultiDetection.cpp.

38  : _memory_group(std::move(memory_manager)),
39  _gradient_kernel(),
40  _orient_bin_kernel(),
41  _block_norm_kernel(),
42  _hog_detect_kernel(),
43  _non_maxima_kernel(),
44  _hog_space(),
45  _hog_norm_space(),
46  _detection_windows(),
47  _mag(),
48  _phase(),
49  _non_maxima_suppression(false),
50  _num_orient_bin_kernel(0),
51  _num_block_norm_kernel(0),
52  _num_hog_detect_kernel(0)
53 {
54 }

◆ CLHOGMultiDetection() [2/2]

Prevent instances of this class from being copied (As this class contains pointers)

Member Function Documentation

◆ configure()

void configure ( ICLTensor input,
const ICLMultiHOG multi_hog,
ICLDetectionWindowArray detection_windows,
ICLSize2DArray detection_window_strides,
BorderMode  border_mode,
uint8_t  constant_border_value = 0,
float  threshold = 0.0f,
bool  non_maxima_suppression = false,
float  min_distance = 1.0f 
)

Initialise the function's source, destination, detection window strides, border mode, threshold and non-maxima suppression.

Parameters
[in,out]inputInput tensor. Data type supported: U8 (Written to only for border_mode != UNDEFINED)
[in]multi_hogContainer of multiple HOG data object. Each HOG data object describes one HOG model to detect. This container should store the HOG data-objects in descending or ascending cell_size width order. This will help to understand if the HOG descriptor computation can be skipped for some HOG data-objects
[out]detection_windowsArray of DetectionWindow used for locating the detected objects
[in]detection_window_stridesArray of Size2D used to specify the distance in pixels between 2 consecutive detection windows in x and y directions for each HOG data-object The dimension of this array must be the same of multi_hog->num_models() The i-th detection_window_stride of this array must be multiple of the block_stride stored in the i-th multi_hog array
[in]border_modeBorder mode to use.
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT.
[in]threshold(Optional) Threshold for the distance between features and SVM classifying plane
[in]non_maxima_suppression(Optional) Flag to specify whether the non-maxima suppression is required or not. True if the non-maxima suppression stage has to be computed
[in]min_distance(Optional) Radial Euclidean distance to use for the non-maxima suppression stage

Definition at line 56 of file CLHOGMultiDetection.cpp.

58 {
61  ARM_COMPUTE_ERROR_ON(nullptr == detection_windows);
62  ARM_COMPUTE_ERROR_ON(detection_window_strides->num_values() != multi_hog->num_models());
63 
64  const size_t width = input->info()->dimension(Window::DimX);
65  const size_t height = input->info()->dimension(Window::DimY);
66  const TensorShape &shape_img = input->info()->tensor_shape();
67  const size_t num_models = multi_hog->num_models();
68  PhaseType phase_type = multi_hog->model(0)->info()->phase_type();
69 
70  size_t prev_num_bins = multi_hog->model(0)->info()->num_bins();
71  Size2D prev_cell_size = multi_hog->model(0)->info()->cell_size();
72  Size2D prev_block_size = multi_hog->model(0)->info()->block_size();
73  Size2D prev_block_stride = multi_hog->model(0)->info()->block_stride();
74 
75  /* Check if CLHOGOrientationBinningKernel and CLHOGBlockNormalizationKernel kernels can be skipped for a specific HOG data-object
76  *
77  * 1) CLHOGOrientationBinningKernel and CLHOGBlockNormalizationKernel are skipped if the cell size and the number of bins don't change.
78  * Since "multi_hog" is sorted,it is enough to check the HOG descriptors at level "ith" and level "(i-1)th
79  * 2) CLHOGBlockNormalizationKernel is skipped if the cell size, the number of bins and block size do not change.
80  * Since "multi_hog" is sorted,it is enough to check the HOG descriptors at level "ith" and level "(i-1)th
81  *
82  * @note Since the orientation binning and block normalization kernels can be skipped, we need to keep track of the input to process for each kernel
83  * with "input_orient_bin", "input_hog_detect" and "input_block_norm"
84  */
85  std::vector<size_t> input_orient_bin;
86  std::vector<size_t> input_hog_detect;
87  std::vector<std::pair<size_t, size_t>> input_block_norm;
88 
89  input_orient_bin.push_back(0);
90  input_hog_detect.push_back(0);
91  input_block_norm.emplace_back(0, 0);
92 
93  for(size_t i = 1; i < num_models; ++i)
94  {
95  size_t cur_num_bins = multi_hog->model(i)->info()->num_bins();
96  Size2D cur_cell_size = multi_hog->model(i)->info()->cell_size();
97  Size2D cur_block_size = multi_hog->model(i)->info()->block_size();
98  Size2D cur_block_stride = multi_hog->model(i)->info()->block_stride();
99 
100  if((cur_num_bins != prev_num_bins) || (cur_cell_size.width != prev_cell_size.width) || (cur_cell_size.height != prev_cell_size.height))
101  {
102  prev_num_bins = cur_num_bins;
103  prev_cell_size = cur_cell_size;
104  prev_block_size = cur_block_size;
105  prev_block_stride = cur_block_stride;
106 
107  // Compute orientation binning and block normalization kernels. Update input to process
108  input_orient_bin.push_back(i);
109  input_block_norm.emplace_back(i, input_orient_bin.size() - 1);
110  }
111  else if((cur_block_size.width != prev_block_size.width) || (cur_block_size.height != prev_block_size.height) || (cur_block_stride.width != prev_block_stride.width)
112  || (cur_block_stride.height != prev_block_stride.height))
113  {
114  prev_block_size = cur_block_size;
115  prev_block_stride = cur_block_stride;
116 
117  // Compute block normalization kernel. Update input to process
118  input_block_norm.emplace_back(i, input_orient_bin.size() - 1);
119  }
120 
121  // Update input to process for hog detector kernel
122  input_hog_detect.push_back(input_block_norm.size() - 1);
123  }
124 
125  _detection_windows = detection_windows;
126  _non_maxima_suppression = non_maxima_suppression;
127  _num_orient_bin_kernel = input_orient_bin.size(); // Number of CLHOGOrientationBinningKernel kernels to compute
128  _num_block_norm_kernel = input_block_norm.size(); // Number of CLHOGBlockNormalizationKernel kernels to compute
129  _num_hog_detect_kernel = input_hog_detect.size(); // Number of CLHOGDetector functions to compute
130 
131  _orient_bin_kernel.resize(_num_orient_bin_kernel);
132  _block_norm_kernel.resize(_num_block_norm_kernel);
133  _hog_detect_kernel.resize(_num_hog_detect_kernel);
134  _hog_space.resize(_num_orient_bin_kernel);
135  _hog_norm_space.resize(_num_block_norm_kernel);
136 
137  // Allocate tensors for magnitude and phase
138  TensorInfo info_mag(shape_img, Format::S16);
139  _mag.allocator()->init(info_mag);
140 
141  TensorInfo info_phase(shape_img, Format::U8);
142  _phase.allocator()->init(info_phase);
143 
144  // Manage intermediate buffers
145  _memory_group.manage(&_mag);
146  _memory_group.manage(&_phase);
147 
148  // Initialise gradient kernel
149  _gradient_kernel.configure(input, &_mag, &_phase, phase_type, border_mode, constant_border_value);
150 
151  // Configure NETensor for the HOG space and orientation binning kernel
152  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
153  {
154  const size_t idx_multi_hog = input_orient_bin[i];
155 
156  // Get the corresponding cell size and number of bins
157  const Size2D &cell = multi_hog->model(idx_multi_hog)->info()->cell_size();
158  const size_t num_bins = multi_hog->model(idx_multi_hog)->info()->num_bins();
159 
160  // Calculate number of cells along the x and y directions for the hog_space
161  const size_t num_cells_x = width / cell.width;
162  const size_t num_cells_y = height / cell.height;
163 
164  // TensorShape of hog space
165  TensorShape shape_hog_space = input->info()->tensor_shape();
166  shape_hog_space.set(Window::DimX, num_cells_x);
167  shape_hog_space.set(Window::DimY, num_cells_y);
168 
169  // Allocate HOG space
170  TensorInfo info_space(shape_hog_space, num_bins, DataType::F32);
171  _hog_space[i].allocator()->init(info_space);
172 
173  // Manage intermediate buffers
174  _memory_group.manage(&_hog_space[i]);
175 
176  // Initialise orientation binning kernel
177  _orient_bin_kernel[i].configure(&_mag, &_phase, &_hog_space[i], multi_hog->model(idx_multi_hog)->info());
178  }
179 
180  // Allocate intermediate tensors
181  _mag.allocator()->allocate();
182  _phase.allocator()->allocate();
183 
184  // Configure CLTensor for the normalized HOG space and block normalization kernel
185  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
186  {
187  const size_t idx_multi_hog = input_block_norm[i].first;
188  const size_t idx_orient_bin = input_block_norm[i].second;
189 
190  // Allocate normalized HOG space
191  TensorInfo tensor_info(*(multi_hog->model(idx_multi_hog)->info()), width, height);
192  _hog_norm_space[i].allocator()->init(tensor_info);
193 
194  // Manage intermediate buffers
195  _memory_group.manage(&_hog_norm_space[i]);
196 
197  // Initialize block normalization kernel
198  _block_norm_kernel[i].configure(&_hog_space[idx_orient_bin], &_hog_norm_space[i], multi_hog->model(idx_multi_hog)->info());
199  }
200 
201  // Allocate intermediate tensors
202  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
203  {
204  _hog_space[i].allocator()->allocate();
205  }
206 
207  detection_window_strides->map(CLScheduler::get().queue(), true);
208 
209  // Configure HOG detector kernel
210  for(size_t i = 0; i < _num_hog_detect_kernel; ++i)
211  {
212  const size_t idx_block_norm = input_hog_detect[i];
213 
214  _hog_detect_kernel[i].configure(&_hog_norm_space[idx_block_norm], multi_hog->cl_model(i), detection_windows, detection_window_strides->at(i), threshold, i);
215  }
216 
217  detection_window_strides->unmap(CLScheduler::get().queue());
218 
219  // Configure non maxima suppression kernel
220  _non_maxima_kernel.configure(_detection_windows, min_distance);
221 
222  // Allocate intermediate tensors
223  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
224  {
225  _hog_norm_space[i].allocator()->allocate();
226  }
227 }
size_t num_bins() const
The number of histogram bins for each cell.
Definition: HOGInfo.cpp:111
Shape of a tensor.
Definition: TensorShape.h:39
virtual size_t dimension(size_t index) const =0
Return the size of the requested dimension.
const Size2D & cell_size() const
The cell size in pixels.
Definition: HOGInfo.cpp:91
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:41
PhaseType
Phase calculation type.
Definition: Types.h:396
void configure(ICLTensor *input, ICLTensor *output_magnitude, ICLTensor *output_phase, PhaseType phase_type, BorderMode border_mode, uint8_t constant_border_value=0)
Initialise the function's source, destinations, phase type and border mode.
1 channel, 1 U8 per channel
1 channel, 1 F32 per channel
void map(cl::CommandQueue &q, bool blocking=true)
Enqueue a map operation of the allocated buffer on the given queue.
Definition: ICLArray.h:72
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:337
CLTensorAllocator * allocator()
Return a pointer to the tensor's allocator.
Definition: CLTensor.cpp:55
PhaseType phase_type() const
The type of PhaseType.
Definition: HOGInfo.cpp:126
virtual ICLHOG * cl_model(size_t index)=0
Return a pointer to the requested OpenCL HOG model.
const Size2D & block_stride() const
The block stride in pixels.
Definition: HOGInfo.cpp:106
void init(const TensorInfo &input, size_t alignment=0)
Initialize a tensor based on the passed TensorInfo.
size_t height
Height of the image region or rectangle.
Definition: Size2D.h:93
#define ARM_COMPUTE_ERROR_ON_INVALID_MULTI_HOG(m)
Definition: Validate.h:924
virtual size_t num_models() const =0
The number of HOG models stored.
IHOG * model(size_t index) override
Return a pointer to the requested HOG model.
Definition: ICLMultiHOG.cpp:30
static constexpr size_t DimX
Alias for dimension 0 also known as X dimension.
Definition: Window.h:43
size_t num_values() const
Number of values currently stored in the array.
Definition: IArray.h:68
void manage(TensorType *obj)
Sets a object to be managed by the given memory group.
virtual const TensorShape & tensor_shape() const =0
Size for each dimension of the tensor.
const Size2D & block_size() const
The block size in pixels.
Definition: HOGInfo.cpp:96
void configure(IDetectionWindowArray *input_output, float min_distance)
Initialise the kernel's input, output and the euclidean minimum distance.
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
1 channel, 1 S16 per channel
#define ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN(t, c,...)
Definition: Validate.h:789
void unmap(cl::CommandQueue &q)
Enqueue an unmap operation of the allocated and mapped buffer on the given queue.
Definition: ICLArray.h:83
void allocate() override
Allocate size specified by TensorInfo of OpenCL memory.
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
size_t width
Width of the image region or rectangle.
Definition: Size2D.h:92
SimpleTensor< T > non_maxima_suppression(const SimpleTensor< T > &src, BorderMode border_mode, T constant_border_value)
Class for specifying the size of an image or rectangle.
Definition: Size2D.h:34
TensorShape & set(size_t dimension, size_t value, bool apply_dim_correction=true)
Accessor to set the value of one of the dimensions.
Definition: TensorShape.h:78
Store the tensor's metadata.
Definition: TensorInfo.h:45
virtual const HOGInfo * info() const =0
Interface to be implemented by the child class to return the HOG's metadata.
SimpleTensor< T > threshold(const SimpleTensor< T > &src, T threshold, T false_value, T true_value, ThresholdType type, T upper)
Definition: Threshold.cpp:35
virtual T & at(size_t index) const
Reference to the element of the array located at the given index.
Definition: IArray.h:117

References CLTensorAllocator::allocate(), CLTensor::allocator(), ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN, ARM_COMPUTE_ERROR_ON_INVALID_MULTI_HOG, IArray< T >::at(), HOGInfo::block_size(), HOGInfo::block_stride(), arm_compute::test::validation::border_mode, HOGInfo::cell_size(), ICLMultiHOG::cl_model(), CLHOGGradient::configure(), CPPDetectionWindowNonMaximaSuppressionKernel::configure(), ITensorInfo::dimension(), Window::DimX, Window::DimY, arm_compute::F32, CLScheduler::get(), Size2D::height, IHOG::info(), ITensor::info(), ITensorAllocator::init(), MemoryGroupBase< TensorType >::manage(), ICLArray< T >::map(), ICLMultiHOG::model(), arm_compute::test::validation::reference::non_maxima_suppression(), HOGInfo::num_bins(), IMultiHOG::num_models(), IArray< T >::num_values(), HOGInfo::phase_type(), arm_compute::S16, TensorShape::set(), ITensorInfo::tensor_shape(), arm_compute::test::validation::reference::threshold(), arm_compute::U8, ICLArray< T >::unmap(), and Size2D::width.

◆ operator=()

CLHOGMultiDetection& operator= ( const CLHOGMultiDetection )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For NEON kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 229 of file CLHOGMultiDetection.cpp.

230 {
231  ARM_COMPUTE_ERROR_ON_MSG(_detection_windows == nullptr, "Unconfigured function");
232 
233  MemoryGroupResourceScope scope_mg(_memory_group);
234 
235  // Reset detection window
236  _detection_windows->clear();
237 
238  // Run gradient
239  _gradient_kernel.run();
240 
241  // Run orientation binning kernel
242  for(size_t i = 0; i < _num_orient_bin_kernel; ++i)
243  {
244  CLScheduler::get().enqueue(_orient_bin_kernel[i], false);
245  }
246 
247  // Run block normalization kernel
248  for(size_t i = 0; i < _num_block_norm_kernel; ++i)
249  {
250  CLScheduler::get().enqueue(_block_norm_kernel[i], false);
251  }
252 
253  // Run HOG detector kernel
254  for(size_t i = 0; i < _num_hog_detect_kernel; ++i)
255  {
256  _hog_detect_kernel[i].run();
257  }
258 
259  // Run non-maxima suppression kernel if enabled
260  if(_non_maxima_suppression)
261  {
262  // Map detection windows array before computing non maxima suppression
263  _detection_windows->map(CLScheduler::get().queue(), true);
264  Scheduler::get().schedule(&_non_maxima_kernel, Window::DimY);
265  _detection_windows->unmap(CLScheduler::get().queue());
266  }
267 }
void clear()
Clear all the points from the array.
Definition: IArray.h:91
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:41
void map(cl::CommandQueue &q, bool blocking=true)
Enqueue a map operation of the allocated buffer on the given queue.
Definition: ICLArray.h:72
void enqueue(ICLKernel &kernel, bool flush=true)
Schedule the execution of the passed kernel if possible.
Definition: CLScheduler.cpp:95
void unmap(cl::CommandQueue &q)
Enqueue an unmap operation of the allocated and mapped buffer on the given queue.
Definition: ICLArray.h:83
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
Memory group resources scope handling class.
Definition: IMemoryGroup.h:46
virtual void schedule(ICPPKernel *kernel, const Hints &hints)=0
Runs the kernel in the same thread as the caller synchronously.
void run() override
Run the kernels contained in the function.
static IScheduler & get()
Access the scheduler singleton.
Definition: Scheduler.cpp:96
#define ARM_COMPUTE_ERROR_ON_MSG(cond,...)
Definition: Error.h:328

References ARM_COMPUTE_ERROR_ON_MSG, IArray< T >::clear(), Window::DimY, CLScheduler::enqueue(), Scheduler::get(), CLScheduler::get(), ICLArray< T >::map(), CLHOGGradient::run(), IScheduler::schedule(), and ICLArray< T >::unmap().


The documentation for this class was generated from the following files: