Compute Library
 21.02
CLHarrisCorners Class Reference

Basic function to execute harris corners detection. More...

#include <CLHarrisCorners.h>

Collaboration diagram for CLHarrisCorners:
[legend]

Public Member Functions

 CLHarrisCorners (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Constructor. More...
 
 CLHarrisCorners (const CLHarrisCorners &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
const CLHarrisCornersoperator= (const CLHarrisCorners &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 ~CLHarrisCorners ()
 Default destructor. More...
 
void configure (ICLImage *input, float threshold, float min_dist, float sensitivity, int32_t gradient_size, int32_t block_size, ICLKeyPointArray *corners, BorderMode border_mode, uint8_t constant_border_value=0, bool use_fp16=false)
 Initialize the function's source, destination, conv and border_mode. More...
 
void configure (const CLCompileContext &compile_context, ICLImage *input, float threshold, float min_dist, float sensitivity, int32_t gradient_size, int32_t block_size, ICLKeyPointArray *corners, BorderMode border_mode, uint8_t constant_border_value=0, bool use_fp16=false)
 Initialize the function's source, destination, conv and border_mode. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Detailed Description

Basic function to execute harris corners detection.

This function calls the following CL and Neon kernels and functions:

Note
Requires CPU support for the kernels: CPPCornerCandidatesKernel and CPPSortEuclideanDistanceKernel.
  1. CLSobel3x3 (if gradient_size == 3) or
    CLSobel5x5 (if gradient_size == 5) or
    CLSobel7x7 (if gradient_size == 7)
  2. CLFillBorderKernel
  3. CLHarrisScoreKernel
  4. CLNonMaximaSuppression3x3
  5. CPPCornerCandidatesKernel
  6. CPPSortEuclideanDistanceKernel
Deprecated:
This function is deprecated and is intended to be removed in 21.05 release

Definition at line 63 of file CLHarrisCorners.h.

Constructor & Destructor Documentation

◆ CLHarrisCorners() [1/2]

CLHarrisCorners ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Constructor.

Definition at line 46 of file CLHarrisCorners.cpp.

References CLHarrisCorners::~CLHarrisCorners().

47  : _memory_group(std::move(memory_manager)),
48  _sobel(nullptr),
49  _harris_score(std::make_unique<CLHarrisScoreKernel>()),
50  _non_max_suppr(),
51  _candidates(),
52  _sort_euclidean(),
53  _border_gx(std::make_unique<CLFillBorderKernel>()),
54  _border_gy(std::make_unique<CLFillBorderKernel>()),
55  _gx(),
56  _gy(),
57  _score(),
58  _nonmax(),
59  _corners_list(),
60  _num_corner_candidates(0),
61  _corners(nullptr)
62 {
63 }

◆ CLHarrisCorners() [2/2]

CLHarrisCorners ( const CLHarrisCorners )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ ~CLHarrisCorners()

~CLHarrisCorners ( )
default

Default destructor.

Referenced by CLHarrisCorners::CLHarrisCorners().

Member Function Documentation

◆ configure() [1/2]

void configure ( ICLImage input,
float  threshold,
float  min_dist,
float  sensitivity,
int32_t  gradient_size,
int32_t  block_size,
ICLKeyPointArray corners,
BorderMode  border_mode,
uint8_t  constant_border_value = 0,
bool  use_fp16 = false 
)

Initialize the function's source, destination, conv and border_mode.

Parameters
[in,out]inputSource image. Data types supported: U8. (Written to only for border_mode != UNDEFINED)
[in]thresholdMinimum threshold with which to eliminate Harris Corner scores (computed using the normalized Sobel kernel).
[in]min_distRadial Euclidean distance for the euclidean distance stage.
[in]sensitivitySensitivity threshold k from the Harris-Stephens equation
[in]gradient_sizeThe gradient window size to use on the input. The implementation supports 3, 5, and 7
[in]block_sizeThe block window size used to compute the Harris Corner score. The implementation supports 3, 5, and 7.
[out]cornersArray of keypoints to store the results.
[in]border_modeBorder mode to use
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT.
[in]use_fp16(Optional) If true the FP16 kernels will be used. If false F32 kernels are used.

Definition at line 67 of file CLHarrisCorners.cpp.

References CLKernelLibrary::get().

70 {
71  configure(CLKernelLibrary::get().get_compile_context(), input, threshold, min_dist, sensitivity, gradient_size, block_size, corners, border_mode, constant_border_value, use_fp16);
72 }
static CLKernelLibrary & get()
Access the KernelLibrary singleton.
void configure(ICLImage *input, float threshold, float min_dist, float sensitivity, int32_t gradient_size, int32_t block_size, ICLKeyPointArray *corners, BorderMode border_mode, uint8_t constant_border_value=0, bool use_fp16=false)
Initialize the function&#39;s source, destination, conv and border_mode.
SimpleTensor< T > threshold(const SimpleTensor< T > &src, T threshold, T false_value, T true_value, ThresholdType type, T upper)
Definition: Threshold.cpp:35

◆ configure() [2/2]

void configure ( const CLCompileContext compile_context,
ICLImage input,
float  threshold,
float  min_dist,
float  sensitivity,
int32_t  gradient_size,
int32_t  block_size,
ICLKeyPointArray corners,
BorderMode  border_mode,
uint8_t  constant_border_value = 0,
bool  use_fp16 = false 
)

Initialize the function's source, destination, conv and border_mode.

Parameters
[in]compile_contextThe compile context to be used.
[in,out]inputSource image. Data types supported: U8. (Written to only for border_mode != UNDEFINED)
[in]thresholdMinimum threshold with which to eliminate Harris Corner scores (computed using the normalized Sobel kernel).
[in]min_distRadial Euclidean distance for the euclidean distance stage.
[in]sensitivitySensitivity threshold k from the Harris-Stephens equation
[in]gradient_sizeThe gradient window size to use on the input. The implementation supports 3, 5, and 7
[in]block_sizeThe block window size used to compute the Harris Corner score. The implementation supports 3, 5, and 7.
[out]cornersArray of keypoints to store the results.
[in]border_modeBorder mode to use
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT.
[in]use_fp16(Optional) If true the FP16 kernels will be used. If false F32 kernels are used.

Definition at line 74 of file CLHarrisCorners.cpp.

References CLTensorAllocator::allocate(), CLTensor::allocator(), ARM_COMPUTE_ERROR, ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN, ARM_COMPUTE_ERROR_ON_TENSOR_NOT_2D, ARM_COMPUTE_UNUSED, CLNonMaximaSuppression3x3::configure(), CPPSortEuclideanDistanceKernel::configure(), CPPCornerCandidatesKernel::configure(), dt, arm_compute::F32, ITensor::info(), ITensorAllocator::init(), MemoryGroup::manage(), arm_compute::S16, arm_compute::S32, arm_compute::test::validation::shape, ITensorInfo::tensor_shape(), arm_compute::U8, arm_compute::UNDEFINED, Dimensions< T >::x(), and Dimensions< T >::y().

77 {
78  ARM_COMPUTE_UNUSED(use_fp16); //TODO(COMPMID-772): Add half float support
81  ARM_COMPUTE_ERROR_ON(!(block_size == 3 || block_size == 5 || block_size == 7));
82  ARM_COMPUTE_ERROR_ON(nullptr == corners);
83 
84  _corners = corners;
85 
86  const TensorShape shape = input->info()->tensor_shape();
87  const DataType dt = (gradient_size < 7) ? DataType::S16 : DataType::S32;
88  TensorInfo tensor_info(shape, 1, dt);
89 
90  _gx.allocator()->init(tensor_info);
91  _gy.allocator()->init(tensor_info);
92 
93  TensorInfo info_f32(shape, 1, DataType::F32);
94  _score.allocator()->init(info_f32);
95  _nonmax.allocator()->init(info_f32);
96 
97  _corners_list.resize(shape.x() * shape.y());
98 
99  // Manage intermediate buffers
100  _memory_group.manage(&_gx);
101  _memory_group.manage(&_gy);
102 
103  /* Set/init Sobel kernel accordingly with gradient_size */
104  switch(gradient_size)
105  {
106  case 3:
107  {
108  auto k = std::make_unique<CLSobel3x3>();
109  k->configure(compile_context, input, &_gx, &_gy, border_mode, constant_border_value);
110  _sobel = std::move(k);
111  break;
112  }
113  case 5:
114  {
115  auto k = std::make_unique<CLSobel5x5>();
116  k->configure(compile_context, input, &_gx, &_gy, border_mode, constant_border_value);
117  _sobel = std::move(k);
118  break;
119  }
120  case 7:
121  {
122  auto k = std::make_unique<CLSobel7x7>();
123  k->configure(compile_context, input, &_gx, &_gy, border_mode, constant_border_value);
124  _sobel = std::move(k);
125  break;
126  }
127  default:
128  ARM_COMPUTE_ERROR("Gradient size not implemented");
129  }
130 
131  // Normalization factor
132  const float norm_factor = 1.0f / (255.0f * pow(4.0f, gradient_size / 2) * block_size);
133  const float pow4_normalization_factor = pow(norm_factor, 4);
134 
135  // Manage intermediate buffers
136  _memory_group.manage(&_score);
137 
138  // Set/init Harris Score kernel accordingly with block_size
139  _harris_score->configure(compile_context, &_gx, &_gy, &_score, block_size, pow4_normalization_factor, threshold, sensitivity, border_mode == BorderMode::UNDEFINED);
140 
141  // Configure border filling using harris score kernel's block size
142  _border_gx->configure(compile_context, &_gx, _harris_score->border_size(), border_mode, PixelValue(constant_border_value));
143  _border_gy->configure(compile_context, &_gy, _harris_score->border_size(), border_mode, PixelValue(constant_border_value));
144 
145  // Allocate intermediate buffers
146  _gx.allocator()->allocate();
147  _gy.allocator()->allocate();
148 
149  // Manage intermediate buffers
150  _memory_group.manage(&_nonmax);
151 
152  // Init non-maxima suppression function
153  _non_max_suppr.configure(compile_context, &_score, &_nonmax, border_mode);
154 
155  // Allocate intermediate buffers
156  _score.allocator()->allocate();
157 
158  // Init corner candidates kernel
159  _candidates.configure(&_nonmax, _corners_list.data(), &_num_corner_candidates);
160 
161  // Allocate intermediate buffers
162  _nonmax.allocator()->allocate();
163 
164  // Init euclidean distance
165  _sort_euclidean.configure(_corners_list.data(), _corners, &_num_corner_candidates, min_dist);
166 }
Class describing the value of a pixel for any image format.
Definition: PixelValue.h:34
Shape of a tensor.
Definition: TensorShape.h:39
#define ARM_COMPUTE_ERROR_ON_TENSOR_NOT_2D(t)
Definition: Validate.h:856
#define ARM_COMPUTE_ERROR(msg)
Print the given message then throw an std::runtime_error.
Definition: Error.h:352
1 channel, 1 U8 per channel
1 channel, 1 F32 per channel
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
CLTensorAllocator * allocator()
Return a pointer to the tensor&#39;s allocator.
Definition: CLTensor.cpp:61
void init(const TensorInfo &input, size_t alignment=0)
Initialize a tensor based on the passed TensorInfo.
DataType dt
1 channel, 1 S32 per channel
void manage(IMemoryManageable *obj) override
Sets a object to be managed by the given memory group.
Definition: MemoryGroup.h:79
T x() const
Alias to access the size of the first dimension.
Definition: Dimensions.h:87
void configure(InternalKeypoint *in_out, IKeyPointArray *output, const int32_t *num_corner_candidates, float min_distance)
Initialise the kernel&#39;s source, destination and border mode.
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:152
void configure(ICLTensor *input, ICLTensor *output, BorderMode border_mode)
Initialise the function&#39;s source, destinations and border mode.
virtual const TensorShape & tensor_shape() const =0
Size for each dimension of the tensor.
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor&#39;s metadata.
1 channel, 1 S16 per channel
#define ARM_COMPUTE_ERROR_ON_DATA_TYPE_CHANNEL_NOT_IN(t, c,...)
Definition: Validate.h:790
void allocate() override
Allocate size specified by TensorInfo of OpenCL memory.
Borders are left undefined.
Store the tensor&#39;s metadata.
Definition: TensorInfo.h:45
T y() const
Alias to access the size of the second dimension.
Definition: Dimensions.h:92
DataType
Available data types.
Definition: Types.h:77
SimpleTensor< T > threshold(const SimpleTensor< T > &src, T threshold, T false_value, T true_value, ThresholdType type, T upper)
Definition: Threshold.cpp:35
void configure(const IImage *input, InternalKeypoint *output, int32_t *num_corner_candidates)
Setup the kernel parameters.

◆ operator=()

const CLHarrisCorners& operator= ( const CLHarrisCorners )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For Neon kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 168 of file CLHarrisCorners.cpp.

References ARM_COMPUTE_ERROR_ON_MSG, Window::DimY, CLScheduler::enqueue(), Scheduler::get(), CLScheduler::get(), ICLArray< T >::map(), CLTensor::map(), ICLSimpleFunction::run(), IScheduler::schedule(), CLTensor::unmap(), and ICLArray< T >::unmap().

169 {
170  ARM_COMPUTE_ERROR_ON_MSG(_sobel == nullptr, "Unconfigured function");
171 
172  MemoryGroupResourceScope scope_mg(_memory_group);
173 
174  // Init to 0 number of corner candidates
175  _num_corner_candidates = 0;
176 
177  // Run Sobel kernel
178  _sobel->run();
179 
180  // Fill border before harris score kernel
181  CLScheduler::get().enqueue(*_border_gx, false);
182  CLScheduler::get().enqueue(*_border_gy, false);
183 
184  // Run harris score kernel
185  CLScheduler::get().enqueue(*_harris_score, false);
186 
187  // Run non-maxima suppression
188  _non_max_suppr.run();
189 
190  // Run corner candidate kernel
191  _nonmax.map(true);
192  Scheduler::get().schedule(&_candidates, Window::DimY);
193  _nonmax.unmap();
194 
195  _corners->map(CLScheduler::get().queue(), true);
196  Scheduler::get().schedule(&_sort_euclidean, Window::DimY);
197  _corners->unmap(CLScheduler::get().queue());
198 }
static CLScheduler & get()
Access the scheduler singleton.
void map(cl::CommandQueue &q, bool blocking=true)
Enqueue a map operation of the allocated buffer on the given queue.
Definition: ICLArray.h:72
void map(bool blocking=true)
Enqueue a map operation of the allocated buffer.
Definition: CLTensor.cpp:66
void run() override final
Run the kernels contained in the function.
#define ARM_COMPUTE_ERROR_ON_MSG(cond, msg)
Definition: Error.h:456
void enqueue(ICLKernel &kernel, bool flush=true)
Schedule the execution of the passed kernel if possible.
void unmap(cl::CommandQueue &q)
Enqueue an unmap operation of the allocated and mapped buffer on the given queue. ...
Definition: ICLArray.h:83
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
Memory group resources scope handling class.
Definition: IMemoryGroup.h:82
virtual void schedule(ICPPKernel *kernel, const Hints &hints)=0
Runs the kernel in the same thread as the caller synchronously.
void unmap()
Enqueue an unmap operation of the allocated and mapped buffer.
Definition: CLTensor.cpp:71
static IScheduler & get()
Access the scheduler singleton.
Definition: Scheduler.cpp:94

The documentation for this class was generated from the following files: