Compute Library
CLOpticalFlow Class Reference

Basic function to execute optical flow. More...

#include <CLOpticalFlow.h>

Collaboration diagram for CLOpticalFlow:

Public Member Functions

 CLOpticalFlow (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 CLOpticalFlow (const CLOpticalFlow &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
CLOpticalFlowoperator= (const CLOpticalFlow &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 CLOpticalFlow (CLOpticalFlow &&)=default
 Allow instances of this class to be moved. More...
CLOpticalFlowoperator= (CLOpticalFlow &&)=default
 Allow instances of this class to be moved. More...
void configure (const CLPyramid *old_pyramid, const CLPyramid *new_pyramid, const ICLKeyPointArray *old_points, const ICLKeyPointArray *new_points_estimates, ICLKeyPointArray *new_points, Termination termination, float epsilon, size_t num_iterations, size_t window_dimension, bool use_initial_estimate, BorderMode border_mode, uint8_t constant_border_value=0)
 Initialise the function input and output. More...
void run () override
 Run the kernels contained in the function. More...
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
virtual void prepare ()
 Prepare the function for executing. More...

Detailed Description

Basic function to execute optical flow.

This function calls the following OpenCL kernels and functions:

  1. CLScharr3x3
  2. CLLKTrackerInitKernel
  3. CLLKTrackerStage0Kernel
  4. CLLKTrackerStage1Kernel
  5. CLLKTrackerFinalizeKernel

Definition at line 61 of file CLOpticalFlow.h.

Constructor & Destructor Documentation

◆ CLOpticalFlow() [1/3]

CLOpticalFlow ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Definition at line 40 of file CLOpticalFlow.cpp.

41  : _memory_group(std::move(memory_manager)),
42  _tracker_init_kernel(),
43  _tracker_stage0_kernel(),
44  _tracker_stage1_kernel(),
45  _tracker_finalize_kernel(),
46  _func_scharr(),
47  _scharr_gx(),
48  _scharr_gy(),
49  _old_points(nullptr),
50  _new_points_estimates(nullptr),
51  _new_points(nullptr),
52  _old_points_internal(),
53  _new_points_internal(),
54  _coefficient_table(),
55  _old_values(),
56  _num_levels(0)
57 {
58 }

◆ CLOpticalFlow() [2/3]

CLOpticalFlow ( const CLOpticalFlow )

Prevent instances of this class from being copied (As this class contains pointers)

◆ CLOpticalFlow() [3/3]

CLOpticalFlow ( CLOpticalFlow &&  )

Allow instances of this class to be moved.

Member Function Documentation

◆ configure()

void configure ( const CLPyramid old_pyramid,
const CLPyramid new_pyramid,
const ICLKeyPointArray old_points,
const ICLKeyPointArray new_points_estimates,
ICLKeyPointArray new_points,
Termination  termination,
float  epsilon,
size_t  num_iterations,
size_t  window_dimension,
bool  use_initial_estimate,
BorderMode  border_mode,
uint8_t  constant_border_value = 0 

Initialise the function input and output.

[in]old_pyramidPointer to the pyramid for the old tensor. Data types supported U8
[in]new_pyramidPointer to the pyramid for the new tensor. Data types supported U8
[in]old_pointsPointer to the IKeyPointArray storing old key points
[in]new_points_estimatesPointer to the IKeyPointArray storing new estimates key points
[out]new_pointsPointer to the IKeyPointArray storing new key points
[in]terminationThe criteria to terminate the search of each keypoint.
[in]epsilonThe error for terminating the algorithm
[in]num_iterationsThe maximum number of iterations before terminate the alogrithm
[in]window_dimensionThe size of the window on which to perform the algorithm
[in]use_initial_estimateThe flag to indicate whether the initial estimated position should be used
[in]border_modeThe border mode applied at scharr kernel stage
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT

Definition at line 60 of file CLOpticalFlow.cpp.

64 {
65  ARM_COMPUTE_ERROR_ON(nullptr == old_pyramid);
66  ARM_COMPUTE_ERROR_ON(nullptr == new_pyramid);
67  ARM_COMPUTE_ERROR_ON(nullptr == old_points);
68  ARM_COMPUTE_ERROR_ON(nullptr == new_points_estimates);
69  ARM_COMPUTE_ERROR_ON(nullptr == new_points);
70  ARM_COMPUTE_ERROR_ON(old_pyramid->info()->num_levels() != new_pyramid->info()->num_levels());
71  ARM_COMPUTE_ERROR_ON(0 == old_pyramid->info()->num_levels());
72  ARM_COMPUTE_ERROR_ON(old_pyramid->info()->width() != new_pyramid->info()->width());
73  ARM_COMPUTE_ERROR_ON(old_pyramid->info()->height() != new_pyramid->info()->height());
74  ARM_COMPUTE_ERROR_ON(use_initial_estimate && old_points->num_values() != new_points_estimates->num_values());
76  // Set member variables
77  _old_points = old_points;
78  _new_points_estimates = new_points_estimates;
79  _new_points = new_points;
80  _num_levels = old_pyramid->info()->num_levels();
82  const float pyr_scale = old_pyramid->info()->scale();
83  const int list_length = old_points->num_values();
84  const int old_values_list_length = list_length * window_dimension * window_dimension;
86  // Create kernels and tensors
87  _tracker_init_kernel.resize(_num_levels);
88  _tracker_stage0_kernel.resize(_num_levels);
89  _tracker_stage1_kernel.resize(_num_levels);
90  _func_scharr.resize(_num_levels);
91  _scharr_gx.resize(_num_levels);
92  _scharr_gy.resize(_num_levels);
94  // Create internal keypoint arrays
95  _old_points_internal = arm_compute::support::cpp14::make_unique<CLLKInternalKeypointArray>(list_length);
96  _old_points_internal->resize(list_length);
97  _new_points_internal = arm_compute::support::cpp14::make_unique<CLLKInternalKeypointArray>(list_length);
98  _new_points_internal->resize(list_length);
99  _coefficient_table = arm_compute::support::cpp14::make_unique<CLCoefficientTableArray>(list_length);
100  _coefficient_table->resize(list_length);
101  _old_values = arm_compute::support::cpp14::make_unique<CLOldValueArray>(old_values_list_length);
102  _old_values->resize(old_values_list_length);
103  _new_points->resize(list_length);
105  for(size_t i = 0; i < _num_levels; ++i)
106  {
107  // Get images from the ith level of old and right pyramid
108  ICLImage *old_ith_input = old_pyramid->get_pyramid_level(i);
109  ICLImage *new_ith_input = new_pyramid->get_pyramid_level(i);
111  // Get width and height of images
112  const unsigned int width_ith = old_ith_input->info()->dimension(0);
113  const unsigned int height_ith = new_ith_input->info()->dimension(1);
115  // Initialize Scharr tensors
116  TensorInfo tensor_info(TensorShape(width_ith, height_ith), 1, DataType::S16);
117  _scharr_gx[i].allocator()->init(tensor_info);
118  _scharr_gy[i].allocator()->init(tensor_info);
120  // Manage intermediate buffers
121  _memory_group.manage(&_scharr_gx[i]);
122  _memory_group.manage(&_scharr_gy[i]);
124  // Init Scharr kernel
125  _func_scharr[i].configure(old_ith_input, &_scharr_gx[i], &_scharr_gy[i], border_mode, constant_border_value);
127  // Init Lucas-Kanade init kernel
128  _tracker_init_kernel[i].configure(old_points, new_points_estimates, _old_points_internal.get(), _new_points_internal.get(), use_initial_estimate, i, _num_levels, pyr_scale);
130  // Init Lucas-Kanade stage0 kernel
131  _tracker_stage0_kernel[i].configure(old_ith_input, &_scharr_gx[i], &_scharr_gy[i],
132  _old_points_internal.get(), _new_points_internal.get(), _coefficient_table.get(), _old_values.get(),
133  window_dimension, i);
135  // Init Lucas-Kanade stage1 kernel
136  _tracker_stage1_kernel[i].configure(new_ith_input, _new_points_internal.get(), _coefficient_table.get(), _old_values.get(),
137  termination, epsilon, num_iterations, window_dimension, i);
139  // Allocate intermediate buffers
140  _scharr_gx[i].allocator()->allocate();
141  _scharr_gy[i].allocator()->allocate();
142  }
144  // Finalize Lucas-Kanade
145  _tracker_finalize_kernel.configure(_new_points_internal.get(), new_points);
146 }
Shape of a tensor.
Definition: TensorShape.h:39
virtual size_t dimension(size_t index) const =0
Return the size of the requested dimension.
const PyramidInfo * info() const override
Interface to be implemented by the child class to return the Pyramid's metadata.
Definition: CLPyramid.cpp:118
void resize(size_t num)
Resizes the array to contain "num" elements.
Definition: IArray.h:128
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
void configure(ICLLKInternalKeypointArray *new_points_internal, ICLKeyPointArray *new_points)
Initialise the kernel input and output.
void manage(IMemoryManageable *obj) override
Sets a object to be managed by the given memory group.
Definition: MemoryGroup.h:79
size_t num_values() const
Number of values currently stored in the array.
Definition: IArray.h:68
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
float scale() const
Return the scale factor of the pyramid.
1 channel, 1 S16 per channel
Interface for OpenCL tensor.
Definition: ICLTensor.h:42
CLTensor * get_pyramid_level(size_t index) const override
Retrieves a level of the pyramid as a ITensor pointer.
Definition: CLPyramid.cpp:123
Store the tensor's metadata.
Definition: TensorInfo.h:45
size_t width() const
Return the width of the 0th level tensor.
Definition: PyramidInfo.cpp:82
size_t num_levels() const
Return the number of the pyramid levels.
Definition: PyramidInfo.cpp:77
size_t height() const
Return the height of the 0th level tensor.
Definition: PyramidInfo.cpp:87

References ARM_COMPUTE_ERROR_ON, arm_compute::test::validation::border_mode, CLLKTrackerFinalizeKernel::configure(), ITensorInfo::dimension(), arm_compute::quantization::epsilon, CLPyramid::get_pyramid_level(), PyramidInfo::height(), ITensor::info(), CLPyramid::info(), MemoryGroup::manage(), PyramidInfo::num_levels(), IArray< T >::num_values(), IArray< T >::resize(), arm_compute::S16, PyramidInfo::scale(), and PyramidInfo::width().

◆ operator=() [1/2]

CLOpticalFlow& operator= ( const CLOpticalFlow )

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

CLOpticalFlow& operator= ( CLOpticalFlow &&  )

Allow instances of this class to be moved.

◆ run()

void run ( )

Run the kernels contained in the function.

For NEON kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 148 of file CLOpticalFlow.cpp.

149 {
150  ARM_COMPUTE_ERROR_ON_MSG(_num_levels == 0, "Unconfigured function");
152  MemoryGroupResourceScope scope_mg(_memory_group);
154  for(unsigned int level = _num_levels; level > 0; --level)
155  {
156  // Run Scharr kernel
157  _func_scharr[level - 1].run();
159  // Run Lucas-Kanade init kernel
160  CLScheduler::get().enqueue(_tracker_init_kernel[level - 1]);
162  // Run Lucas-Kanade stage0 kernel
163  CLScheduler::get().enqueue(_tracker_stage0_kernel[level - 1]);
165  // Run Lucas-Kanade stage1 kernel
166  CLScheduler::get().enqueue(_tracker_stage1_kernel[level - 1]);
167  }
169  CLScheduler::get().enqueue(_tracker_finalize_kernel, true);
170 }
static CLScheduler & get()
Access the scheduler singleton.
Definition: CLScheduler.cpp:99
#define ARM_COMPUTE_ERROR_ON_MSG(cond, msg)
Definition: Error.h:456
void enqueue(ICLKernel &kernel, bool flush=true)
Schedule the execution of the passed kernel if possible.
Memory group resources scope handling class.
Definition: IMemoryGroup.h:82

References ARM_COMPUTE_ERROR_ON_MSG, CLScheduler::enqueue(), CLScheduler::get(), and GemmTuner::level.

The documentation for this class was generated from the following files: