Compute Library
 21.02
CLOpticalFlow Class Reference

Basic function to execute optical flow. More...

#include <CLOpticalFlow.h>

Collaboration diagram for CLOpticalFlow:
[legend]

Public Member Functions

 CLOpticalFlow (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 
 CLOpticalFlow (const CLOpticalFlow &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
CLOpticalFlowoperator= (const CLOpticalFlow &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 CLOpticalFlow (CLOpticalFlow &&)=default
 Allow instances of this class to be moved. More...
 
CLOpticalFlowoperator= (CLOpticalFlow &&)=default
 Allow instances of this class to be moved. More...
 
 ~CLOpticalFlow ()
 Default destructor. More...
 
void configure (const CLPyramid *old_pyramid, const CLPyramid *new_pyramid, const ICLKeyPointArray *old_points, const ICLKeyPointArray *new_points_estimates, ICLKeyPointArray *new_points, Termination termination, float epsilon, size_t num_iterations, size_t window_dimension, bool use_initial_estimate, BorderMode border_mode, uint8_t constant_border_value=0)
 Initialise the function input and output. More...
 
void configure (const CLCompileContext &compile_context, const CLPyramid *old_pyramid, const CLPyramid *new_pyramid, const ICLKeyPointArray *old_points, const ICLKeyPointArray *new_points_estimates, ICLKeyPointArray *new_points, Termination termination, float epsilon, size_t num_iterations, size_t window_dimension, bool use_initial_estimate, BorderMode border_mode, uint8_t constant_border_value=0)
 Initialise the function input and output. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Detailed Description

Basic function to execute optical flow.

This function calls the following OpenCL kernels and functions:

  1. CLScharr3x3
  2. CLLKTrackerInitKernel
  3. CLLKTrackerStage0Kernel
  4. CLLKTrackerStage1Kernel
  5. CLLKTrackerFinalizeKernel
Deprecated:
This function is deprecated and is intended to be removed in 21.05 release

Definition at line 67 of file CLOpticalFlow.h.

Constructor & Destructor Documentation

◆ CLOpticalFlow() [1/3]

CLOpticalFlow ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Definition at line 40 of file CLOpticalFlow.cpp.

References CLOpticalFlow::~CLOpticalFlow().

41  : _memory_group(std::move(memory_manager)),
42  _tracker_init_kernel(),
43  _tracker_stage0_kernel(),
44  _tracker_stage1_kernel(),
45  _tracker_finalize_kernel(std::make_unique<CLLKTrackerFinalizeKernel>()),
46  _func_scharr(),
47  _scharr_gx(),
48  _scharr_gy(),
49  _old_points(nullptr),
50  _new_points_estimates(nullptr),
51  _new_points(nullptr),
52  _old_points_internal(),
53  _new_points_internal(),
54  _coefficient_table(),
55  _old_values(),
56  _num_levels(0)
57 {
58 }

◆ CLOpticalFlow() [2/3]

CLOpticalFlow ( const CLOpticalFlow )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ CLOpticalFlow() [3/3]

CLOpticalFlow ( CLOpticalFlow &&  )
default

Allow instances of this class to be moved.

◆ ~CLOpticalFlow()

~CLOpticalFlow ( )
default

Default destructor.

Referenced by CLOpticalFlow::CLOpticalFlow().

Member Function Documentation

◆ configure() [1/2]

void configure ( const CLPyramid old_pyramid,
const CLPyramid new_pyramid,
const ICLKeyPointArray old_points,
const ICLKeyPointArray new_points_estimates,
ICLKeyPointArray new_points,
Termination  termination,
float  epsilon,
size_t  num_iterations,
size_t  window_dimension,
bool  use_initial_estimate,
BorderMode  border_mode,
uint8_t  constant_border_value = 0 
)

Initialise the function input and output.

Parameters
[in]old_pyramidPointer to the pyramid for the old tensor. Data types supported U8
[in]new_pyramidPointer to the pyramid for the new tensor. Data types supported U8
[in]old_pointsPointer to the IKeyPointArray storing old key points
[in]new_points_estimatesPointer to the IKeyPointArray storing new estimates key points
[out]new_pointsPointer to the IKeyPointArray storing new key points
[in]terminationThe criteria to terminate the search of each keypoint.
[in]epsilonThe error for terminating the algorithm
[in]num_iterationsThe maximum number of iterations before terminate the alogrithm
[in]window_dimensionThe size of the window on which to perform the algorithm
[in]use_initial_estimateThe flag to indicate whether the initial estimated position should be used
[in]border_modeThe border mode applied at scharr kernel stage
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT

Definition at line 62 of file CLOpticalFlow.cpp.

References CLKernelLibrary::get().

66 {
67  configure(CLKernelLibrary::get().get_compile_context(), old_pyramid, new_pyramid, old_points, new_points_estimates, new_points, termination, epsilon, num_iterations, window_dimension,
68  use_initial_estimate, border_mode, constant_border_value);
69 }
static CLKernelLibrary & get()
Access the KernelLibrary singleton.
void configure(const CLPyramid *old_pyramid, const CLPyramid *new_pyramid, const ICLKeyPointArray *old_points, const ICLKeyPointArray *new_points_estimates, ICLKeyPointArray *new_points, Termination termination, float epsilon, size_t num_iterations, size_t window_dimension, bool use_initial_estimate, BorderMode border_mode, uint8_t constant_border_value=0)
Initialise the function input and output.

◆ configure() [2/2]

void configure ( const CLCompileContext compile_context,
const CLPyramid old_pyramid,
const CLPyramid new_pyramid,
const ICLKeyPointArray old_points,
const ICLKeyPointArray new_points_estimates,
ICLKeyPointArray new_points,
Termination  termination,
float  epsilon,
size_t  num_iterations,
size_t  window_dimension,
bool  use_initial_estimate,
BorderMode  border_mode,
uint8_t  constant_border_value = 0 
)

Initialise the function input and output.

Parameters
[in]compile_contextThe compile context to be used.
[in]old_pyramidPointer to the pyramid for the old tensor. Data types supported U8
[in]new_pyramidPointer to the pyramid for the new tensor. Data types supported U8
[in]old_pointsPointer to the IKeyPointArray storing old key points
[in]new_points_estimatesPointer to the IKeyPointArray storing new estimates key points
[out]new_pointsPointer to the IKeyPointArray storing new key points
[in]terminationThe criteria to terminate the search of each keypoint.
[in]epsilonThe error for terminating the algorithm
[in]num_iterationsThe maximum number of iterations before terminate the alogrithm
[in]window_dimensionThe size of the window on which to perform the algorithm
[in]use_initial_estimateThe flag to indicate whether the initial estimated position should be used
[in]border_modeThe border mode applied at scharr kernel stage
[in]constant_border_value(Optional) Constant value to use for borders if border_mode is set to CONSTANT

Definition at line 71 of file CLOpticalFlow.cpp.

References ARM_COMPUTE_ERROR_ON, ITensorInfo::dimension(), arm_compute::quantization::epsilon, CLPyramid::get_pyramid_level(), PyramidInfo::height(), ITensor::info(), CLPyramid::info(), MemoryGroup::manage(), PyramidInfo::num_levels(), IArray< T >::num_values(), IArray< T >::resize(), arm_compute::S16, PyramidInfo::scale(), and PyramidInfo::width().

75 {
76  ARM_COMPUTE_ERROR_ON(nullptr == old_pyramid);
77  ARM_COMPUTE_ERROR_ON(nullptr == new_pyramid);
78  ARM_COMPUTE_ERROR_ON(nullptr == old_points);
79  ARM_COMPUTE_ERROR_ON(nullptr == new_points_estimates);
80  ARM_COMPUTE_ERROR_ON(nullptr == new_points);
81  ARM_COMPUTE_ERROR_ON(old_pyramid->info()->num_levels() != new_pyramid->info()->num_levels());
82  ARM_COMPUTE_ERROR_ON(0 == old_pyramid->info()->num_levels());
83  ARM_COMPUTE_ERROR_ON(old_pyramid->info()->width() != new_pyramid->info()->width());
84  ARM_COMPUTE_ERROR_ON(old_pyramid->info()->height() != new_pyramid->info()->height());
85  ARM_COMPUTE_ERROR_ON(use_initial_estimate && old_points->num_values() != new_points_estimates->num_values());
86 
87  // Set member variables
88  _old_points = old_points;
89  _new_points_estimates = new_points_estimates;
90  _new_points = new_points;
91  _num_levels = old_pyramid->info()->num_levels();
92 
93  const float pyr_scale = old_pyramid->info()->scale();
94  const int list_length = old_points->num_values();
95  const int old_values_list_length = list_length * window_dimension * window_dimension;
96 
97  // Create kernels and tensors
98  _tracker_init_kernel.reserve(_num_levels);
99  _tracker_stage0_kernel.reserve(_num_levels);
100  _tracker_stage1_kernel.reserve(_num_levels);
101  _func_scharr.resize(_num_levels);
102  _scharr_gx.resize(_num_levels);
103  _scharr_gy.resize(_num_levels);
104 
105  // Create internal keypoint arrays
106  _old_points_internal = std::make_unique<CLLKInternalKeypointArray>(list_length);
107  _old_points_internal->resize(list_length);
108  _new_points_internal = std::make_unique<CLLKInternalKeypointArray>(list_length);
109  _new_points_internal->resize(list_length);
110  _coefficient_table = std::make_unique<CLCoefficientTableArray>(list_length);
111  _coefficient_table->resize(list_length);
112  _old_values = std::make_unique<CLOldValueArray>(old_values_list_length);
113  _old_values->resize(old_values_list_length);
114  _new_points->resize(list_length);
115 
116  for(size_t i = 0; i < _num_levels; ++i)
117  {
118  // Get images from the ith level of old and right pyramid
119  ICLImage *old_ith_input = old_pyramid->get_pyramid_level(i);
120  ICLImage *new_ith_input = new_pyramid->get_pyramid_level(i);
121 
122  // Get width and height of images
123  const unsigned int width_ith = old_ith_input->info()->dimension(0);
124  const unsigned int height_ith = new_ith_input->info()->dimension(1);
125 
126  // Initialize Scharr tensors
127  TensorInfo tensor_info(TensorShape(width_ith, height_ith), 1, DataType::S16);
128  _scharr_gx[i].allocator()->init(tensor_info);
129  _scharr_gy[i].allocator()->init(tensor_info);
130 
131  // Manage intermediate buffers
132  _memory_group.manage(&_scharr_gx[i]);
133  _memory_group.manage(&_scharr_gy[i]);
134 
135  // Init Scharr kernel
136  _func_scharr[i].configure(compile_context, old_ith_input, &_scharr_gx[i], &_scharr_gy[i], border_mode, constant_border_value);
137 
138  // Init Lucas-Kanade init kernel
139  _tracker_init_kernel.emplace_back(std::make_unique<CLLKTrackerInitKernel>());
140  _tracker_init_kernel.back()->configure(compile_context, old_points, new_points_estimates, _old_points_internal.get(), _new_points_internal.get(), use_initial_estimate, i, _num_levels, pyr_scale);
141 
142  // Init Lucas-Kanade stage0 kernel
143  _tracker_stage0_kernel.emplace_back(std::make_unique<CLLKTrackerStage0Kernel>());
144  _tracker_stage0_kernel.back()->configure(compile_context, old_ith_input, &_scharr_gx[i], &_scharr_gy[i],
145  _old_points_internal.get(), _new_points_internal.get(), _coefficient_table.get(), _old_values.get(),
146  window_dimension, i);
147 
148  // Init Lucas-Kanade stage1 kernel
149  _tracker_stage1_kernel.emplace_back(std::make_unique<CLLKTrackerStage1Kernel>());
150  _tracker_stage1_kernel.back()->configure(compile_context, new_ith_input, _new_points_internal.get(), _coefficient_table.get(), _old_values.get(),
151  termination, epsilon, num_iterations, window_dimension, i);
152 
153  // Allocate intermediate buffers
154  _scharr_gx[i].allocator()->allocate();
155  _scharr_gy[i].allocator()->allocate();
156  }
157 
158  // Finalize Lucas-Kanade
159  _tracker_finalize_kernel->configure(compile_context, _new_points_internal.get(), new_points);
160 }
Shape of a tensor.
Definition: TensorShape.h:39
virtual size_t dimension(size_t index) const =0
Return the size of the requested dimension.
const PyramidInfo * info() const override
Interface to be implemented by the child class to return the Pyramid&#39;s metadata.
Definition: CLPyramid.cpp:117
void resize(size_t num)
Resizes the array to contain "num" elements.
Definition: IArray.h:128
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
void manage(IMemoryManageable *obj) override
Sets a object to be managed by the given memory group.
Definition: MemoryGroup.h:79
size_t num_values() const
Number of values currently stored in the array.
Definition: IArray.h:68
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor&#39;s metadata.
float scale() const
Return the scale factor of the pyramid.
1 channel, 1 S16 per channel
Interface for OpenCL tensor.
Definition: ICLTensor.h:42
CLTensor * get_pyramid_level(size_t index) const override
Retrieves a level of the pyramid as a ITensor pointer.
Definition: CLPyramid.cpp:122
Store the tensor&#39;s metadata.
Definition: TensorInfo.h:45
size_t width() const
Return the width of the 0th level tensor.
Definition: PyramidInfo.cpp:82
size_t num_levels() const
Return the number of the pyramid levels.
Definition: PyramidInfo.cpp:77
size_t height() const
Return the height of the 0th level tensor.
Definition: PyramidInfo.cpp:87

◆ operator=() [1/2]

CLOpticalFlow& operator= ( const CLOpticalFlow )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

CLOpticalFlow& operator= ( CLOpticalFlow &&  )
default

Allow instances of this class to be moved.

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For Neon kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 162 of file CLOpticalFlow.cpp.

References ARM_COMPUTE_ERROR_ON_MSG, CLScheduler::enqueue(), CLScheduler::get(), and GemmTuner::level.

163 {
164  ARM_COMPUTE_ERROR_ON_MSG(_num_levels == 0, "Unconfigured function");
165 
166  MemoryGroupResourceScope scope_mg(_memory_group);
167 
168  for(unsigned int level = _num_levels; level > 0; --level)
169  {
170  // Run Scharr kernel
171  _func_scharr[level - 1].run();
172 
173  // Run Lucas-Kanade init kernel
174  CLScheduler::get().enqueue(*_tracker_init_kernel[level - 1]);
175 
176  // Run Lucas-Kanade stage0 kernel
177  CLScheduler::get().enqueue(*_tracker_stage0_kernel[level - 1]);
178 
179  // Run Lucas-Kanade stage1 kernel
180  CLScheduler::get().enqueue(*_tracker_stage1_kernel[level - 1]);
181  }
182 
183  CLScheduler::get().enqueue(*_tracker_finalize_kernel, true);
184 }
static CLScheduler & get()
Access the scheduler singleton.
#define ARM_COMPUTE_ERROR_ON_MSG(cond, msg)
Definition: Error.h:456
void enqueue(ICLKernel &kernel, bool flush=true)
Schedule the execution of the passed kernel if possible.
Memory group resources scope handling class.
Definition: IMemoryGroup.h:82

The documentation for this class was generated from the following files: