Compute Library
 22.11
CLDeviceBackend Class Referencefinal

OpenCL device backend. More...

#include <CLDeviceBackend.h>

Collaboration diagram for CLDeviceBackend:
[legend]

Public Member Functions

 CLDeviceBackend ()
 Default Constructor. More...
 
 ~CLDeviceBackend ()
 Destructor. More...
 
void set_kernel_tuning (bool enable_tuning)
 Switchs on or off the kernel tuning. More...
 
void set_kernel_tuning_mode (CLTunerMode tuning_mode)
 Set kernel tuning mode. More...
 
void initialize_backend () override
 Initializes the backend. More...
 
void setup_backend_context (GraphContext &ctx) override
 Setups the given graph context. More...
 
void release_backend_context (GraphContext &ctx) override
 Release the backend specific resources associated to a given graph context. More...
 
bool is_backend_supported () override
 Checks if an instantiated backend is actually supported. More...
 
IAllocatorbackend_allocator () override
 Gets a backend memory allocator. More...
 
std::unique_ptr< ITensorHandlecreate_tensor (const Tensor &tensor) override
 Create a backend Tensor. More...
 
std::unique_ptr< ITensorHandlecreate_subtensor (ITensorHandle *parent, TensorShape shape, Coordinates coords, bool extend_parent) override
 Create a backend Sub-Tensor. More...
 
std::unique_ptr< arm_compute::IFunctionconfigure_node (INode &node, GraphContext &ctx) override
 Configure a backend Node. More...
 
Status validate_node (INode &node) override
 Validate a node. More...
 
std::shared_ptr< arm_compute::IMemoryManagercreate_memory_manager (MemoryManagerAffinity affinity) override
 Create a backend memory manager given its affinity. More...
 
std::shared_ptr< arm_compute::IWeightsManagercreate_weights_manager () override
 Create a backend weights manager. More...
 
void sync () override
 Synchronize kernels execution on the backend. More...
 
- Public Member Functions inherited from IDeviceBackend
virtual ~IDeviceBackend ()=default
 Virtual Destructor. More...
 

Detailed Description

OpenCL device backend.

Definition at line 41 of file CLDeviceBackend.h.

Constructor & Destructor Documentation

◆ CLDeviceBackend()

Default Constructor.

Definition at line 66 of file CLDeviceBackend.cpp.

References arm_compute::Native.

67  : _context_count(0), _tuner(), _gemm_heuristics(), _allocator(nullptr), _tuner_file(), _backend_type(CLBackendType::Native)
68 {
69 }
OpenCL native backend.

◆ ~CLDeviceBackend()

Destructor.

Definition at line 71 of file CLDeviceBackend.cpp.

References CLTuner::save_to_file().

72 {
73  _tuner.save_to_file(_tuner_file);
74 }
bool save_to_file(const std::string &filename) const
Save the content of the tuning parameters table to file.
Definition: CLTuner.cpp:362

Member Function Documentation

◆ backend_allocator()

IAllocator * backend_allocator ( )
overridevirtual

Gets a backend memory allocator.

Returns
Backend memory allocator

Implements IDeviceBackend.

Definition at line 159 of file CLDeviceBackend.cpp.

160 {
161  return _allocator.get();
162 }

◆ configure_node()

std::unique_ptr< arm_compute::IFunction > configure_node ( INode node,
GraphContext ctx 
)
overridevirtual

Configure a backend Node.

Note
This creates an appropriate configured backend function for the given node
Parameters
[in]nodeThe node we want to configure
[in]ctxContext to use
Returns
Backend execution function

Implements IDeviceBackend.

Definition at line 187 of file CLDeviceBackend.cpp.

References ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_LOG_GRAPH_VERBOSE, INode::assigned_target(), arm_compute::graph::CL, CLFunctionFactory::create(), and INode::id().

188 {
189  ARM_COMPUTE_LOG_GRAPH_VERBOSE("Configuring CL node with ID : " << node.id() << std::endl);
190  ARM_COMPUTE_ERROR_ON(node.assigned_target() != Target::CL);
191 
192  // Configure node
193  return CLFunctionFactory::create(&node, ctx);
194 }
static std::unique_ptr< arm_compute::IFunction > create(INode *node, GraphContext &ctx)
Create a backend execution function depending on the node type.
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
#define ARM_COMPUTE_LOG_GRAPH_VERBOSE(x)
Definition: Logger.h:50
OpenCL capable target device.

◆ create_memory_manager()

std::shared_ptr< arm_compute::IMemoryManager > create_memory_manager ( MemoryManagerAffinity  affinity)
overridevirtual

Create a backend memory manager given its affinity.

Parameters
[in]affinityMemory Manager affinity
Returns
Memory manager

Implements IDeviceBackend.

Definition at line 204 of file CLDeviceBackend.cpp.

References ARM_COMPUTE_LOG_GRAPH_WARNING, and arm_compute::graph::Offset.

Referenced by CLDeviceBackend::setup_backend_context().

205 {
206  if(affinity == MemoryManagerAffinity::Offset)
207  {
208  ARM_COMPUTE_LOG_GRAPH_WARNING("CL Backend does not support offset affinity memory management!");
209  return nullptr;
210  }
211 
212  auto lifetime_mgr = std::make_shared<BlobLifetimeManager>();
213  auto pool_mgr = std::make_shared<PoolManager>();
214  auto mm = std::make_shared<MemoryManagerOnDemand>(lifetime_mgr, pool_mgr);
215 
216  return mm;
217 }
#define ARM_COMPUTE_LOG_GRAPH_WARNING(x)
Definition: Logger.h:58

◆ create_subtensor()

std::unique_ptr< ITensorHandle > create_subtensor ( ITensorHandle parent,
TensorShape  shape,
Coordinates  coords,
bool  extend_parent 
)
overridevirtual

Create a backend Sub-Tensor.

Parameters
[in]parentParent sub-tensor handle
[in]shapeShape of the sub-tensor
[in]coordsStarting coordinates of the sub-tensor
[in]extend_parentExtends parent shape if true
Returns
Backend sub-tensor handle

Implements IDeviceBackend.

Definition at line 177 of file CLDeviceBackend.cpp.

References arm_compute::test::validation::shape.

178 {
179  if(parent == nullptr)
180  {
181  return nullptr;
182  }
183 
184  return std::make_unique<CLSubTensorHandle>(parent, shape, coords, extend_parent);
185 }

◆ create_tensor()

std::unique_ptr< ITensorHandle > create_tensor ( const Tensor tensor)
overridevirtual

Create a backend Tensor.

Parameters
[in]tensorThe tensor we want to create a backend tensor for
Returns
Backend tensor handle

Implements IDeviceBackend.

Definition at line 164 of file CLDeviceBackend.cpp.

References ARM_COMPUTE_ERROR_ON, arm_compute::graph::CL, TensorDescriptor::data_type, Tensor::desc(), arm_compute::test::validation::info, TensorDescriptor::layout, TensorDescriptor::quant_info, TensorInfo::set_data_layout(), TensorDescriptor::shape, and TensorDescriptor::target.

165 {
166  // Get tensor descriptor
167  const TensorDescriptor &tensor_desc = tensor.desc();
168  ARM_COMPUTE_ERROR_ON(tensor_desc.target != Target::CL);
169 
170  // Create backend tensor handle
171  TensorInfo info(tensor_desc.shape, 1, tensor_desc.data_type, tensor_desc.quant_info);
172  info.set_data_layout(tensor_desc.layout);
173 
174  return std::make_unique<CLTensorHandle>(info);
175 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
ScaleKernelInfo info(interpolation_policy, default_border_mode, PixelValue(), sampling_policy, false)
CLTensor * tensor
Pointer to the auxiliary tensor.
OpenCL capable target device.

◆ create_weights_manager()

std::shared_ptr< arm_compute::IWeightsManager > create_weights_manager ( )
overridevirtual

Create a backend weights manager.

Returns
Weights manager

Implements IDeviceBackend.

Definition at line 219 of file CLDeviceBackend.cpp.

Referenced by CLDeviceBackend::setup_backend_context().

220 {
221  auto weights_mgr = std::make_shared<IWeightsManager>();
222  return weights_mgr;
223 }

◆ initialize_backend()

void initialize_backend ( )
overridevirtual

Initializes the backend.

Implements IDeviceBackend.

Definition at line 86 of file CLDeviceBackend.cpp.

References CLScheduler::default_init(), and CLScheduler::get().

Referenced by CLDeviceBackend::setup_backend_context().

87 {
88  // Setup Scheduler
89  CLScheduler::get().default_init(&_tuner, &_gemm_heuristics, _backend_type);
90  // Create allocator with new context
91  _allocator = std::make_unique<CLBufferAllocator>();
92 }
static CLScheduler & get()
Access the scheduler singleton.
void default_init(ICLTuner *cl_tuner=nullptr, CLGEMMHeuristicsHandle *gemm_h=nullptr, CLBackendType cl_backend_type=CLBackendType::Native)
Initialises the context and command queue used by the scheduler to default values and sets a default ...

◆ is_backend_supported()

bool is_backend_supported ( )
overridevirtual

Checks if an instantiated backend is actually supported.

Returns
True if the backend is supported else false

Implements IDeviceBackend.

Definition at line 154 of file CLDeviceBackend.cpp.

References arm_compute::opencl_is_available().

155 {
157 }
bool opencl_is_available()
Check if OpenCL is available.
Definition: OpenCL.cpp:188

◆ release_backend_context()

void release_backend_context ( GraphContext ctx)
overridevirtual

Release the backend specific resources associated to a given graph context.

Parameters
[in,out]ctxGraph context

Implements IDeviceBackend.

Definition at line 94 of file CLDeviceBackend.cpp.

References ARM_COMPUTE_UNUSED.

95 {
96  ARM_COMPUTE_UNUSED(ctx);
97  _context_count--;
98  if(_context_count == 0) // No more context using the backend: free resources
99  {
100  _allocator = nullptr;
101  }
102 }
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:152

◆ set_kernel_tuning()

void set_kernel_tuning ( bool  enable_tuning)

Switchs on or off the kernel tuning.

Note
When true the tuner set is used, if no tuner is set a new default one is created
Parameters
[in]enable_tuningEnables tuning if false else true

Definition at line 76 of file CLDeviceBackend.cpp.

References CLTuner::set_tune_new_kernels().

Referenced by CLDeviceBackend::setup_backend_context().

77 {
78  _tuner.set_tune_new_kernels(enable_tuning);
79 }
void set_tune_new_kernels(bool tune_new_kernels)
Setter for tune_new_kernels option.
Definition: CLTuner.cpp:97

◆ set_kernel_tuning_mode()

void set_kernel_tuning_mode ( CLTunerMode  tuning_mode)

Set kernel tuning mode.

Parameters
[in]tuning_modeIndicates how exhaustive the search for the optimal LWS should be while tuning

Definition at line 81 of file CLDeviceBackend.cpp.

References CLTuner::set_tuner_mode().

Referenced by CLDeviceBackend::setup_backend_context().

82 {
83  _tuner.set_tuner_mode(tuning_mode);
84 }
void set_tuner_mode(CLTunerMode mode)
Set OpenCL tuner mode.
Definition: CLTuner.cpp:106

◆ setup_backend_context()

void setup_backend_context ( GraphContext ctx)
overridevirtual

Setups the given graph context.

Parameters
[in,out]ctxGraph context

Implements IDeviceBackend.

Definition at line 104 of file CLDeviceBackend.cpp.

References MemoryManagerContext::allocator, ARM_COMPUTE_ERROR_ON, GraphConfig::backend_type, arm_compute::graph::Buffer, arm_compute::graph::CL, GraphContext::config(), CLDeviceBackend::create_memory_manager(), CLDeviceBackend::create_weights_manager(), MemoryManagerContext::cross_group, MemoryManagerContext::cross_mm, CLScheduler::gemm_heuristics(), CLScheduler::get(), CLDeviceBackend::initialize_backend(), GraphContext::insert_memory_management_ctx(), GraphContext::insert_weights_management_ctx(), MemoryManagerContext::intra_mm, CLTuner::load_from_file(), GraphContext::memory_management_ctx(), GraphConfig::mlgo_file, CLGEMMHeuristicsHandle::reload_from_file(), CLDeviceBackend::set_kernel_tuning(), CLDeviceBackend::set_kernel_tuning_mode(), MemoryManagerContext::target, WeightsManagerContext::target, GraphConfig::tuner_file, GraphConfig::tuner_mode, GraphConfig::use_tuner, GraphContext::weights_management_ctx(), and WeightsManagerContext::wm.

105 {
106  // Force backend initialization
107  _context_count++;
108  if(_context_count == 1)
109  {
110  _backend_type = ctx.config().backend_type;
112  }
113 
114  // Setup tuner
115  _tuner_file = ctx.config().tuner_file;
116 
117  // Load tuner data if available
118  if(file_exists(_tuner_file))
119  {
120  _tuner.load_from_file(_tuner_file);
121  }
122 
123  set_kernel_tuning(ctx.config().use_tuner);
124  set_kernel_tuning_mode(ctx.config().tuner_mode);
125 
126  // Attempt to load mlgo heuristics
127  ARM_COMPUTE_ERROR_ON(CLScheduler::get().gemm_heuristics() == nullptr);
128  CLScheduler::get().gemm_heuristics()->reload_from_file(ctx.config().mlgo_file);
129 
130  // Setup a management backend
131  if(ctx.memory_management_ctx(Target::CL) == nullptr)
132  {
133  MemoryManagerContext mm_ctx;
134  mm_ctx.target = Target::CL;
137  mm_ctx.cross_group = std::make_shared<MemoryGroup>(mm_ctx.cross_mm);
138  mm_ctx.allocator = _allocator.get();
139 
140  ctx.insert_memory_management_ctx(std::move(mm_ctx));
141  }
142 
143  // Create function level weights manager
144  if(ctx.weights_management_ctx(Target::CL) == nullptr)
145  {
146  WeightsManagerContext wm_ctx;
147  wm_ctx.target = Target::CL;
148  wm_ctx.wm = create_weights_manager();
149 
150  ctx.insert_weights_management_ctx(std::move(wm_ctx));
151  }
152 }
void set_kernel_tuning(bool enable_tuning)
Switchs on or off the kernel tuning.
static CLScheduler & get()
Access the scheduler singleton.
void set_kernel_tuning_mode(CLTunerMode tuning_mode)
Set kernel tuning mode.
void load_from_file(const std::string &filename)
Load the tuning parameters table from file.
Definition: CLTuner.cpp:310
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
void initialize_backend() override
Initializes the backend.
CLGEMMHeuristicsHandle * gemm_heuristics() const
Accessor for the associated CLGEMMHeuristicsHandle.
Definition: CLScheduler.cpp:54
std::shared_ptr< arm_compute::IWeightsManager > create_weights_manager() override
Create a backend weights manager.
bool reload_from_file(const std::string &filename)
(Re)Load the heuristics from reading a dotmlgo file
std::shared_ptr< arm_compute::IMemoryManager > create_memory_manager(MemoryManagerAffinity affinity) override
Create a backend memory manager given its affinity.
OpenCL capable target device.

◆ sync()

void sync ( )
overridevirtual

Synchronize kernels execution on the backend.

On GPU, this results in a blocking call waiting for all kernels to be completed.

Implements IDeviceBackend.

Definition at line 225 of file CLDeviceBackend.cpp.

References CLScheduler::get(), and CLScheduler::sync().

226 {
228 }
static CLScheduler & get()
Access the scheduler singleton.
void sync()
Blocks until all commands in the associated command queue have finished.
Definition: CLScheduler.cpp:74

◆ validate_node()

arm_compute::Status validate_node ( INode node)
overridevirtual

Validate a node.

Parameters
[in]nodeThe node we want to validate
Returns
An error status

Implements IDeviceBackend.

Definition at line 196 of file CLDeviceBackend.cpp.

References ARM_COMPUTE_ERROR_ON, ARM_COMPUTE_LOG_GRAPH_VERBOSE, INode::assigned_target(), arm_compute::graph::CL, INode::id(), and CLNodeValidator::validate().

197 {
198  ARM_COMPUTE_LOG_GRAPH_VERBOSE("Validating CL node with ID : " << node.id() << std::endl);
199  ARM_COMPUTE_ERROR_ON(node.assigned_target() != Target::CL);
200 
201  return CLNodeValidator::validate(&node);
202 }
#define ARM_COMPUTE_ERROR_ON(cond)
If the condition is true then an error message is printed and an exception thrown.
Definition: Error.h:466
static Status validate(INode *node)
Validate a node.
#define ARM_COMPUTE_LOG_GRAPH_VERBOSE(x)
Definition: Logger.h:50
OpenCL capable target device.

The documentation for this class was generated from the following files: