Scheduler interface to run kernels. More...

#include <IScheduler.h>

Data Structures
class	Hints
	Scheduler hints. More...

Public Types
enum	StrategyHint { STATIC, DYNAMIC }
	Strategies available to split a workload. More...

using	BindFunc = std::function< int(int, int)>
	Function to be used and map a given thread id to a logical core id. More...

using	Workload = std::function< void(const ThreadInfo &)>
	Signature for the workloads to execute. More...

Public Member Functions
	IScheduler ()
	Default constructor. More...

virtual	~IScheduler ()=default
	Destructor. More...

virtual void	set_num_threads (unsigned int num_threads)=0
	Sets the number of threads the scheduler will use to run the kernels. More...

virtual void	set_num_threads_with_affinity (unsigned int num_threads, BindFunc func)
	Sets the number of threads the scheduler will use to run the kernels but also using a binding function to pin the threads to given logical cores. More...

virtual unsigned int	num_threads () const =0
	Returns the number of threads that the SingleThreadScheduler has in its pool. More...

virtual void	schedule (ICPPKernel *kernel, const Hints &hints)=0
	Runs the kernel in the same thread as the caller synchronously. More...

virtual void	schedule_op (ICPPKernel *kernel, const Hints &hints, const Window &window, ITensorPack &tensors)=0
	Runs the kernel in the same thread as the caller synchronously. More...

virtual void	run_tagged_workloads (std::vector< Workload > &workloads, const char *tag)
	Execute all the passed workloads. More...

CPUInfo &	cpu_info ()
	Get CPU info. More...

unsigned int	num_threads_hint () const
	Get a hint for the best possible number of execution threads. More...

Static Public Attributes
static constexpr unsigned int	split_dimensions_all = std::numeric_limits<unsigned>::max()
	When arm_compute::ISchedular::Hints::_split_dimension is initialized with this value then the schedular is free to break down the problem space over as many dimensions as it wishes. More...

Detailed Description

Scheduler interface to run kernels.

Definition at line 41 of file IScheduler.h.

Member Typedef Documentation

◆ BindFunc

using BindFunc = std::function<int(int, int)>

Function to be used and map a given thread id to a logical core id.

Mapping function expects the thread index and total number of cores as input, and returns the logical core index to bind against

Definition at line 56 of file IScheduler.h.

◆ Workload

using Workload = std::function<void(const ThreadInfo &)>

Signature for the workloads to execute.

Definition at line 135 of file IScheduler.h.

Member Enumeration Documentation

◆ StrategyHint

enum StrategyHint

strong

Strategies available to split a workload.

Enumerator
STATIC	Split the workload evenly among the threads.
DYNAMIC	Split the workload dynamically using a bucket system.

Definition at line 45 of file IScheduler.h.

     {
         STATIC,  /**< Split the workload evenly among the threads */
         DYNAMIC, /**< Split the workload dynamically using a bucket system */
     };

Constructor & Destructor Documentation

◆ IScheduler()

IScheduler ( )

Default constructor.

Definition at line 35 of file IScheduler.cpp.

References arm_compute::cpuinfo::num_threads_hint().

 {
     // Work out the best possible number of execution threads
     _num_threads_hint = cpuinfo::num_threads_hint();
 }

◆ ~IScheduler()

virtual ~IScheduler ( )

virtualdefault

Destructor.

Member Function Documentation

◆ cpu_info()

CPUInfo & cpu_info ( )

Get CPU info.

Returns: CPU info.

Definition at line 41 of file IScheduler.cpp.

References CPUInfo::get().

Referenced by CpuDepthwiseConv2dAssemblyDispatch::configure(), CpuPool2d::configure(), CpuConv2d::get_convolution_method(), CpuGemmAssemblyDispatch::has_opt_impl(), main(), CPPScheduler::num_threads(), IScheduler::num_threads_hint(), IScheduler::run_tagged_workloads(), SingleThreadScheduler::schedule(), SingleThreadScheduler::schedule_op(), and OMPScheduler::schedule_op().

 {
     return CPUInfo::get();
 }

◆ num_threads()

virtual unsigned int num_threads ( ) const

pure virtual

Returns the number of threads that the SingleThreadScheduler has in its pool.

Returns: Number of threads available in SingleThreadScheduler.

Implemented in CPPScheduler, OMPScheduler, and SingleThreadScheduler.

Referenced by CpuDepthwiseConv2dAssemblyDispatch::configure(), CpuPool2d::configure(), CpuWinogradConv2d::configure(), CpuGemmAssemblyDispatch::has_opt_impl(), IScheduler::num_threads_hint(), and CpuWinogradConv2d::run().

◆ num_threads_hint()

unsigned int num_threads_hint ( ) const

Get a hint for the best possible number of execution threads.

Warning: In case we can't work out the best number of threads, std::thread::hardware_concurrency() is returned else 1 in case of bare metal builds

Returns: Best possible number of execution threads to use

Definition at line 52 of file IScheduler.cpp.

References ARM_COMPUTE_ERROR, ARM_COMPUTE_ERROR_ON_MSG, ARM_COMPUTE_UNUSED, ThreadInfo::cpu_info, IScheduler::cpu_info(), Window::DimX, Window::DimY, IScheduler::DYNAMIC, ITensorPack::empty(), arm_compute::test::validation::info, IKernel::is_parallelisable(), arm_compute::test::validation::m, arm_compute::test::validation::n, Window::num_iterations(), IScheduler::num_threads(), ICPPKernel::run(), ICPPKernel::run_nd(), ICPPKernel::run_op(), Window::set(), arm_compute::scheduler_utils::split_2d(), IScheduler::Hints::split_dimension(), IScheduler::split_dimensions_all, Window::split_window(), IScheduler::STATIC, IScheduler::Hints::strategy(), tf_frozen_model_extractor::t, IScheduler::Hints::threshold(), and Window::validate().

Referenced by CPPScheduler::set_num_threads(), and CPPScheduler::set_num_threads_with_affinity().

 {
     return _num_threads_hint;
 }

◆ run_tagged_workloads()

void run_tagged_workloads	(	std::vector< Workload > &	workloads,
		const char *	tag
	)

virtual

Execute all the passed workloads.

Note: There is no guarantee regarding the order in which the workloads will be executed or whether or not they will be executed in parallel.

Parameters

[in]	workloads	List of workloads to run
[in]	tag	String that can be used by profiling tools to identify the workloads run by the scheduler (Can be null).

Definition at line 172 of file IScheduler.cpp.

References ARM_COMPUTE_LOG_INFO_MSG_CORE, ARM_COMPUTE_LOG_INFO_MSG_WITH_FORMAT_CORE, ARM_COMPUTE_UNUSED, IScheduler::cpu_info(), Window::DimW, Window::DimX, Window::DimY, ICPPKernel::get_mws(), Window::num_iterations(), and tf_frozen_model_extractor::t.

 {
     ARM_COMPUTE_UNUSED(tag);
     run_workloads(workloads);
 }

◆ schedule()

virtual void schedule	(	ICPPKernel *	kernel,
		const Hints &	hints
	)

pure virtual

Runs the kernel in the same thread as the caller synchronously.

Parameters

[in]	kernel	Kernel to execute.
[in]	hints	Hints for the scheduler.

Implemented in CPPScheduler, OMPScheduler, and SingleThreadScheduler.

Referenced by ICPPSimpleFunction::run(), INESimpleFunction::run(), NEFillBorder::run(), NEROIPoolingLayer::run(), NESpaceToDepthLayer::run(), CPPBoxWithNonMaximaSuppressionLimit::run(), NEStackLayer::run(), NEFFT1D::run(), NEL2NormalizeLayer::run(), NERange::run(), NEReductionOperation::run(), NENormalizationLayer::run(), NEInstanceNormalizationLayer::run(), NEPadLayer::run(), NECropResize::run(), NEFuseBatchNormalization::run(), NEBatchNormalizationLayer::run(), NESpaceToBatchLayer::run(), NEGenerateProposalsLayer::run(), NESynthetizeFunctionWithZeroConstantKernelBorder< K >::run(), NEQLSTMLayer::run(), and arm_compute::utils::schedule_kernel_on_ctx().

◆ schedule_op()

virtual void schedule_op	(	ICPPKernel *	kernel,
		const Hints &	hints,
		const Window &	window,
		ITensorPack &	tensors
	)

pure virtual

Runs the kernel in the same thread as the caller synchronously.

Parameters

[in]	kernel	Kernel to execute.
[in]	hints	Hints for the scheduler.
[in]	window	Window to use for kernel execution.
[in]	tensors	Vector containing the tensors to operate on.

Implemented in OMPScheduler, CPPScheduler, and SingleThreadScheduler.

Referenced by CpuFullyConnected::prepare(), CpuGemmLowpMatrixMultiplyCore::prepare(), CpuGemmConv2d::prepare(), NEQLSTMLayer::prepare(), CpuDequantize::run(), CpuQuantize::run(), CpuConvertFullyConnectedWeights::run(), CpuActivation::run(), CpuAddMulAdd::run(), CpuScale::run(), INEOperator::run(), CpuPool3d::run(), CpuSub::run(), CpuConcatenate::run(), CpuAdd::run(), CpuDepthwiseConv2dAssemblyDispatch::run(), CpuPool2d::run(), CpuMul::run(), CpuMatMul::run(), CpuSoftmaxGeneric< IS_LOG >::run(), NELogicalAnd::run(), CpuGemmLowpOutputStage::run(), CpuDirectConv2d::run(), CpuDirectConv3d::run(), CpuWinogradConv2d::run(), CpuComplexMul::run(), CpuGemm::run(), CpuGemmLowpMatrixMultiplyCore::run(), NESynthetizeFunctionWithZeroConstantKernelBorder< K >::run(), NELogicalOr::run(), CpuGemmConv2d::run(), and NELogicalNot::run().

◆ set_num_threads()

virtual void set_num_threads ( unsigned int num_threads )

pure virtual

Sets the number of threads the scheduler will use to run the kernels.

Parameters

[in] num_threads If set to 0, then one thread per CPU core available on the system will be used, otherwise the number of threads specified.

Implemented in CPPScheduler, OMPScheduler, and SingleThreadScheduler.

Referenced by main(), and NEDeviceBackend::setup_backend_context().

◆ set_num_threads_with_affinity()

void set_num_threads_with_affinity	(	unsigned int	num_threads,
		BindFunc	func
	)

virtual

Sets the number of threads the scheduler will use to run the kernels but also using a binding function to pin the threads to given logical cores.

Parameters

[in]	num_threads	If set to 0, then one thread per CPU core available on the system will be used, otherwise the number of threads specified.
[in]	func	Binding function to use.

Reimplemented in CPPScheduler.

Definition at line 46 of file IScheduler.cpp.

References ARM_COMPUTE_ERROR, and ARM_COMPUTE_UNUSED.

 {
     ARM_COMPUTE_UNUSED(num_threads, func);
     ARM_COMPUTE_ERROR("Feature for affinity setting is not implemented");
 }

Field Documentation

◆ split_dimensions_all

constexpr unsigned int split_dimensions_all = std::numeric_limits<unsigned>::max()

static

When arm_compute::ISchedular::Hints::_split_dimension is initialized with this value then the schedular is free to break down the problem space over as many dimensions as it wishes.

Definition at line 62 of file IScheduler.h.

Referenced by IScheduler::num_threads_hint(), and SingleThreadScheduler::schedule().

The documentation for this class was generated from the following files:

arm_compute/runtime/IScheduler.h
src/runtime/IScheduler.cpp

Data Structures

Public Types

Public Member Functions

Static Public Attributes

Detailed Description

Member Typedef Documentation

◆ BindFunc

◆ Workload

Member Enumeration Documentation

◆ StrategyHint

Constructor & Destructor Documentation

◆ IScheduler()

◆ ~IScheduler()

Member Function Documentation

◆ cpu_info()

◆ num_threads()

◆ num_threads_hint()

◆ run_tagged_workloads()

◆ schedule()

◆ schedule_op()

◆ set_num_threads()

◆ set_num_threads_with_affinity()

Field Documentation

◆ split_dimensions_all