NEConvolutionLayer Class Reference

Basic function to simulate a convolution layer. More...

#include <NEConvolutionLayer.h>

Collaboration diagram for NEConvolutionLayer:

Public Member Functions
	NEConvolutionLayer (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
	Constructor. More...
	NEConvolutionLayer (const NEConvolutionLayer &)=delete
	Prevent instances of this class from being copied (As this class contains pointers) More...
NEConvolutionLayer &	operator= (const NEConvolutionLayer &)=delete
	Prevent instances of this class from being copied (As this class contains pointers) More...
	NEConvolutionLayer (NEConvolutionLayer &&)=default
	Default move constructor. More...
NEConvolutionLayer &	operator= (NEConvolutionLayer &&)=default
	Prevent instances of this class from being moved (As this class contains non movable objects) More...
	~NEConvolutionLayer ()
	Default destructor. More...
void	configure (ITensor *input, const ITensor *weights, const ITensor *biases, ITensor *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info=WeightsInfo(), const Size2D &dilation=Size2D(1U, 1U), const ActivationLayerInfo &act_info=ActivationLayerInfo(), bool enable_fast_math=false, unsigned int num_groups=1)
	Set the input and output tensors. More...
void	run () override
	Run the kernels contained in the function. More...
void	prepare () override
	Prepare the function for executing. More...
Public Member Functions inherited from IFunction
virtual	~IFunction ()=default
	Destructor. More...

Static Public Member Functions
static Status	validate (const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info=WeightsInfo(), const Size2D &dilation=Size2D(1U, 1U), const ActivationLayerInfo &act_info=ActivationLayerInfo(), bool enable_fast_math=false, unsigned int num_groups=1)
	Static function to check if given info will lead to a valid configuration of NEConvolutionLayer. More...
static ConvolutionMethod	get_convolution_method (const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *output, const PadStrideInfo &conv_info, const WeightsInfo &weights_info=WeightsInfo(), const Size2D &dilation=Size2D(1U, 1U), const ActivationLayerInfo &act_info=ActivationLayerInfo(), bool enable_fast_math=false)
	Static function to check if given info will return the convolution called by NEConvolutionLayer. More...

Detailed Description

Basic function to simulate a convolution layer.

This function calls one of the following functions:

1. cpu::CpuGemmConv2d (executed only in case GEMM is required for the operation)
2. cpu::CpuWinogradConv2d (executed only in case Winograd is required for the operation)
3. cpu::CpuDirectConv2d (executed only in case Direct Convolution is required for the operation)
4. NEFFTConvolutionLayer (executed only in case FFT is required for the operation)

The function selects one of the algorithms mentioned above based on:

• The size of the kernel
• Number of input/output feature maps
• Amount of memory needed

Generally GEMM-based convolution is executed when neither Winograd nor FFT nor Direct convolution can be performed.

FP32 Algorithm	Filter Size	Input/Output feature maps
Winograd	3x3 1x3 3x1 5x1 1x5 5x5(fast maths) 7x1 1x7	Input channels is greater than 3
FFT	Squared kernels and greater than 9x9	Input feature maps > Output feature maps
DirectConv	9x9
GEMM	Any size

Winograd 5x5 requires fast maths enabled.

FP16 Algorithm	Filter Size
Winograd	Not supported
FFT	Not supported
DirectConv	9x9
GEMM	Any size

Definition at line 72 of file NEConvolutionLayer.h.

Constructor & Destructor Documentation

NEConvolutionLayer() [1/3]

NEConvolutionLayer

(

std::shared_ptr< IMemoryManager >

memory_manager = nullptr

)

Constructor.

Definition at line 56 of file NEConvolutionLayer.cpp.

                                                                                   : _impl(std::make_unique<Impl>())
{
    _impl->memory_manager = std::move(memory_manager);
}

NEConvolutionLayer() [2/3]

NEConvolutionLayer ( const NEConvolutionLayer &  )

delete

Prevent instances of this class from being copied (As this class contains pointers)

NEConvolutionLayer() [3/3]

NEConvolutionLayer ( NEConvolutionLayer &&  )

default

Default move constructor.

~NEConvolutionLayer()

~NEConvolutionLayer ( )

default

Default destructor.

Member Function Documentation

configure()

void configure	(	ITensor *	input,
		const ITensor *	weights,
		const ITensor *	biases,
		ITensor *	output,
		const PadStrideInfo &	conv_info,
		const WeightsInfo &	weights_info = WeightsInfo(),
		const Size2D &	dilation = Size2D(1U, 1U),
		const ActivationLayerInfo &	act_info = ActivationLayerInfo(),
		bool	enable_fast_math = false,
		unsigned int	num_groups = 1
	)

Set the input and output tensors.

Valid data layouts:

• NHWC
• NCHW

Valid data type configurations:

src0	src1	src2	dst
F16	F16	F16	F16
F32	F32	F32	F32
QASYMM8	QASYMM8	S32	QASYMM8
QASYMM8	QSYMM8_PER_CHANNEL	S32	QASYMM8
QASYMM8_SIGNED	QASYMM8_SIGNED	S32	QASYMM8_SIGNED
QASYMM8_SIGNED	QSYMM8_PER_CHANNEL	S32	QASYMM8_SIGNED

Parameters

[in]	input	Source tensor. 3 lower dimensions represent a single input [width, height, IFM], while every optional dimension from 4 and above represent a batch of inputs. Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
[in]	weights	Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM]. Data type supported: Same as input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
[in]	biases	Biases tensor. Shared biases supported. Biases are 1D tensor with dimensions [OFM]. Data type supported: Same as input, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
[out]	output	Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs. Data types supported: Same as input.
[in]	conv_info	Contains padding and stride information described in PadStrideInfo.
[in]	weights_info	Specifies if the weights tensor has been reshaped with NEWeightsReshapeKernel. If this is not part of the fully connected layer the weights tensor has also been transposed with cpu::kernels::CpuGemmTranspose1xWKernel. Data type supported: Same as input.
[in]	dilation	(Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
[in]	act_info	(Optional) Activation layer information in case of a fused activation. Only RELU, BOUNDED_RELU and LU_BOUNDED_RELU supported.
[in]	enable_fast_math	(Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation available which may introduce a drop of accuracy as well. Default is false
[in]	num_groups	(Optional) Number of groups when performing a grouped convolution. num_groups != 1 is not supported

Definition at line 63 of file NEConvolutionLayer.cpp.

{
    // Perform validate step
    ARM_COMPUTE_ERROR_ON_NULLPTR(input, weights, output);
    ARM_COMPUTE_UNUSED(num_groups);
    ARM_COMPUTE_ERROR_THROW_ON(NEConvolutionLayer::validate(
        input->info(), weights->info(), ((biases != nullptr) ? biases->info() : nullptr), output->info(), conv_info,
        weights_info, dilation, act_info, enable_fast_math, num_groups));
    ARM_COMPUTE_LOG_PARAMS(input, weights, biases, output, conv_info, weights_info, dilation, act_info,
                           enable_fast_math, num_groups);
 
    const Conv2dInfo info(conv_info, dilation, act_info, enable_fast_math, num_groups);
    switch (cpu::CpuConv2d::get_convolution_method(input->info(), weights->info(), output->info(), conv_info,
                                                   weights_info, dilation, act_info, enable_fast_math))
    {
        case ConvolutionMethod::WINOGRAD:
        case ConvolutionMethod::GEMM:
        case ConvolutionMethod::GEMM_CONV2D:
        case ConvolutionMethod::DIRECT:
        {
            auto f = std::make_unique<cpu::CpuConv2d>();
            f->configure(input->info(), weights->info(), ((biases != nullptr) ? biases->info() : nullptr),
                         output->info(), conv_info, weights_info, dilation, act_info, enable_fast_math, num_groups);
            _impl->op = std::move(f);
            break;
        }
        case ConvolutionMethod::FFT:
        {
            auto f = std::make_unique<NEFFTConvolutionLayer>(_impl->memory_manager);
            f->configure(input, weights, biases, output, conv_info, act_info);
            _impl->func = std::move(f);
            break;
        }
        default:
            ARM_COMPUTE_ERROR("Not supported.");
            break;
    }
 
    if (_impl->op)
    {
        _impl->memory_group = MemoryGroup(std::move(_impl->memory_manager));
        _impl->aux_mem_req  = _impl->op->workspace();
        _impl->run_pack     = {{ACL_SRC_0, input}, {ACL_SRC_1, weights}, {ACL_SRC_2, biases}, {ACL_DST, output}};
        _impl->prep_pack    = {{ACL_SRC_1, weights}, {ACL_SRC_2, biases}};
        _impl->workspace =
            manage_workspace<Tensor>(_impl->aux_mem_req, _impl->memory_group, _impl->run_pack, _impl->prep_pack);
    }
}

References arm_compute::ACL_DST, arm_compute::ACL_SRC_0, arm_compute::ACL_SRC_1, arm_compute::ACL_SRC_2, arm_compute::test::validation::act_info, ARM_COMPUTE_ERROR, ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, ARM_COMPUTE_LOG_PARAMS, ARM_COMPUTE_UNUSED, arm_compute::test::validation::conv_info, arm_compute::DIRECT, arm_compute::FFT, arm_compute::GEMM, arm_compute::GEMM_CONV2D, CpuConv2d::get_convolution_method(), ITensor::info(), arm_compute::test::validation::info, arm_compute::test::validation::input, arm_compute::test::validation::num_groups, NEConvolutionLayer::validate(), arm_compute::test::validation::weights_info, and arm_compute::WINOGRAD.

Referenced by NEDeconvolutionLayer::configure().

get_convolution_method()

ConvolutionMethod get_convolution_method ( const ITensorInfo *  input, const ITensorInfo *  weights, const ITensorInfo *  output, const PadStrideInfo &  conv_info, const WeightsInfo &  weights_info = WeightsInfo(), const Size2D &  dilation = Size2D(1U, 1U), const ActivationLayerInfo &  act_info = ActivationLayerInfo(), bool  enable_fast_math = false  )

static

Static function to check if given info will return the convolution called by NEConvolutionLayer.

Parameters

[in]	input	Source tensor. 3 lower dimensions represent a single input [width, height, IFM], while every optional dimension from 4 and above represent a batch of inputs. Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
[in]	weights	Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM]. Data type supported:Same as input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
[in]	output	Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs. Data types supported: Same as input.
[in]	conv_info	Contains padding and stride information described in PadStrideInfo.
[in]	weights_info	Specifies if the weights tensor has been reshaped with NEWeightsReshapeKernel. If this is not part of the fully connected layer the weights tensor has also been transposed with cpu::kernels::CpuGemmTranspose1xWKernel. Data type supported: Same as input.
[in]	dilation	(Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
[in]	act_info	(Optional) Activation layer information in case of a fused activation.
[in]	enable_fast_math	(Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation available which may introduce a drop of accuracy as well. Default is false

Returns

the Convolution Method Hint

Definition at line 165 of file NEConvolutionLayer.cpp.

{
    return cpu::CpuConv2d::get_convolution_method(input, weights, output, conv_info, weights_info, dilation, act_info,
                                                  enable_fast_math);
}

References arm_compute::test::validation::act_info, arm_compute::test::validation::conv_info, CpuConv2d::get_convolution_method(), arm_compute::test::validation::input, and arm_compute::test::validation::weights_info.

operator=() [1/2]

NEConvolutionLayer& operator= ( const NEConvolutionLayer &  )

delete

Prevent instances of this class from being copied (As this class contains pointers)

operator=() [2/2]

NEConvolutionLayer& operator= ( NEConvolutionLayer &&  )

default

Prevent instances of this class from being moved (As this class contains non movable objects)

prepare()

void prepare ( )

overridevirtual

Prepare the function for executing.

Any one off pre-processing step required by the function is handled here

Note

Prepare stage might not need all the function's buffers' backing memory to be available in order to execute

Reimplemented from IFunction.

Definition at line 194 of file NEConvolutionLayer.cpp.

{
    if (_impl->func)
    {
        _impl->func->prepare();
    }
    else
    {
        _impl->op->prepare(_impl->prep_pack);
 
        // Release temporary tensors that are only used in prepare stage
        release_temporaries<Tensor>(_impl->aux_mem_req, _impl->workspace);
    }
}

Referenced by NEDeconvolutionLayer::prepare(), and NEConvolutionLayer::run().

run()

void run ( )

overridevirtual

Run the kernels contained in the function.

For CPU kernels:

• Multi-threading is used for the kernels which are parallelisable.
• By default std::thread::hardware_concurrency() threads are used.

Note

CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

• All the kernels are enqueued on the queue associated with CLScheduler.
• The queue is then flushed.

Note

The function will not block until the kernels are executed. It is the user's responsibility to wait.

Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 178 of file NEConvolutionLayer.cpp.

{
    prepare();
 
    MemoryGroupResourceScope scope_mg(_impl->memory_group);
 
    if (_impl->func)
    {
        _impl->func->run();
    }
    else
    {
        _impl->op->run(_impl->run_pack);
    }
}

References NEConvolutionLayer::prepare().

Referenced by NEDeconvolutionLayer::run().

validate()

Status validate ( const ITensorInfo *  input, const ITensorInfo *  weights, const ITensorInfo *  biases, const ITensorInfo *  output, const PadStrideInfo &  conv_info, const WeightsInfo &  weights_info = WeightsInfo(), const Size2D &  dilation = Size2D(1U, 1U), const ActivationLayerInfo &  act_info = ActivationLayerInfo(), bool  enable_fast_math = false, unsigned int  num_groups = 1  )

static

Static function to check if given info will lead to a valid configuration of NEConvolutionLayer.

Parameters

[in]	input	Source tensor. 3 lower dimensions represent a single input [width, height, IFM], while every optional dimension from 4 and above represent a batch of inputs. Data types supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
[in]	weights	Weights tensor. Weights are 4D tensor with dimensions [kernel_x, kernel_y, IFM, OFM]. Data type supported:Same as input, also could be QSYMM8_PER_CHANNEL if input is QASYMM8/QASYMM8_SIGNED.
[in]	biases	Biases tensor. Shared biases supported. Biases are 1D tensor with dimensions [OFM]. Data type supported: Same as input, except for input of QASYMM8/QASYMM8_SIGNED type where biases should be of S32 type.
[in]	output	Destination tensor. 3 lower dimensions represent a single output [width, height, OFM], while the rest represent batch of outputs. Data types supported: Same as input.
[in]	conv_info	Contains padding and stride information described in PadStrideInfo.
[in]	weights_info	Specifies if the weights tensor has been reshaped with NEWeightsReshapeKernel. If this is not part of the fully connected layer the weights tensor has also been transposed with cpu::kernels::CpuGemmTranspose1xWKernel. Data type supported: Same as input.
[in]	dilation	(Optional) Dilation, in elements, across x and y. Defaults to (1, 1).
[in]	act_info	(Optional) Activation layer information in case of a fused activation.
[in]	enable_fast_math	(Optional) Enable fast math computation. In case this flag were set, the function could dispatch the fastest implementation available which may introduce a drop of accuracy as well. Default is false
[in]	num_groups	(Optional) Number of groups when performing a grouped convolution. num_groups != 1 is not supported

Returns

a status

Definition at line 121 of file NEConvolutionLayer.cpp.

{
    const Conv2dInfo info(conv_info, dilation, act_info, enable_fast_math, num_groups);
 
    ARM_COMPUTE_RETURN_ERROR_ON_MSG(!weights->are_values_constant(), "Dynamic weights are not supported");
 
    // Biases with dynamic values are not supported with quantized inputs.
    if (biases)
    {
        ARM_COMPUTE_RETURN_ERROR_ON_MSG((!biases->are_values_constant() && is_data_type_quantized(input->data_type())),
                                        "Dynamic Biases are not supported with quantized input data.");
    }
 
    switch (cpu::CpuConv2d::get_convolution_method(input, weights, output, conv_info, weights_info, dilation, act_info,
                                                   enable_fast_math))
    {
        case ConvolutionMethod::WINOGRAD:
        case ConvolutionMethod::GEMM:
        case ConvolutionMethod::GEMM_CONV2D:
        case ConvolutionMethod::DIRECT:
            ARM_COMPUTE_RETURN_ON_ERROR(cpu::CpuConv2d::validate(input, weights, biases, output, conv_info,
                                                                 weights_info, dilation, act_info, enable_fast_math,
                                                                 num_groups));
            break;
        case ConvolutionMethod::FFT:
            ARM_COMPUTE_RETURN_ON_ERROR(
                NEFFTConvolutionLayer::validate(input, weights, biases, output, conv_info, act_info));
            break;
        default:
            ARM_COMPUTE_ERROR("Not supported.");
            break;
    }
    return Status{};
}

References arm_compute::test::validation::act_info, ITensorInfo::are_values_constant(), ARM_COMPUTE_ERROR, ARM_COMPUTE_RETURN_ERROR_ON_MSG, ARM_COMPUTE_RETURN_ON_ERROR, arm_compute::test::validation::conv_info, arm_compute::DIRECT, arm_compute::FFT, arm_compute::GEMM, arm_compute::GEMM_CONV2D, CpuConv2d::get_convolution_method(), arm_compute::test::validation::info, arm_compute::test::validation::input, arm_compute::is_data_type_quantized(), arm_compute::test::validation::num_groups, NEFFTConvolutionLayer::validate(), CpuConv2d::validate(), arm_compute::test::validation::weights_info, and arm_compute::WINOGRAD.

Referenced by NEConvolutionLayer::configure(), and NEDeconvolutionLayer::validate().

---

The documentation for this class was generated from the following files:

• arm_compute/runtime/NEON/functions/NEConvolutionLayer.h
• src/runtime/NEON/functions/NEConvolutionLayer.cpp