NEFullyConnectedLayer Class Reference

Basic function to compute a Fully Connected layer. More...

#include <NEFullyConnectedLayer.h>

Collaboration diagram for NEFullyConnectedLayer:

Public Member Functions
	NEFullyConnectedLayer (std::shared_ptr< IMemoryManager > memory_manager=nullptr, IWeightsManager *weights_manager=nullptr)
	Constructor. More...
	NEFullyConnectedLayer (const NEFullyConnectedLayer &)=delete
	Prevent instances of this class from being copied (As this class contains pointers) More...
	NEFullyConnectedLayer (NEFullyConnectedLayer &&)=delete
	Prevent instances of this class from being moved (As this class contains pointers) More...
NEFullyConnectedLayer &	operator= (const NEFullyConnectedLayer &)=delete
	Prevent instances of this class from being copied (As this class contains pointers) More...
NEFullyConnectedLayer &	operator= (NEFullyConnectedLayer &&)=delete
	Prevent instances of this class from being moved (As this class contains pointers) More...
	~NEFullyConnectedLayer ()
	Default destructor. More...
void	configure (const ITensor *input, const ITensor *weights, const ITensor *biases, ITensor *output, FullyConnectedLayerInfo fc_info=FullyConnectedLayerInfo(), const WeightsInfo &weights_info=WeightsInfo())
	Set the input and output tensors. More...
void	run () override
	Run the kernels contained in the function. More...
void	prepare () override
	Prepare the function for executing. More...
Public Member Functions inherited from IFunction
virtual	~IFunction ()=default
	Destructor. More...

Static Public Member Functions
static Status	validate (const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, FullyConnectedLayerInfo fc_info=FullyConnectedLayerInfo(), const WeightsInfo &weights_info=WeightsInfo())
	Static function to check if given info will lead to a valid configuration of NEFullyConnectedLayer. More...
static Status	has_opt_impl (arm_compute::WeightFormat &expected_weight_format, const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, const FullyConnectedLayerInfo &fc_info, const WeightsInfo &weights_info)
	Static function that queries whether fixed-format kernel exists for a given problem description. More...

Detailed Description

Basic function to compute a Fully Connected layer.

This function calls the following kernels:

1. cpu::kernels::CpuIm2ColKernel (called when the input comes from a convolutional layer)
2. NETranspose (if are_weights_reshaped is set to false and transpose_weights is set to true ) (called once)
3. NEGEMM or NEGEMMLowpMatrixMultiplyCore (if quantized asymmetric)
4. cpu::kernels::CpuGemmMatrixAdditionKernel or NEGEMMLowpOutputStage (if quantized asymmetric) (if biases is not equal to nullptr)

Note

The fully connected layer accepts "weights" tensors only with 2 dimensions.

Definition at line 86 of file NEFullyConnectedLayer.h.

Constructor & Destructor Documentation

NEFullyConnectedLayer() [1/3]

NEFullyConnectedLayer	(	std::shared_ptr< IMemoryManager >	memory_manager = nullptr,
		IWeightsManager *	weights_manager = nullptr
	)

Constructor.

Definition at line 56 of file NEFullyConnectedLayer.cpp.

    : _impl(std::make_unique<Impl>())
{
    _impl->memory_group    = MemoryGroup(std::move(memory_manager));
    _impl->weights_manager = weights_manager;
}

NEFullyConnectedLayer() [2/3]

NEFullyConnectedLayer ( const NEFullyConnectedLayer &  )

delete

Prevent instances of this class from being copied (As this class contains pointers)

NEFullyConnectedLayer() [3/3]

NEFullyConnectedLayer ( NEFullyConnectedLayer &&  )

delete

Prevent instances of this class from being moved (As this class contains pointers)

~NEFullyConnectedLayer()

~NEFullyConnectedLayer ( )

default

Default destructor.

Member Function Documentation

configure()

void configure	(	const ITensor *	input,
		const ITensor *	weights,
		const ITensor *	biases,
		ITensor *	output,
		FullyConnectedLayerInfo	fc_info = FullyConnectedLayerInfo(),
		const WeightsInfo &	weights_info = WeightsInfo()
	)

Set the input and output tensors.

Valid data layouts:

• NHWC
• NCHW

Valid data type configurations:

src0	src1	src2	dst
F16	F16	F16	F16
F32	F32	F32	F32
QASYMM8	QASYMM8	S32	QASYMM8
QASYMM8_SIGNED	QASYMM8_SIGNED	S32	QASYMM8_SIGNED

Parameters

[in]	input	Source tensor. Data type supported: QASYMM8/QASYMM8_SIGNED/F16/F32.
[in]	weights	Weights tensor. The weights must be 2 dimensional. If this function is called after a Convolution Layer, the (transposed) weights will have as many rows as the product of the first 3 input's dimensions. If it is called after another FullyConnected Layer, the (transposed) weights will have as many rows as the input's first dimension. Data type supported: Same as input.
[in]	biases	Bias tensor. Can be nullptr. Data type supported: Same as weights, S32 if weights is QASYMM8/QASYMM8_SIGNED.
[out]	output	Destination tensor. Its shape should be equal to the output of a matrix multiplication between: The output of im2col on the input and the (transposed) 2D weights, if the function is called after a Convolution Layer The input tensor and the (transposed) 2D weights, if the function is called after another FullyConnected Layer. Data type supported: Same as input.
[in]	fc_info	(Optional) Fully connected layer additional info
[in]	weights_info	(Optional) Stores neccessary compute information when weights are already reshaped

Definition at line 63 of file NEFullyConnectedLayer.cpp.

References arm_compute::ACL_DST, arm_compute::ACL_SRC_0, arm_compute::ACL_SRC_1, arm_compute::ACL_SRC_2, ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, ARM_COMPUTE_LOG_PARAMS, ITensor::info(), NEFullyConnectedLayer::validate(), and arm_compute::test::validation::weights_info.

Referenced by NERNNLayer::configure(), and NELSTMLayer::configure().

{
    // Perform validate step
    ARM_COMPUTE_ERROR_ON_NULLPTR(input, weights, output);
    ARM_COMPUTE_ERROR_THROW_ON(NEFullyConnectedLayer::validate(input->info(),
                                                               weights->info(),
                                                               biases != nullptr ? biases->info() : nullptr,
                                                               output->info(),
                                                               fc_info,
                                                               weights_info));
    ARM_COMPUTE_LOG_PARAMS(input, weights, biases, output, fc_info);

    _impl->op               = std::make_unique<cpu::CpuFullyConnected>();
    _impl->original_weights = weights;
    _impl->is_prepared      = false;

    _impl->op->configure(input->info(), weights->info(), (biases != nullptr) ? biases->info() : nullptr, output->info(), fc_info, weights_info);

    if(_impl->weights_manager != nullptr)
    {
        _impl->weights_manager->manage(_impl->original_weights);
    }

    _impl->aux_mem_req = _impl->op->workspace();
    _impl->run_pack    = { { ACL_SRC_0, input }, { ACL_SRC_1, weights }, { ACL_SRC_2, biases }, { ACL_DST, output } };
    _impl->workspace   = manage_workspace<Tensor>(_impl->aux_mem_req, _impl->memory_group, _impl->run_pack, _impl->run_pack);
}

has_opt_impl()

Status has_opt_impl ( arm_compute::WeightFormat &  expected_weight_format, const ITensorInfo *  input, const ITensorInfo *  weights, const ITensorInfo *  biases, const ITensorInfo *  output, const FullyConnectedLayerInfo &  fc_info, const WeightsInfo &  weights_info  )

static

Static function that queries whether fixed-format kernel exists for a given problem description.

Parameters

[out]	expected_weight_format	Format in which weights should be for found fixed format kernel
[in]	input	Source tensor
[in]	weights	Weights tensor.
[in]	biases	Bias tensor. Can be nullptr. Data type supported: Same as weights, S32 if weights is QASYMM8/QASYMM8_SIGNED.
[in]	output	Destination tensor
[in]	fc_info	Fully connected layer additional info
[in]	weights_info	Describes weights shape

Returns

a status

Definition at line 92 of file NEFullyConnectedLayer.cpp.

References CpuFullyConnected::has_opt_impl().

{
    return cpu::CpuFullyConnected::has_opt_impl(expected_weight_format, input, weights, biases, output, fc_info, weights_info);
}

operator=() [1/2]

NEFullyConnectedLayer& operator= ( const NEFullyConnectedLayer &  )

delete

Prevent instances of this class from being copied (As this class contains pointers)

operator=() [2/2]

NEFullyConnectedLayer& operator= ( NEFullyConnectedLayer &&  )

delete

Prevent instances of this class from being moved (As this class contains pointers)

prepare()

void prepare ( )

overridevirtual

Prepare the function for executing.

Any one off pre-processing step required by the function is handled here

Note

Prepare stage might not need all the function's buffers' backing memory to be available in order to execute

Reimplemented from IFunction.

Definition at line 113 of file NEFullyConnectedLayer.cpp.

References ITensor::is_used().

Referenced by NERNNLayer::prepare(), and NEFullyConnectedLayer::run().

{
    if(!_impl->is_prepared)
    {
        _impl->op->prepare(_impl->run_pack);

        // Release temporary tensors that are only used in prepare stage
        release_temporaries<Tensor>(_impl->aux_mem_req, _impl->workspace);
        _impl->is_prepared = true;

        // Handle weights managed infrastructure
        if(_impl->weights_manager != nullptr && _impl->weights_manager->are_weights_managed(_impl->original_weights))
        {
            // Ensure that b gets marked as unused (memory released) only after the last function which uses b also finishes its prepare
            // This is for cases where multiple functions share the same b (weights)
            // Therefore when a function marks original b as unused, we pre-mark it in weights manager, and mark it back to used so that it doesn't get released before its last reference
            const ITensor *original_b = _impl->original_weights;
            if(!original_b->is_used())
            {
                _impl->weights_manager->pre_mark_as_unused(original_b);
            }
            _impl->original_weights->mark_as_used();
            _impl->weights_manager->release(_impl->original_weights);
        }
    }
}

run()

void run ( )

overridevirtual

Run the kernels contained in the function.

For CPU kernels:

• Multi-threading is used for the kernels which are parallelisable.
• By default std::thread::hardware_concurrency() threads are used.

Note

CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

• All the kernels are enqueued on the queue associated with CLScheduler.
• The queue is then flushed.

Note

The function will not block until the kernels are executed. It is the user's responsibility to wait.

Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 105 of file NEFullyConnectedLayer.cpp.

References NEFullyConnectedLayer::prepare().

Referenced by NERNNLayer::run(), and NELSTMLayer::run().

{
    prepare();

    MemoryGroupResourceScope scope_mg(_impl->memory_group);
    _impl->op->run(_impl->run_pack);
}

validate()

Status validate ( const ITensorInfo *  input, const ITensorInfo *  weights, const ITensorInfo *  biases, const ITensorInfo *  output, FullyConnectedLayerInfo  fc_info = FullyConnectedLayerInfo(), const WeightsInfo &  weights_info = WeightsInfo()  )

static

Static function to check if given info will lead to a valid configuration of NEFullyConnectedLayer.

Similar to NEFullyConnectedLayer::configure()

Returns

a status

Definition at line 99 of file NEFullyConnectedLayer.cpp.

References CpuFullyConnected::validate().

Referenced by NEFullyConnectedLayer::configure(), arm_compute::test::validation::DATA_TEST_CASE(), NERNNLayer::validate(), and NELSTMLayer::validate().

{
    return cpu::CpuFullyConnected::validate(input, weights, biases, output, fc_info, weights_info);
}

---

The documentation for this class was generated from the following files:

• arm_compute/runtime/NEON/functions/NEFullyConnectedLayer.h
• src/runtime/NEON/functions/NEFullyConnectedLayer.cpp