Compute Library
 19.08
NERNNLayer Class Reference

Basic function to run NERNNLayer. More...

#include <NERNNLayer.h>

Collaboration diagram for NERNNLayer:
[legend]

Public Member Functions

 NERNNLayer (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Default constructor. More...
 
 NERNNLayer (const NERNNLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 NERNNLayer (NERNNLayer &&)=default
 Default move constructor. More...
 
NERNNLayeroperator= (const NERNNLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NERNNLayeroperator= (NERNNLayer &&)=default
 Default move assignment operator. More...
 
void configure (const ITensor *input, const ITensor *weights, const ITensor *recurrent_weights, const ITensor *bias, ITensor *hidden_state, ITensor *output, ActivationLayerInfo &info)
 Initialize the function. More...
 
void run () override
 Run the kernels contained in the function. More...
 
void prepare () override
 Prepare the function for executing. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *recurrent_weights, const ITensorInfo *bias, const ITensorInfo *hidden_state, const ITensorInfo *output, const ActivationLayerInfo &info)
 Initialize the function. More...
 

Detailed Description

Basic function to run NERNNLayer.

Definition at line 41 of file NERNNLayer.h.

Constructor & Destructor Documentation

◆ NERNNLayer() [1/3]

NERNNLayer ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Default constructor.

Definition at line 36 of file NERNNLayer.cpp.

37  : _memory_group(std::move(memory_manager)), _gemm_state_f(), _add_kernel(), _activation_kernel(), _fully_connected_kernel(), _copy_kernel(), _fully_connected_out(), _gemm_output(), _add_output(),
38  _is_prepared(false)
39 {
40 }

◆ NERNNLayer() [2/3]

NERNNLayer ( const NERNNLayer )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ NERNNLayer() [3/3]

NERNNLayer ( NERNNLayer &&  )
default

Default move constructor.

Member Function Documentation

◆ configure()

void configure ( const ITensor input,
const ITensor weights,
const ITensor recurrent_weights,
const ITensor bias,
ITensor hidden_state,
ITensor output,
ActivationLayerInfo info 
)

Initialize the function.

Parameters
[in]inputInput is a 2-D tensor of shape [input_size, batch_size]. Data types supported: F16/F32
[in]weightsWeights tensor of shape [input_size, num_units] that multiplies the input. Data types supported: Same as input
[in]recurrent_weightsWeights tensor of shape [num_units, num_units] that multiplies the current 'state'. Data types supported: Same as input
[in]biasBias vector of shape [num_units]. Data types supported: Same as input
[out]outputOutput tensor of shape [num_units, batch_size]. Data types supported: Same as input
[in,out]hidden_stateOutput tensor of shape [num_units, batch_size]. Data types supported: Same as input
[in]infoActivation layer parameter.

Definition at line 67 of file NERNNLayer.cpp.

69 {
70  ARM_COMPUTE_ERROR_ON_NULLPTR(input, weights, recurrent_weights, bias, hidden_state, output);
71  ARM_COMPUTE_ERROR_THROW_ON(NERNNLayer::validate(input->info(), weights->info(), recurrent_weights->info(), bias->info(), hidden_state->info(), output->info(), info));
72 
73  const int idx_height = get_data_layout_dimension_index(input->info()->data_layout(), DataLayoutDimension::HEIGHT);
74  TensorShape shape = misc::shape_calculator::compute_rnn_shape(recurrent_weights->info(), hidden_state->info()->dimension(idx_height));
75 
76  _is_prepared = false;
77 
78  // Manage intermediate buffers and configure
79  _fully_connected_out.allocator()->init(TensorInfo(shape, 1, input->info()->data_type()));
80  _gemm_output.allocator()->init(TensorInfo(shape, 1, input->info()->data_type()));
81 
82  // Manage intermediate buffers and configure
83  _memory_group.manage(&_fully_connected_out);
84  _fully_connected_kernel.configure(input, weights, bias, &_fully_connected_out);
85 
86  _memory_group.manage(&_gemm_output);
87  _gemm_state_f.configure(hidden_state, recurrent_weights, nullptr, &_gemm_output, 1.f, 0.f);
88 
89  _add_output.allocator()->init(TensorInfo(shape, 1, input->info()->data_type()));
90  _memory_group.manage(&_add_output);
91 
92  _add_kernel.configure(&_fully_connected_out, &_gemm_output, &_add_output, ConvertPolicy::SATURATE);
93 
94  _fully_connected_out.allocator()->allocate();
95  _gemm_output.allocator()->allocate();
96 
97  _activation_kernel.configure(&_add_output, hidden_state, info);
98  _add_output.allocator()->allocate();
99 
100  _copy_kernel.configure(hidden_state, output);
101 }
TensorInfo * info() const override
Interface to be implemented by the child class to return the tensor's metadata.
Definition: CLTensor.cpp:35
void init(const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
Shares the same backing memory with another tensor allocator, while the tensor info might be differen...
void configure(ITensor *input, ITensor *output, ActivationLayerInfo activation_info)
Set the input and output tensor.
void configure(const ITensor *input1, const ITensor *input2, ITensor *output, ConvertPolicy policy)
Initialise the kernel's input, output and border mode.
TensorShape compute_rnn_shape(const ITensorInfo *input, const unsigned int batch_size)
Calculate the RNN shape of a tensor.
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:327
TensorAllocator * allocator()
Return a pointer to the tensor's allocator.
Definition: Tensor.cpp:48
void manage(TensorType *obj)
Sets a object to be managed by the given memory group.
void configure(const ITensor *input, const ITensor *weights, const ITensor *biases, ITensor *output, FullyConnectedLayerInfo fc_info=FullyConnectedLayerInfo())
Set the input and output tensors.
void allocate() override
Allocate size specified by TensorInfo of CPU memory.
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:161
void configure(const ITensor *a, const ITensor *b, const ITensor *c, ITensor *d, float alpha, float beta, const GEMMInfo &gemm_info=GEMMInfo())
Initialise the kernel's inputs, output.
Definition: NEGEMM.cpp:51
static Status validate(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *recurrent_weights, const ITensorInfo *bias, const ITensorInfo *hidden_state, const ITensorInfo *output, const ActivationLayerInfo &info)
Initialize the function.
Definition: NERNNLayer.cpp:42
size_t get_data_layout_dimension_index(const DataLayout data_layout, const DataLayoutDimension data_layout_dimension)
Get the index of the given dimension.
Definition: Helpers.inl:326
void configure(const ITensor *input, ITensor *output)
Initialize the kernel's input, output.

References TensorAllocator::allocate(), Tensor::allocator(), ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, arm_compute::test::validation::bias, arm_compute::misc::shape_calculator::compute_rnn_shape(), NECopyKernel::configure(), NEActivationLayerKernel::configure(), NEArithmeticAdditionKernel::configure(), NEGEMM::configure(), NEFullyConnectedLayer::configure(), ITensorInfo::data_layout(), ITensorInfo::data_type(), ITensorInfo::dimension(), arm_compute::get_data_layout_dimension_index(), arm_compute::HEIGHT, ITensor::info(), CLTensor::info(), arm_compute::test::validation::info, TensorAllocator::init(), MemoryGroupBase< TensorType >::manage(), arm_compute::SATURATE, arm_compute::test::validation::shape, NERNNLayer::validate(), and arm_compute::test::validation::weights.

◆ operator=() [1/2]

NERNNLayer& operator= ( const NERNNLayer )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

NERNNLayer& operator= ( NERNNLayer &&  )
default

Default move assignment operator.

◆ prepare()

void prepare ( )
overridevirtual

Prepare the function for executing.

Any one off pre-processing step required by the function is handled here

Note
Prepare stage might not need all the function's buffers' backing memory to be available in order to execute

Reimplemented from IFunction.

Definition at line 120 of file NERNNLayer.cpp.

121 {
122  if(!_is_prepared)
123  {
124  _fully_connected_kernel.prepare();
125  _gemm_state_f.prepare();
126 
127  _is_prepared = true;
128  }
129 }
void prepare() override
Prepare the function for executing.
Definition: NEGEMM.cpp:273
void prepare() override
Prepare the function for executing.

References NEGEMM::prepare(), and NEFullyConnectedLayer::prepare().

Referenced by NERNNLayer::run().

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For NEON kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 103 of file NERNNLayer.cpp.

104 {
105  prepare();
106 
107  MemoryGroupResourceScope scope_mg(_memory_group);
108 
109  _fully_connected_kernel.run();
110 
111  _gemm_state_f.run();
112 
113  NEScheduler::get().schedule(&_add_kernel, Window::DimY);
114  NEScheduler::get().schedule(&_activation_kernel, Window::DimY);
115 
116  // copy hidden out to output
117  NEScheduler::get().schedule(&_copy_kernel, Window::DimY);
118 }
void prepare() override
Prepare the function for executing.
Definition: NERNNLayer.cpp:120
void run() override
Run the kernels contained in the function.
Definition: NEGEMM.cpp:239
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
virtual void schedule(ICPPKernel *kernel, const Hints &hints)=0
Runs the kernel in the same thread as the caller synchronously.
void run() override
Run the kernels contained in the function.
static IScheduler & get()
Access the scheduler singleton.
Definition: Scheduler.cpp:96

References Window::DimY, Scheduler::get(), NERNNLayer::prepare(), NEGEMM::run(), NEFullyConnectedLayer::run(), and IScheduler::schedule().

◆ validate()

Status validate ( const ITensorInfo input,
const ITensorInfo weights,
const ITensorInfo recurrent_weights,
const ITensorInfo bias,
const ITensorInfo hidden_state,
const ITensorInfo output,
const ActivationLayerInfo info 
)
static

Initialize the function.

Parameters
[in]inputInput is a 2-D tensor of shape [input_size, batch_size]. Data types supported: F16/F32
[in]weightsWeights tensor of shape [input_size, num_units] that multiplies the input. Data types supported: Same as input
[in]recurrent_weightsWeights tensor of shape [num_units, num_units] that multiplies the current 'state'. Data types supported: Same as input
[in]biasBias vector of shape [num_units]. Data types supported: Same as input
[in]outputOutput tensor of shape [num_units, batch_size]. Data types supported: Same as input
[in]hidden_stateOutput tensor of shape [num_units, batch_size]. Data types supported: Same as input
[in]infoActivation layer parameter.
Returns
a status

Definition at line 42 of file NERNNLayer.cpp.

44 {
45  ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(input, weights, recurrent_weights, bias, hidden_state, output);
46 
47  const int idx_width = get_data_layout_dimension_index(input->data_layout(), DataLayoutDimension::WIDTH);
48  const int idx_height = get_data_layout_dimension_index(input->data_layout(), DataLayoutDimension::HEIGHT);
49  ARM_COMPUTE_RETURN_ERROR_ON(input->dimension(idx_width) != weights->dimension(idx_width));
50  ARM_COMPUTE_RETURN_ERROR_ON(weights->dimension(idx_height) != recurrent_weights->dimension(idx_width));
51  ARM_COMPUTE_RETURN_ERROR_ON(recurrent_weights->dimension(idx_width) != recurrent_weights->dimension(idx_height));
52  ARM_COMPUTE_RETURN_ERROR_ON(bias->num_dimensions() != 1);
53  ARM_COMPUTE_RETURN_ERROR_ON(bias->dimension(idx_width) != weights->dimension(idx_height));
54  ARM_COMPUTE_RETURN_ERROR_ON(hidden_state->dimension(idx_width) != weights->dimension(idx_height));
55  ARM_COMPUTE_RETURN_ERROR_ON(hidden_state->dimension(idx_height) != input->dimension(idx_height));
56  ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DIMENSIONS(output->tensor_shape(), hidden_state->tensor_shape());
57 
58  auto shape_info = TensorInfo(misc::shape_calculator::compute_rnn_shape(recurrent_weights, hidden_state->dimension(idx_height)), 1, input->data_type());
59 
63 
64  return Status{};
65 }
static Status validate(const ITensorInfo *input1, const ITensorInfo *input2, const ITensorInfo *output, ConvertPolicy policy)
Static function to check if given info will lead to a valid configuration of NEArithmeticAdditionKern...
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:193
TensorShape compute_rnn_shape(const ITensorInfo *input, const unsigned int batch_size)
Calculate the RNN shape of a tensor.
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:244
#define ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DIMENSIONS(...)
Definition: Validate.h:288
#define ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(...)
Definition: Validate.h:163
static Status validate(const ITensorInfo *input, const ITensorInfo *output, const ActivationLayerInfo &act_info)
Static function to check if given info will lead to a valid configuration of NEActivationLayerKernel.
static Status validate(const ITensorInfo *input, const ITensorInfo *weights, const ITensorInfo *biases, const ITensorInfo *output, FullyConnectedLayerInfo fc_info=FullyConnectedLayerInfo())
Static function to check if given info will lead to a valid configuration of NEFullyConnectedLayer.
size_t get_data_layout_dimension_index(const DataLayout data_layout, const DataLayoutDimension data_layout_dimension)
Get the index of the given dimension.
Definition: Helpers.inl:326

References ARM_COMPUTE_RETURN_ERROR_ON, ARM_COMPUTE_RETURN_ERROR_ON_MISMATCHING_DIMENSIONS, ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR, ARM_COMPUTE_RETURN_ON_ERROR, arm_compute::test::validation::bias, arm_compute::misc::shape_calculator::compute_rnn_shape(), ITensorInfo::data_layout(), ITensorInfo::data_type(), ITensorInfo::dimension(), arm_compute::get_data_layout_dimension_index(), arm_compute::HEIGHT, arm_compute::test::validation::info, arm_compute::SATURATE, ITensorInfo::tensor_shape(), NEActivationLayerKernel::validate(), NEArithmeticAdditionKernel::validate(), NEFullyConnectedLayer::validate(), arm_compute::test::validation::weights, and arm_compute::WIDTH.

Referenced by NERNNLayer::configure().


The documentation for this class was generated from the following files: