Compute Library
 21.02
NELSTMLayer.h
Go to the documentation of this file.
1 /*
2  * Copyright (c) 2018-2020 Arm Limited.
3  *
4  * SPDX-License-Identifier: MIT
5  *
6  * Permission is hereby granted, free of charge, to any person obtaining a copy
7  * of this software and associated documentation files (the "Software"), to
8  * deal in the Software without restriction, including without limitation the
9  * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
10  * sell copies of the Software, and to permit persons to whom the Software is
11  * furnished to do so, subject to the following conditions:
12  *
13  * The above copyright notice and this permission notice shall be included in all
14  * copies or substantial portions of the Software.
15  *
16  * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17  * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18  * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19  * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
22  * SOFTWARE.
23  */
24 #ifndef ARM_COMPUTE_NELSTMLAYER_H
25 #define ARM_COMPUTE_NELSTMLAYER_H
26 
27 #include "arm_compute/core/Types.h"
39 
40 namespace arm_compute
41 {
42 // Forward declarations
43 class ITensor;
44 
45 /** Basic function to run @ref NELSTMLayer */
46 class NELSTMLayer : public IFunction
47 {
48 public:
49  /** Default constructor */
50  NELSTMLayer(std::shared_ptr<IMemoryManager> memory_manager = nullptr);
51  /** Prevent instances of this class from being copied (As this class contains pointers) */
52  NELSTMLayer(const NELSTMLayer &) = delete;
53  /** Prevent instances of this class from being copied (As this class contains pointers) */
54  NELSTMLayer &operator=(const NELSTMLayer &) = delete;
55  /** Prevent instances of this class from being moved (As this class contains non movable objects) */
56  NELSTMLayer(NELSTMLayer &&) = delete;
57  /** Prevent instances of this class from being moved (As this class contains non movable objects) */
58  NELSTMLayer &operator=(NELSTMLayer &&) = delete;
59  /** Default destructor */
60  ~NELSTMLayer();
61  /** Initialize function's tensors.
62  *
63  * @param[in] input Source tensor. Input is a 2D tensor with dimensions [input_size, batch_size]. Data types supported: F16/F32.
64  * @param[in] input_to_forget_weights 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
65  * @param[in] input_to_cell_weights 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
66  * @param[in] input_to_output_weights 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
67  * @param[in] recurrent_to_forget_weights 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
68  * @param[in] recurrent_to_cell_weights 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
69  * @param[in] recurrent_to_output_weights 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
70  * @param[in] forget_gate_bias 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
71  * @param[in] cell_bias 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
72  * @param[in] output_gate_bias 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
73  * @param[in] output_state_in 2D weights tensor with dimensions [output_size, batch_size]. Data type supported: Same as @p input.
74  * @param[in] cell_state_in 2D tensor with dimensions [num_units, batch_size]. Data type supported: Same as @p input.
75  * @param[out] scratch_buffer 2D tensor with dimensions [num_units * 4, batch_size] with CIFG or [num_units * 3, batch_size] without CIGF. Data type supported: Same as @p input.
76  * @param[out] output_state_out 2D weights tensor with dimensions [output_size, batch_size]. Data type supported: Same as @p input.
77  * @param[out] cell_state_out 2D tensor with dimensions [num_units, batch_size]. Data type supported: Same as @p input.
78  * @param[out] output Destination tensor. Output is a 2D tensor with dimensions [output_size, batch_size].
79  * Data types supported: Same as @p input.
80  * @param[in] lstm_params Weights tensors used in peephole optimization:
81  * input_to_input_weights (Optional) 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
82  * recurrent_to_input_weights (Optional) 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
83  * cell_to_input_weights (Optional) 1D weights tensor with dimensions [num_units]. Can be nullptr. Data type supported: Same as @p input.
84  * cell_to_forget_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
85  * cell_to_output_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
86  * input_gate_bias (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input
87  * projection_weights (Optional) 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
88  * projection_bias (Optional) 1D weights tensor with dimensions [output_size]. Data type supported: Same as @p input.
89  * input_layer_norm_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
90  * forget_layer_norm_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
91  * cell_layer_norm_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
92  * output_layer_norm_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
93  * @param[in] activation_info Contains activation information described in @ref ActivationLayerInfo.
94  * @param[in] cell_threshold The clipping threshold for the cell state, such that values are bound within [-cell_clip, cell_clip]. If set to 0.0 then clipping is disabled.
95  * @param[in] projection_threshold The clipping threshold for the output from the projection layer, such that values are bound within [-proj_clip, proj_clip].
96  * If set to 0.0 then clipping is disabled.
97  */
98  void configure(const ITensor *input,
101  const ITensor *forget_gate_bias, const ITensor *cell_bias, const ITensor *output_gate_bias,
102  const ITensor *output_state_in, const ITensor *cell_state_in,
103  ITensor *scratch_buffer, ITensor *output_state_out, ITensor *cell_state_out, ITensor *output,
104  const LSTMParams<ITensor> &lstm_params, const ActivationLayerInfo &activation_info, float cell_threshold = 0.f, float projection_threshold = 0.f);
105 
106  /** Static function to check if given info will lead to a valid configuration of @ref NELSTMLayer
107  *
108  * @param[in] input Source tensor. Input is a 2D tensor with dimensions [input_size, batch_size]. Data types supported: F16/F32.
109  * @param[in] input_to_forget_weights 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
110  * @param[in] input_to_cell_weights 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
111  * @param[in] input_to_output_weights 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
112  * @param[in] recurrent_to_forget_weights 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
113  * @param[in] recurrent_to_cell_weights 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
114  * @param[in] recurrent_to_output_weights 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
115  * @param[in] forget_gate_bias 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
116  * @param[in] cell_bias 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
117  * @param[in] output_gate_bias 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
118  * @param[in] output_state_in 2D weights tensor with dimensions [output_size, batch_size]. Data type supported: Same as @p input.
119  * @param[in] cell_state_in 2D tensor with dimensions [num_units, batch_size]. Data type supported: Same as @p input.
120  * @param[in] scratch_buffer 2D tensor with dimensions [num_units * 4, batch_size] with CIFG or [num_units * 3, batch_size] without CIGF. Data type supported: Same as @p input.
121  * @param[in] output_state_out 2D weights tensor with dimensions [output_size, batch_size]. Data type supported: Same as @p input.
122  * @param[in] cell_state_out 2D tensor with dimensions [num_units, batch_size]. Data type supported: Same as @p input.
123  * @param[in] output Destination tensor. Output is a 2D tensor with dimensions [output_size, batch_size].
124  * Data types supported: Same as @p input.
125  * @param[in] lstm_params Weights tensors used in peephole optimization:
126  * input_to_input_weights (Optional) 2D weights tensor with dimensions [input_size, num_units]. Data type supported: Same as @p input.
127  * recurrent_to_input_weights (Optional) 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
128  * cell_to_input_weights (Optional) 1D weights tensor with dimensions [num_units]. Can be nullptr. Data type supported: Same as @p input.
129  * cell_to_forget_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
130  * cell_to_output_weights (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input.
131  * input_gate_bias (Optional) 1D weights tensor with dimensions [num_units]. Data type supported: Same as @p input
132  * projection_weights (Optional) 2D weights tensor with dimensions [output_size, num_units]. Data type supported: Same as @p input.
133  * projection_bias (Optional) 1D weights tensor with dimensions [output_size]. Data type supported: Same as @p input.
134  * input_layer_norm_weights (Optional) 1D weights tensor info with dimensions [num_units]. Data type supported: Same as @p input.
135  * forget_layer_norm_weights (Optional) 1D weights tensor info with dimensions [num_units]. Data type supported: Same as @p input.
136  * cell_layer_norm_weights (Optional) 1D weights tensor info with dimensions [num_units]. Data type supported: Same as @p input.
137  * output_layer_norm_weights (Optional) 1D weights tensor info with dimensions [num_units]. Data type supported: Same as @p input.
138  * @param[in] activation_info Contains activation information described in @ref ActivationLayerInfo.
139  * @param[in] cell_threshold The clipping threshold for the cell state, such that values are bound within [-cell_clip, cell_clip]. If set to 0.0 then clipping is disabled.
140  * @param[in] projection_threshold The clipping threshold for the output from the projection layer, such that values are bound within [-proj_clip, proj_clip].
141  * If set to 0.0 then clipping is disabled.
142  *
143  * @return a status
144  */
145  static Status validate(const ITensorInfo *input,
146  const ITensorInfo *input_to_forget_weights, const ITensorInfo *input_to_cell_weights, const ITensorInfo *input_to_output_weights,
147  const ITensorInfo *recurrent_to_forget_weights, const ITensorInfo *recurrent_to_cell_weights, const ITensorInfo *recurrent_to_output_weights,
148  const ITensorInfo *forget_gate_bias, const ITensorInfo *cell_bias, const ITensorInfo *output_gate_bias,
149  const ITensorInfo *output_state_in, const ITensorInfo *cell_state_in,
150  const ITensorInfo *scratch_buffer, const ITensorInfo *output_state_out, const ITensorInfo *cell_state_out, const ITensorInfo *output,
151  const LSTMParams<ITensorInfo> &lstm_params, const ActivationLayerInfo &activation_info, float cell_threshold = 0.f, float projection_threshold = 0.f);
152 
153  // Inherited methods overridden:
154  void run() override;
155  void prepare() override;
156 
157 private:
158  MemoryGroup _memory_group;
159  NEFullyConnectedLayer _fully_connected_input_gate;
160  NEArithmeticAddition _accum_input_gate1;
161  NEArithmeticSubtraction _subtract_input_gate;
162  NEPixelWiseMultiplication _pixelwise_mul_input_gate;
163  NEActivationLayer _activation_input_gate;
164  NEFullyConnectedLayer _fully_connected_forget_gate;
165  NEArithmeticAddition _accum_forget_gate1;
166  NEPixelWiseMultiplication _pixelwise_mul_forget_gate;
167  NEActivationLayer _activation_forget_gate;
168  NEFullyConnectedLayer _fully_connected_cell_state;
169  NEGEMM _gemm_cell_state1;
170  NETranspose _transpose_cell_state;
171  NEArithmeticAddition _accum_cell_state1;
172  NEArithmeticAddition _accum_cell_state2;
173  NEPixelWiseMultiplication _pixelwise_mul_cell_state1;
174  NEActivationLayer _activation_cell_state;
175  NEActivationLayer _cell_clip;
176  NEPixelWiseMultiplication _pixelwise_mul_cell_state2;
177  NEFullyConnectedLayer _fully_connected_output;
178  NEPixelWiseMultiplication _pixelwise_mul_output_state1;
179  NEArithmeticAddition _accum_output1;
180  NEActivationLayer _activation_output;
181  NEActivationLayer _activation_output_state;
182  NEPixelWiseMultiplication _pixelwise_mul_output_state2;
183  NEFullyConnectedLayer _fully_connected_output_state;
184  NEActivationLayer _projection_clip;
185  NECopy _copy_cell_state;
186  NECopy _copy_output;
187  NEConcatenateLayer _concat_scratch_buffer;
188  NEConcatenateLayer _concat_inputs_forget_gate;
189  NEConcatenateLayer _concat_weights_forget_gate;
190  NEConcatenateLayer _concat_weights_input_gate;
191  NEConcatenateLayer _concat_weights_output;
192  NEMeanStdDevNormalizationLayer _mean_std_norm_input_gate;
193  NEPixelWiseMultiplication _pixelwise_mul_input_gate_coeff;
194  NEArithmeticAddition _accum_input_gate_bias;
195  NEMeanStdDevNormalizationLayer _mean_std_norm_forget_gate;
196  NEPixelWiseMultiplication _pixelwise_mul_forget_gate_coeff;
197  NEArithmeticAddition _accum_forget_gate_bias;
198  NEMeanStdDevNormalizationLayer _mean_std_norm_cell_gate;
199  NEPixelWiseMultiplication _pixelwise_mul_cell_gate_coeff;
200  NEArithmeticAddition _accum_cell_gate_bias;
201  NEMeanStdDevNormalizationLayer _mean_std_norm_output_gate;
202  NEPixelWiseMultiplication _pixelwise_mul_output_gate_coeff;
203  NEArithmeticAddition _accum_output_gate_bias;
204  Tensor _input_gate_out1;
205  Tensor _input_gate_out2;
206  Tensor _input_gate_out3;
207  Tensor _input_gate_out4;
208  Tensor _forget_gate_out1;
209  Tensor _forget_gate_out2;
210  Tensor _forget_gate_out3;
211  Tensor _forget_gate_out4;
212  Tensor _forget_gate_out5;
213  Tensor _forget_gate_out6;
214  Tensor _cell_state_out1;
215  Tensor _cell_state_out2;
216  Tensor _cell_state_out3;
217  Tensor _cell_state_out4;
218  Tensor _cell_state_out5;
219  Tensor _output1;
220  Tensor _output2;
221  Tensor _output3;
222  Tensor _output4;
223  Tensor _cell_state_activation;
224  Tensor _output_state1;
225  Tensor _ones;
226  Tensor _input_layer_norm_out1;
227  Tensor _input_layer_norm_out2;
228  Tensor _forget_layer_norm_out1;
229  Tensor _forget_layer_norm_out2;
230  Tensor _cell_layer_norm_out1;
231  Tensor _cell_layer_norm_out2;
232  Tensor _output_layer_norm_out1;
233  Tensor _output_layer_norm_out2;
234  bool _run_peephole_opt;
235  bool _run_cifg_opt;
236  bool _perform_cell_clipping;
237  bool _has_projection_weights;
238  bool _perform_projection_clipping;
239  bool _is_prepared;
240  bool _is_layer_norm_lstm;
241 };
242 } // namespace arm_compute
243 #endif /* ARM_COMPUTE_NELSTMLAYER_H */
Base class for all functions.
Definition: IFunction.h:30
Basic function to run cpu::kernels::CpuAddKernel.
Basic function to execute GEMM on Neon.
Definition: NEGEMM.h:62
static Status validate(const ITensorInfo *input, const ITensorInfo *input_to_forget_weights, const ITensorInfo *input_to_cell_weights, const ITensorInfo *input_to_output_weights, const ITensorInfo *recurrent_to_forget_weights, const ITensorInfo *recurrent_to_cell_weights, const ITensorInfo *recurrent_to_output_weights, const ITensorInfo *forget_gate_bias, const ITensorInfo *cell_bias, const ITensorInfo *output_gate_bias, const ITensorInfo *output_state_in, const ITensorInfo *cell_state_in, const ITensorInfo *scratch_buffer, const ITensorInfo *output_state_out, const ITensorInfo *cell_state_out, const ITensorInfo *output, const LSTMParams< ITensorInfo > &lstm_params, const ActivationLayerInfo &activation_info, float cell_threshold=0.f, float projection_threshold=0.f)
Static function to check if given info will lead to a valid configuration of NELSTMLayer.
void configure(const ITensor *input, const ITensor *input_to_forget_weights, const ITensor *input_to_cell_weights, const ITensor *input_to_output_weights, const ITensor *recurrent_to_forget_weights, const ITensor *recurrent_to_cell_weights, const ITensor *recurrent_to_output_weights, const ITensor *forget_gate_bias, const ITensor *cell_bias, const ITensor *output_gate_bias, const ITensor *output_state_in, const ITensor *cell_state_in, ITensor *scratch_buffer, ITensor *output_state_out, ITensor *cell_state_out, ITensor *output, const LSTMParams< ITensor > &lstm_params, const ActivationLayerInfo &activation_info, float cell_threshold=0.f, float projection_threshold=0.f)
Initialize function&#39;s tensors.
Definition: NELSTMLayer.cpp:66
Store the tensor&#39;s metadata.
Definition: ITensorInfo.h:40
Status class.
Definition: Error.h:52
NELSTMLayer & operator=(const NELSTMLayer &)=delete
Prevent instances of this class from being copied (As this class contains pointers) ...
Activation Layer Information class.
Definition: Types.h:1550
Interface for Neon tensor.
Definition: ITensor.h:36
Copyright (c) 2017-2021 Arm Limited.
Basic function to run NELSTMLayer.
Definition: NELSTMLayer.h:46
Basic function to run cpu::kernels::CpuSubKernel.
void run() override
Run the kernels contained in the function.
Basic implementation of the tensor interface.
Definition: Tensor.h:37
Basic function to transpose a matrix on Neon.
Definition: NETranspose.h:40
NELSTMLayer(std::shared_ptr< IMemoryManager > memory_manager=nullptr)
Default constructor.
Definition: NELSTMLayer.cpp:50
void prepare() override
Prepare the function for executing.
Basic function to run cpu::kernels::CpuActivationKernel.
Basic function to execute mean and standard deviation normalization by calling NEMeanStdDevNormalizat...
Basic function to compute a Fully Connected layer on Neon.
Basic function to execute concatenate tensors along a given axis.
Basic function to run NEPixelWiseMultiplicationKernel.
~NELSTMLayer()
Default destructor.
Basic function to run cpu::kernels::CpuCopyKernel.
Definition: NECopy.h:39