CMSIS-NN  Version 4.0.0
CMSIS NN Software Library
 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages
Fully-connected Layer Functions

Functions

arm_cmsis_nn_status arm_fully_connected_mat_q7_vec_q15 (const q15_t *pV, const q7_t *pM, const uint16_t dim_vec, const uint16_t num_of_rows, const uint16_t bias_shift, const uint16_t out_shift, const q7_t *bias, q15_t *pOut, q15_t *vec_buffer)
 Mixed Q15-Q7 fully-connected layer function. More...
 
arm_cmsis_nn_status arm_fully_connected_mat_q7_vec_q15_opt (const q15_t *pV, const q7_t *pM, const uint16_t dim_vec, const uint16_t num_of_rows, const uint16_t bias_shift, const uint16_t out_shift, const q7_t *bias, q15_t *pOut, q15_t *vec_buffer)
 Mixed Q15-Q7 opt fully-connected layer function. More...
 
arm_cmsis_nn_status arm_fully_connected_q15 (const q15_t *pV, const q15_t *pM, const uint16_t dim_vec, const uint16_t num_of_rows, const uint16_t bias_shift, const uint16_t out_shift, const q15_t *bias, q15_t *pOut, q15_t *vec_buffer)
 Q15 basic fully-connected layer function. More...
 
arm_cmsis_nn_status arm_fully_connected_q15_opt (const q15_t *pV, const q15_t *pM, const uint16_t dim_vec, const uint16_t num_of_rows, const uint16_t bias_shift, const uint16_t out_shift, const q15_t *bias, q15_t *pOut, q15_t *vec_buffer)
 Q15 opt fully-connected layer function. More...
 
arm_cmsis_nn_status arm_fully_connected_q7 (const q7_t *pV, const q7_t *pM, const uint16_t dim_vec, const uint16_t num_of_rows, const uint16_t bias_shift, const uint16_t out_shift, const q7_t *bias, q7_t *pOut, q15_t *vec_buffer)
 Q7 basic fully-connected layer function. More...
 
arm_cmsis_nn_status arm_fully_connected_q7_opt (const q7_t *pV, const q7_t *pM, const uint16_t dim_vec, const uint16_t num_of_rows, const uint16_t bias_shift, const uint16_t out_shift, const q7_t *bias, q7_t *pOut, q15_t *vec_buffer)
 Q7 opt fully-connected layer function. More...
 
arm_cmsis_nn_status arm_fully_connected_s16 (const cmsis_nn_context *ctx, const cmsis_nn_fc_params *fc_params, const cmsis_nn_per_tensor_quant_params *quant_params, const cmsis_nn_dims *input_dims, const q15_t *input, const cmsis_nn_dims *filter_dims, const q7_t *kernel, const cmsis_nn_dims *bias_dims, const int64_t *bias, const cmsis_nn_dims *output_dims, q15_t *output)
 Basic s16 Fully Connected function. More...
 
int32_t arm_fully_connected_s16_get_buffer_size (const cmsis_nn_dims *filter_dims)
 Get the required buffer size for S16 basic fully-connected and matrix multiplication layer function for TF Lite. More...
 
arm_cmsis_nn_status arm_fully_connected_s8 (const cmsis_nn_context *ctx, const cmsis_nn_fc_params *fc_params, const cmsis_nn_per_tensor_quant_params *quant_params, const cmsis_nn_dims *input_dims, const q7_t *input, const cmsis_nn_dims *filter_dims, const q7_t *kernel, const cmsis_nn_dims *bias_dims, const int32_t *bias, const cmsis_nn_dims *output_dims, q7_t *output)
 Basic s8 Fully Connected function. More...
 
int32_t arm_fully_connected_s8_get_buffer_size (const cmsis_nn_dims *filter_dims)
 Get the required buffer size for S8 basic fully-connected and matrix multiplication layer function for TF Lite. More...
 

Description

Collection of fully-connected and matrix multiplication functions.

Fully-connected layer is basically a matrix-vector multiplication with bias. The matrix is the weights and the input/output vectors are the activation values. Supported {weight, activation} precisions include {8-bit, 8-bit}, {16-bit, 16-bit}, and {8-bit, 16-bit}.

Here we have two types of kernel functions. The basic function implements the function using regular GEMV approach. The opt functions operates with weights in interleaved formats.

Function Documentation

arm_cmsis_nn_status arm_fully_connected_mat_q7_vec_q15 ( const q15_t pV,
const q7_t pM,
const uint16_t  dim_vec,
const uint16_t  num_of_rows,
const uint16_t  bias_shift,
const uint16_t  out_shift,
const q7_t bias,
q15_t pOut,
q15_t vec_buffer 
)
Parameters
[in]pVpointer to input vector
[in]pMpointer to matrix weights
[in]dim_veclength of the vector
[in]num_of_rowsnumber of rows in weight matrix
[in]bias_shiftamount of left-shift for bias
[in]out_shiftamount of right-shift for output
[in]biaspointer to bias
[in,out]pOutpointer to output vector
[in,out]vec_bufferpointer to buffer space for input
Returns
The function returns ARM_CMSIS_NN_SUCCESS

References ARM_CMSIS_NN_SUCCESS, arm_nn_read_q15x2_ia(), and NN_ROUND.

arm_cmsis_nn_status arm_fully_connected_mat_q7_vec_q15_opt ( const q15_t pV,
const q7_t pM,
const uint16_t  dim_vec,
const uint16_t  num_of_rows,
const uint16_t  bias_shift,
const uint16_t  out_shift,
const q7_t bias,
q15_t pOut,
q15_t vec_buffer 
)
Parameters
[in]pVpointer to input vector
[in]pMpointer to matrix weights
[in]dim_veclength of the vector
[in]num_of_rowsnumber of rows in weight matrix
[in]bias_shiftamount of left-shift for bias
[in]out_shiftamount of right-shift for output
[in]biaspointer to bias
[in,out]pOutpointer to output vector
[in,out]vec_bufferpointer to buffer space for input
Returns
The function returns ARM_CMSIS_NN_SUCCESS

References ARM_CMSIS_NN_SUCCESS, arm_nn_read_q15x2_ia(), arm_nn_read_q7x4_ia(), and NN_ROUND.

arm_cmsis_nn_status arm_fully_connected_q15 ( const q15_t pV,
const q15_t pM,
const uint16_t  dim_vec,
const uint16_t  num_of_rows,
const uint16_t  bias_shift,
const uint16_t  out_shift,
const q15_t bias,
q15_t pOut,
q15_t vec_buffer 
)
Parameters
[in]pVpointer to input vector
[in]pMpointer to matrix weights
[in]dim_veclength of the vector
[in]num_of_rowsnumber of rows in weight matrix
[in]bias_shiftamount of left-shift for bias
[in]out_shiftamount of right-shift for output
[in]biaspointer to bias
[in,out]pOutpointer to output vector
[in,out]vec_bufferpointer to buffer space for input
Returns
The function returns ARM_CMSIS_NN_SUCCESS

References ARM_CMSIS_NN_SUCCESS, arm_nn_read_q15x2_ia(), and NN_ROUND.

arm_cmsis_nn_status arm_fully_connected_q15_opt ( const q15_t pV,
const q15_t pM,
const uint16_t  dim_vec,
const uint16_t  num_of_rows,
const uint16_t  bias_shift,
const uint16_t  out_shift,
const q15_t bias,
q15_t pOut,
q15_t vec_buffer 
)
Parameters
[in]pVpointer to input vector
[in]pMpointer to matrix weights
[in]dim_veclength of the vector
[in]num_of_rowsnumber of rows in weight matrix
[in]bias_shiftamount of left-shift for bias
[in]out_shiftamount of right-shift for output
[in]biaspointer to bias
[in,out]pOutpointer to output vector
[in,out]vec_bufferpointer to buffer space for input
Returns
The function returns ARM_CMSIS_NN_SUCCESS

References ARM_CMSIS_NN_SUCCESS, arm_nn_read_q15x2_ia(), and NN_ROUND.

arm_cmsis_nn_status arm_fully_connected_q7 ( const q7_t pV,
const q7_t pM,
const uint16_t  dim_vec,
const uint16_t  num_of_rows,
const uint16_t  bias_shift,
const uint16_t  out_shift,
const q7_t bias,
q7_t pOut,
q15_t vec_buffer 
)
Parameters
[in]pVpointer to input vector
[in]pMpointer to matrix weights
[in]dim_veclength of the vector
[in]num_of_rowsnumber of rows in weight matrix
[in]bias_shiftamount of left-shift for bias
[in]out_shiftamount of right-shift for output
[in]biaspointer to bias
[in,out]pOutpointer to output vector
[in,out]vec_bufferpointer to buffer space for input
Returns
The function returns ARM_CMSIS_NN_SUCCESS

References ARM_CMSIS_NN_SUCCESS, arm_nn_read_q15x2_ia(), arm_q7_to_q15_reordered_no_shift(), and NN_ROUND.

arm_cmsis_nn_status arm_fully_connected_q7_opt ( const q7_t pV,
const q7_t pM,
const uint16_t  dim_vec,
const uint16_t  num_of_rows,
const uint16_t  bias_shift,
const uint16_t  out_shift,
const q7_t bias,
q7_t pOut,
q15_t vec_buffer 
)
Parameters
[in]pVpointer to input vector
[in]pMpointer to matrix weights
[in]dim_veclength of the vector
[in]num_of_rowsnumber of rows in weight matrix
[in]bias_shiftamount of left-shift for bias
[in]out_shiftamount of right-shift for output
[in]biaspointer to bias
[in,out]pOutpointer to output vector
[in,out]vec_bufferpointer to buffer space for input
Returns
The function returns ARM_CMSIS_NN_SUCCESS

References ARM_CMSIS_NN_SUCCESS, arm_nn_read_q15x2_ia(), arm_nn_read_q7x4_ia(), arm_q7_to_q15_reordered_no_shift(), and NN_ROUND.

arm_cmsis_nn_status arm_fully_connected_s16 ( const cmsis_nn_context ctx,
const cmsis_nn_fc_params fc_params,
const cmsis_nn_per_tensor_quant_params quant_params,
const cmsis_nn_dims input_dims,
const q15_t input_data,
const cmsis_nn_dims filter_dims,
const q7_t filter_data,
const cmsis_nn_dims bias_dims,
const int64_t *  bias_data,
const cmsis_nn_dims output_dims,
q15_t output_data 
)
Parameters
[in,out]ctxFunction context (e.g. temporary buffer). Check the function definition file to see if an additional buffer is required. Optional function {API}_get_buffer_size() provides the buffer size if an additional buffer is required. The caller is expected to clear the buffer ,if applicable, for security reasons.
[in]fc_paramsFully Connected layer parameters. fc_params->input_offset : 0 fc_params->filter_offset : 0 fc_params->output_offset : 0
[in]quant_paramsPer-tensor quantization info. It contains the multiplier and shift values to be applied to the output tensor.
[in]input_dimsInput (activation) tensor dimensions. Format: [N, H, W, C_IN] Input dimension is taken as Nx(H * W * C_IN)
[in]input_dataInput (activation) data pointer. Data type: int16
[in]filter_dimsTwo dimensional filter dimensions. Format: [N, C] N : accumulation depth and equals (H * W * C_IN) from input_dims C : output depth and equals C_OUT in output_dims H & W : Not used
[in]filter_dataFilter data pointer. Data type: int8
[in]bias_dimsBias tensor dimensions. Format: [C_OUT] N, H, W : Not used
[in]bias_dataBias data pointer. Data type: int64
[in]output_dimsOutput tensor dimensions. Format: [N, C_OUT] N : Batches C_OUT : Output depth H & W : Not used.
[in,out]output_dataOutput data pointer. Data type: int16
Returns
The function returns ARM_CMSIS_NN_SUCCESS
  • Supported framework: TensorFlow Lite
  • q15 is used as data type eventhough it is s16 data. It is done so to be consistent with existing APIs.

References cmsis_nn_fc_params::activation, ARM_CMSIS_NN_SUCCESS, arm_nn_vec_mat_mult_t_s16(), cmsis_nn_dims::c, cmsis_nn_fc_params::filter_offset, cmsis_nn_activation::max, cmsis_nn_activation::min, cmsis_nn_per_tensor_quant_params::multiplier, cmsis_nn_dims::n, REDUCE_MULTIPLIER, and cmsis_nn_per_tensor_quant_params::shift.

int32_t arm_fully_connected_s16_get_buffer_size ( const cmsis_nn_dims filter_dims)
Parameters
[in]filter_dimsdimension of filter
Returns
The function returns required buffer size in bytes
arm_cmsis_nn_status arm_fully_connected_s8 ( const cmsis_nn_context ctx,
const cmsis_nn_fc_params fc_params,
const cmsis_nn_per_tensor_quant_params quant_params,
const cmsis_nn_dims input_dims,
const q7_t input_data,
const cmsis_nn_dims filter_dims,
const q7_t filter_data,
const cmsis_nn_dims bias_dims,
const int32_t *  bias_data,
const cmsis_nn_dims output_dims,
q7_t output_data 
)
Parameters
[in,out]ctxFunction context (e.g. temporary buffer). Check the function definition file to see if an additional buffer is required. Optional function {API}_get_buffer_size() provides the buffer size if an additional buffer is required. The caller is expected to clear the buffer ,if applicable, for security reasons.
[in]fc_paramsFully Connected layer parameters. Range of fc_params->input_offset : [-127, 128] fc_params->filter_offset : 0 Range of fc_params->output_offset : [-128, 127]
[in]quant_paramsPer-tensor quantization info. It contains the multiplier and shift values to be applied to the output tensor.
[in]input_dimsInput (activation) tensor dimensions. Format: [N, H, W, C_IN] Input dimension is taken as Nx(H * W * C_IN)
[in]input_dataInput (activation) data pointer. Data type: int8
[in]filter_dimsTwo dimensional filter dimensions. Format: [N, C] N : accumulation depth and equals (H * W * C_IN) from input_dims C : output depth and equals C_OUT in output_dims H & W : Not used
[in]filter_dataFilter data pointer. Data type: int8
[in]bias_dimsBias tensor dimensions. Format: [C_OUT] N, H, W : Not used
[in]bias_dataBias data pointer. Data type: int32
[in]output_dimsOutput tensor dimensions. Format: [N, C_OUT] N : Batches C_OUT : Output depth H & W : Not used.
[in,out]output_dataOutput data pointer. Data type: int8
Returns
The function returns ARM_CMSIS_NN_SUCCESS
  • Supported framework: TensorFlow Lite
  • q7 is used as data type eventhough it is s8 data. It is done so to be consistent with existing APIs.

References cmsis_nn_fc_params::activation, ARM_CMSIS_NN_SUCCESS, arm_nn_vec_mat_mult_t_s8(), cmsis_nn_dims::c, cmsis_nn_fc_params::filter_offset, cmsis_nn_fc_params::input_offset, cmsis_nn_activation::max, cmsis_nn_activation::min, cmsis_nn_per_tensor_quant_params::multiplier, cmsis_nn_dims::n, cmsis_nn_fc_params::output_offset, and cmsis_nn_per_tensor_quant_params::shift.

int32_t arm_fully_connected_s8_get_buffer_size ( const cmsis_nn_dims filter_dims)
Parameters
[in]filter_dimsdimension of filter
Returns
The function returns required buffer size in bytes