Compute Library
 19.08
NEGEMMLowpMatrixMultiplyKernel Class Reference

NEON kernel to multiply matrices. More...

#include <NEGEMMLowpMatrixMultiplyKernel.h>

Collaboration diagram for NEGEMMLowpMatrixMultiplyKernel:
[legend]

Public Member Functions

const char * name () const override
 Name of the kernel. More...
 
 NEGEMMLowpMatrixMultiplyKernel ()
 Constructor. More...
 
 NEGEMMLowpMatrixMultiplyKernel (const NEGEMMLowpMatrixMultiplyKernel &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NEGEMMLowpMatrixMultiplyKerneloperator= (const NEGEMMLowpMatrixMultiplyKernel &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 NEGEMMLowpMatrixMultiplyKernel (NEGEMMLowpMatrixMultiplyKernel &&)=default
 Allow instances of this class to be moved. More...
 
NEGEMMLowpMatrixMultiplyKerneloperator= (NEGEMMLowpMatrixMultiplyKernel &&)=default
 Allow instances of this class to be moved. More...
 
void configure (const ITensor *input0, const ITensor *input1, ITensor *output)
 Initialise the kernel's input and output. More...
 
void run (const Window &window, const ThreadInfo &info) override
 Execute the kernel on the passed window. More...
 
- Public Member Functions inherited from ICPPKernel
virtual ~ICPPKernel ()=default
 Default destructor. More...
 
- Public Member Functions inherited from IKernel
 IKernel ()
 Constructor. More...
 
virtual ~IKernel ()=default
 Destructor. More...
 
virtual bool is_parallelisable () const
 Indicates whether or not the kernel is parallelisable. More...
 
virtual BorderSize border_size () const
 The size of the border for that kernel. More...
 
const Windowwindow () const
 The maximum window the kernel can be executed on. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *input0, const ITensorInfo *input1, const ITensorInfo *output)
 Static function to check if given info will lead to a valid configuration of NEGEMMLowpMatrixMultiplyKernel. More...
 

Detailed Description

NEON kernel to multiply matrices.

Note
NEGEMMLowpMatrixMultiplyKernel low precision matrix product kernel This kernel performs the following computation:
  1. Convert a values from int8 to int32
  2. Convert b values from int8 to int32
  3. Compute the int32 matrix product of the resulting a * b and store the result as int32

Definition at line 43 of file NEGEMMLowpMatrixMultiplyKernel.h.

Constructor & Destructor Documentation

◆ NEGEMMLowpMatrixMultiplyKernel() [1/3]

Constructor.

Definition at line 797 of file NEGEMMLowpMatrixMultiplyKernel.cpp.

798  : _input0(nullptr), _input1(nullptr), _output(nullptr), _slide_matrix_b(true)
799 {
800 }

◆ NEGEMMLowpMatrixMultiplyKernel() [2/3]

Prevent instances of this class from being copied (As this class contains pointers)

◆ NEGEMMLowpMatrixMultiplyKernel() [3/3]

Allow instances of this class to be moved.

Member Function Documentation

◆ configure()

void configure ( const ITensor input0,
const ITensor input1,
ITensor output 
)

Initialise the kernel's input and output.

The input matrices input0 and input1 must be the output of the kernels: NEGEMMInterleave4x4Kernel and NEGEMMTranspose1xWKernel. These two kernels change the layout of the original matrices to be more cache-friendly.

Parameters
[in]input0Input tensor containing the interleaved Matrix A. Data type supported: QASYMM8
[in]input1Input tensor containing the transposed1xW Matrix B. Data type supported: same as input0
[out]outputOutput tensor to store the result of matrix multiplication. Data type supported: S32

Definition at line 802 of file NEGEMMLowpMatrixMultiplyKernel.cpp.

803 {
804  ARM_COMPUTE_ERROR_ON_NULLPTR(input0, input1, output);
805  ARM_COMPUTE_ERROR_THROW_ON(validate_arguments(input0->info(), input1->info(), output->info()));
806 
807  TensorShape in1_shape = input1->info()->tensor_shape();
808  in1_shape.collapse(2);
809 
810  _input0 = input0;
811  _input1 = input1;
812  _output = output;
813  _slide_matrix_b = in1_shape[2] != 1;
814 
815  // Configure kernel window
816  auto win_config = validate_and_configure_window(input0->info(), input1->info(), output->info());
817  ARM_COMPUTE_ERROR_THROW_ON(win_config.first);
818  INEKernel::configure(win_config.second);
819 }
Shape of a tensor.
Definition: TensorShape.h:39
std::pair< Status, Window > validate_and_configure_window(ITensorInfo *input, ITensorInfo *weights, ITensorInfo *biases, ITensorInfo *output, const PadStrideInfo &conv_info, unsigned int depth_multiplier, const Size2D &dilation)
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:327
virtual const TensorShape & tensor_shape() const =0
Size for each dimension of the tensor.
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:161
void collapse(size_t n, size_t first=0)
Collapse the first n dimensions.
Definition: TensorShape.h:132

References ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, TensorShape::collapse(), ITensor::info(), ITensorInfo::tensor_shape(), and arm_compute::validate_and_configure_window().

◆ name()

const char* name ( ) const
inlineoverridevirtual

Name of the kernel.

Returns
Kernel name

Implements ICPPKernel.

Definition at line 46 of file NEGEMMLowpMatrixMultiplyKernel.h.

47  {
48  return "NEGEMMLowpMatrixMultiplyKernel";
49  }

◆ operator=() [1/2]

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

Allow instances of this class to be moved.

◆ run()

void run ( const Window window,
const ThreadInfo info 
)
overridevirtual

Execute the kernel on the passed window.

Warning
If is_parallelisable() returns false then the passed window must be equal to window()
Note
The window has to be a region within the window returned by the window() method
The width of the window has to be a multiple of num_elems_processed_per_iteration().
Parameters
[in]windowRegion on which to execute the kernel. (Must be a region of the window returned by window())
[in]infoInfo about executing thread and CPU.

Implements ICPPKernel.

Definition at line 829 of file NEGEMMLowpMatrixMultiplyKernel.cpp.

830 {
834 
835  // Check if the output tensor is a vector. If so,the kernel runs the vector-matrix multiplication path
836  if((_output->info()->dimension(1) == 1))
837  {
838  const auto width_matrix_a = static_cast<int>(_input0->info()->dimension(0));
839  const auto width_matrix_b = static_cast<int>(_input1->info()->dimension(0));
840  const auto in_b_stride = static_cast<int>(_input1->info()->strides_in_bytes()[1] / data_size_from_type(_input1->info()->data_type()));
841 
842  // The implementation computes 16 elements per iteration
843  const int window_start_x = 16 * info.thread_id;
844  const int window_step_x = 16 * info.num_threads;
845  // Make sure (window_end_x - window_start_x) is a multiple of window_step_x
846  const int window_end_x = ceil_to_multiple(width_matrix_b - window_start_x, window_step_x) + window_start_x;
847 
848  Window win_out(window);
849  win_out.set(Window::DimX, Window::Dimension(window_start_x, window_end_x, window_step_x));
850  win_out.set(Window::DimY, Window::Dimension(0, 1, 1));
851 
852  Window win_a(window);
853  win_a.set(Window::DimX, Window::Dimension(0, 0, 0));
854  win_a.set(Window::DimY, Window::Dimension(0, 0, 0));
855 
856  Window win_b;
857  // Don't slice matrix B along the z dimension if matrix B has just 2 dimensions and matrix A more than 2
858  // This scenario can happen when the the matrix multiplication is used to perform a convolution operation
859  if(_input1->info()->num_dimensions() >= 3)
860  {
861  win_b = window;
862  }
863  win_b.set(Window::DimX, Window::Dimension(window_start_x, window_end_x, window_step_x));
864  win_b.set(Window::DimY, Window::Dimension(0, 1, 1));
865 
866  Iterator ina(_input0, win_a);
867  Iterator inb(_input1, win_b);
868  Iterator out(_output, win_out);
869 
870  switch(_input0->info()->data_type())
871  {
872  case DataType::S8:
873  {
874  vector_matrix_multiply_s8(ina, inb, out, width_matrix_a, width_matrix_b, in_b_stride, window);
875  break;
876  }
877  case DataType::U8:
878  case DataType::QASYMM8:
879  {
880  vector_matrix_multiply_u8(ina, inb, out, width_matrix_a, width_matrix_b, in_b_stride, window);
881  break;
882  }
883  default:
884  {
885  ARM_COMPUTE_ERROR("Not supported");
886  break;
887  }
888  }
889  }
890  else
891  {
892  const size_t in_b_stride = _input1->info()->strides_in_bytes()[1];
893  const size_t out_stride = _output->info()->strides_in_bytes()[1] / _output->info()->element_size();
894 
895  // Set step_x and step_y for matrix A. Scale by a factor of 4 the Y range as the input interleaved matrix A has 4 times less the rows of the output matrix
896  Window win_a(window);
897  win_a.set(Window::DimX, Window::Dimension(0, 0, 0));
898  win_a.set(Window::DimY, Window::Dimension(window.y().start() / 4, window.y().end() / 4, 1));
899 
900  // Set step_x and step_y for matrix B. Scale by a factor of 16 the X range as the input transposed matrix A has 16 times less the columns of the output matrix
901  Window win_b;
902  // Don't slice matrix B along the z dimension if matrix B has just 2 dimensions and matrix A more than 2
903  // This scenario can happen when the the matrix multiplication is used to perform a convolution operation
904  if(_slide_matrix_b)
905  {
906  win_b = window;
907  }
908  win_b.set(Window::DimX, Window::Dimension(window.x().start() / 16, window.x().end() / 16, in_b_stride));
909  win_b.set(Window::DimY, Window::Dimension(0, 0, 0));
910 
911  // The step x and step y for the output matrix has been already set using in configure()
912  Iterator ina(_input0, win_a);
913  Iterator inb(_input1, win_b);
914  Iterator out(_output, window);
915 
916  const int width_b = _input1->info()->dimension(0);
917  switch(_input0->info()->data_type())
918  {
919  case DataType::S8:
920  {
921  matrix_multiply_s8(ina, inb, out, width_b, out_stride, window);
922  break;
923  }
924  case DataType::U8:
925  case DataType::QASYMM8:
926  {
927  matrix_multiply_u8(ina, inb, out, width_b, out_stride, window);
928  break;
929  }
930  default:
931  {
932  ARM_COMPUTE_ERROR("Not supported");
933  break;
934  }
935  }
936  }
937 }
#define ARM_COMPUTE_ERROR(...)
Print the given message then throw an std::runtime_error.
Definition: Error.h:261
virtual size_t num_dimensions() const =0
The number of dimensions of the tensor (rank)
const Window & window() const
The maximum window the kernel can be executed on.
Definition: IKernel.cpp:28
virtual size_t dimension(size_t index) const =0
Return the size of the requested dimension.
1 channel, 1 U8 per channel
virtual DataType data_type() const =0
Data type used for each element of the tensor.
Describe one of the image's dimensions with a start, end and step.
Definition: Window.h:75
static constexpr size_t DimX
Alias for dimension 0 also known as X dimension.
Definition: Window.h:43
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:160
auto ceil_to_multiple(S value, T divisor) -> decltype(((value+divisor - 1)/divisor) *divisor)
Computes the smallest number larger or equal to value that is a multiple of divisor.
Definition: Utils.h:66
quantized, asymmetric fixed-point 8-bit number
virtual ITensorInfo * info() const =0
Interface to be implemented by the child class to return the tensor's metadata.
size_t data_size_from_type(DataType data_type)
The size in bytes of the data type.
Definition: Utils.h:109
virtual size_t element_size() const =0
Element size in bytes calculated as data_size() * num_channels()
void set(size_t dimension, const Dimension &dim)
Set the values of a given dimension.
Definition: Window.inl:48
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
constexpr const Dimension & y() const
Alias to access the second dimension of the window.
Definition: Window.h:152
virtual const Strides & strides_in_bytes() const =0
The strides in bytes for accessing each dimension of the tensor.
constexpr int end() const
Return the end of the dimension.
Definition: Window.h:97
Iterator updated by execute_window_loop for each window element.
Definition: Helpers.h:318
#define ARM_COMPUTE_ERROR_ON_INVALID_SUBWINDOW(f, s)
Definition: Validate.h:205
constexpr int start() const
Return the start of the dimension.
Definition: Window.h:92
signed 8-bit number
Describe a multidimensional execution window.
Definition: Window.h:39
#define ARM_COMPUTE_ERROR_ON_UNCONFIGURED_KERNEL(k)
Definition: Validate.h:940
constexpr const Dimension & x() const
Alias to access the first dimension of the window.
Definition: Window.h:143

References ARM_COMPUTE_ERROR, ARM_COMPUTE_ERROR_ON_INVALID_SUBWINDOW, ARM_COMPUTE_ERROR_ON_UNCONFIGURED_KERNEL, ARM_COMPUTE_UNUSED, arm_compute::ceil_to_multiple(), arm_compute::data_size_from_type(), ITensorInfo::data_type(), ITensorInfo::dimension(), Window::DimX, Window::DimY, ITensorInfo::element_size(), Window::Dimension::end(), ITensor::info(), arm_compute::test::validation::info, ITensorInfo::num_dimensions(), arm_compute::QASYMM8, arm_compute::S8, Window::set(), Window::Dimension::start(), ITensorInfo::strides_in_bytes(), arm_compute::U8, IKernel::window(), Window::x(), and Window::y().

◆ validate()

Status validate ( const ITensorInfo input0,
const ITensorInfo input1,
const ITensorInfo output 
)
static

Static function to check if given info will lead to a valid configuration of NEGEMMLowpMatrixMultiplyKernel.

Parameters
[in]input0Input tensor info containing the interleaved Matrix A. Data type supported: QASYMM8
[in]input1Input tensor info containing the transposed Matrix B. Data type supported: same as input0
[in]outputOutput tensor info to store the result of matrix multiplication. Data type supported: S32
Returns
a status

Definition at line 821 of file NEGEMMLowpMatrixMultiplyKernel.cpp.

822 {
823  ARM_COMPUTE_RETURN_ON_ERROR(validate_arguments(input0, input1, output));
824  ARM_COMPUTE_RETURN_ON_ERROR(validate_and_configure_window(input0->clone().get(), input1->clone().get(), output->clone().get()).first);
825 
826  return Status{};
827 }
std::pair< Status, Window > validate_and_configure_window(ITensorInfo *input, ITensorInfo *weights, ITensorInfo *biases, ITensorInfo *output, const PadStrideInfo &conv_info, unsigned int depth_multiplier, const Size2D &dilation)
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:193
Status class.
Definition: Error.h:52
virtual std::unique_ptr< T > clone() const =0
Provide a clone of the current object of class T.

References ARM_COMPUTE_RETURN_ON_ERROR, ICloneable< T >::clone(), and arm_compute::validate_and_configure_window().

Referenced by NEGEMMLowpMatrixMultiplyCore::validate().


The documentation for this class was generated from the following files: