Compute Library
 19.08
NESoftmaxLayer Class Reference

Basic function to compute a SoftmaxLayer. More...

#include <NESoftmaxLayer.h>

Collaboration diagram for NESoftmaxLayer:
[legend]

Public Member Functions

 NESoftmaxLayer (std::shared_ptr< IMemoryManager > memory_manager=nullptr)
 Constructor. More...
 
 NESoftmaxLayer (const NESoftmaxLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
 NESoftmaxLayer (NESoftmaxLayer &&)=default
 Default move constructor. More...
 
NESoftmaxLayeroperator= (const NESoftmaxLayer &)=delete
 Prevent instances of this class from being copied (As this class contains pointers) More...
 
NESoftmaxLayeroperator= (NESoftmaxLayer &&)=default
 Default move assignment operator. More...
 
void configure (ITensor *input, ITensor *output, float beta=1.0f, size_t axis=1)
 Set the input and output tensors. More...
 
void run () override
 Run the kernels contained in the function. More...
 
- Public Member Functions inherited from IFunction
virtual ~IFunction ()=default
 Destructor. More...
 
virtual void prepare ()
 Prepare the function for executing. More...
 

Static Public Member Functions

static Status validate (const ITensorInfo *input, const ITensorInfo *output, float beta=1.0f, size_t axis=1)
 Static function to check if given info will lead to a valid configuration of NESoftmaxLayer. More...
 

Detailed Description

Basic function to compute a SoftmaxLayer.

Softmax is calculated by :

\[ out = \frac{e^{x - max(x)}}{\sum{e^{x - max(x)}}} \]

This function runs the following kernels:

  1. NEFillBorderKernel
  2. NELogits1DMaxKernel
  3. NELogits1DSoftmaxKernel

Definition at line 49 of file NESoftmaxLayer.h.

Constructor & Destructor Documentation

◆ NESoftmaxLayer() [1/3]

NESoftmaxLayer ( std::shared_ptr< IMemoryManager memory_manager = nullptr)

Constructor.

Definition at line 36 of file NESoftmaxLayer.cpp.

37  : _memory_group(std::move(memory_manager)), _max_kernel(), _softmax_kernel(), _flat_or_reshape_kernel_ptr(nullptr), _fill_border_kernel(), _reshape_kernel(), _max(), _tmp(), _input_flattened(),
38  _output_flattened(), _needs_flattening(false)
39 {
40 }

◆ NESoftmaxLayer() [2/3]

NESoftmaxLayer ( const NESoftmaxLayer )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ NESoftmaxLayer() [3/3]

NESoftmaxLayer ( NESoftmaxLayer &&  )
default

Default move constructor.

Member Function Documentation

◆ configure()

void configure ( ITensor input,
ITensor output,
float  beta = 1.0f,
size_t  axis = 1 
)

Set the input and output tensors.

Parameters
[in,out]inputSource tensor. Data types supported: QASYMM8/F16/F32. If the width is not a multiple of the internal processing block size, NEFillBorderKernel replicates the last value of each row to the nearest multiple.
[out]outputDestination tensor. Data types supported: same as input.
[in]beta(Optional) A scaling factor for the exponent.
[in]axis(Optional) Reduction axis. Defaults to 1. Must be in range [1, input_num_dimensions). It has the purpose of squashing the first axis dimensions together. For instance, given a [4x4x4x4] image, when axis is 2, the Softmax reduction will be applied on each of the [4x4] planes of the input image.

Definition at line 71 of file NESoftmaxLayer.cpp.

72 {
73  // Perform validation step
74  ARM_COMPUTE_ERROR_ON_NULLPTR(input, output);
75  ARM_COMPUTE_ERROR_THROW_ON(NESoftmaxLayer::validate(input->info(), output->info(), beta, axis));
76 
77  // We don't need flattening only in the case the input is 2D and axis is 1
78  _needs_flattening = axis != 1;
79 
80  // If we are dealing with a 4D tensor, we will:
81  // - Flatten the input, so that we end up with a [width*height*depth] * batches 2D tensor
82  // - Execute all the pipeline (reduction + normalization) on the flattened tensor
83  // - Reshape the flattened output into the real output
84  if(_needs_flattening)
85  {
86  // Add to the memory manager _input_flattened
87  _memory_group.manage(&_input_flattened);
88 
89  // Configure _flatten_kernel and _input_flattened
90  configure_reshape_input_kernel(input, output, axis);
91  }
92 
93  // We want to deal with a 2D input. Either it is the flattened version of the original input (4D case)
94  // or it is the original input case (2D case)
95  ITensor *input_2D = (_needs_flattening ? &_input_flattened : input);
96 
97  // Create intermediate tensors shapes
98  const TensorInfo input_info = input_2D->info()->clone()->reset_padding().set_is_resizable(true);
99  DataType tmp_data_type = is_data_type_quantized_asymmetric(input_2D->info()->data_type()) ? DataType::F32 : input_2D->info()->data_type();
100  TensorInfo tensor_info_tmp(input_info.clone()->set_data_type(tmp_data_type));
101 
102  // Init intermediate tensors
103  TensorShape max_sum_shape = input_2D->info()->tensor_shape();
104  max_sum_shape.set(0, 1);
105  _max.allocator()->init(input_info.clone()->set_tensor_shape(max_sum_shape));
106  _tmp.allocator()->init(tensor_info_tmp);
107 
108  // Manage intermediate buffers
109  _memory_group.manage(&_max);
110  _memory_group.manage(&_tmp);
111 
112  // Configure Kernels
113  _max_kernel.configure(input_2D, &_max);
114  if(_needs_flattening)
115  {
116  // Add to the memory manager _output_flattened
117  _memory_group.manage(&_output_flattened);
118 
119  // The normalization kernel stores the result in a flat output tensor
120  _softmax_kernel.configure(input_2D, &_max, &_output_flattened, beta, &_tmp);
121  _input_flattened.allocator()->allocate();
122 
123  // Reshape the flat output into the requested (4D) output
124  _reshape_kernel.configure(&_output_flattened, output);
125 
126  // Allocate the intermediate flat tensors
127  _output_flattened.allocator()->allocate();
128  }
129  else
130  {
131  // Softmax 2D case
132  _fill_border_kernel.configure(input_2D, _max_kernel.border_size(), BorderMode::REPLICATE);
133  _softmax_kernel.configure(input_2D, &_max, output, beta, &_tmp);
134  }
135 
136  // Allocate intermediate buffers
137  _max.allocator()->allocate();
138  _tmp.allocator()->allocate();
139 }
void init(const TensorAllocator &allocator, const Coordinates &coords, TensorInfo &sub_info)
Shares the same backing memory with another tensor allocator, while the tensor info might be differen...
1 channel, 1 F32 per channel
#define ARM_COMPUTE_ERROR_THROW_ON(status)
Definition: Error.h:327
void configure(ITensor *tensor, BorderSize border_size, BorderMode border_mode, const PixelValue &constant_border_value=PixelValue())
Initialise the function.
TensorAllocator * allocator()
Return a pointer to the tensor's allocator.
Definition: Tensor.cpp:48
void configure(const ITensor *input, const ITensor *max, ITensor *output, const float beta, ITensor *tmp)
Set the input and output tensors.
void manage(TensorType *obj)
Sets a object to be managed by the given memory group.
void allocate() override
Allocate size specified by TensorInfo of CPU memory.
void configure(const ITensor *input, ITensor *output)
Set the input and output tensors.
void configure(const ITensor *input, ITensor *output)
Set the input and output of the kernel.
bool is_data_type_quantized_asymmetric(DataType dt)
Check if a given data type is of asymmetric quantized type.
Definition: Utils.h:1030
#define ARM_COMPUTE_ERROR_ON_NULLPTR(...)
Definition: Validate.h:161
BorderSize border_size() const override
The size of the border for that kernel.
Pixels outside the image are assumed to have the same value as the closest image pixel.
static Status validate(const ITensorInfo *input, const ITensorInfo *output, float beta=1.0f, size_t axis=1)
Static function to check if given info will lead to a valid configuration of NESoftmaxLayer.
DataType
Available data types.
Definition: Types.h:74

References TensorAllocator::allocate(), Tensor::allocator(), ARM_COMPUTE_ERROR_ON_NULLPTR, ARM_COMPUTE_ERROR_THROW_ON, arm_compute::test::validation::axis, NELogits1DMaxKernel::border_size(), ICloneable< T >::clone(), NEReshapeLayerKernel::configure(), NELogits1DMaxKernel::configure(), NEFillBorderKernel::configure(), NELogits1DSoftmaxKernel::configure(), ITensorInfo::data_type(), arm_compute::F32, ITensor::info(), TensorAllocator::init(), arm_compute::test::validation::input_info, arm_compute::is_data_type_quantized_asymmetric(), MemoryGroupBase< TensorType >::manage(), arm_compute::REPLICATE, TensorShape::set(), ITensorInfo::tensor_shape(), and NESoftmaxLayer::validate().

Referenced by arm_compute::test::validation::DATA_TEST_CASE().

◆ operator=() [1/2]

NESoftmaxLayer& operator= ( const NESoftmaxLayer )
delete

Prevent instances of this class from being copied (As this class contains pointers)

◆ operator=() [2/2]

NESoftmaxLayer& operator= ( NESoftmaxLayer &&  )
default

Default move assignment operator.

◆ run()

void run ( )
overridevirtual

Run the kernels contained in the function.

For NEON kernels:

  • Multi-threading is used for the kernels which are parallelisable.
  • By default std::thread::hardware_concurrency() threads are used.
Note
CPPScheduler::set_num_threads() can be used to manually set the number of threads

For OpenCL kernels:

  • All the kernels are enqueued on the queue associated with CLScheduler.
  • The queue is then flushed.
Note
The function will not block until the kernels are executed. It is the user's responsibility to wait.
Will call prepare() on first run if hasn't been done

Implements IFunction.

Definition at line 181 of file NESoftmaxLayer.cpp.

182 {
183  MemoryGroupResourceScope scope_mg(_memory_group);
184 
185  if(_needs_flattening)
186  {
187  NEScheduler::get().schedule(_flat_or_reshape_kernel_ptr.get(), Window::DimY);
188  }
189 
190  NEScheduler::get().schedule(&_fill_border_kernel, Window::DimY);
191  NEScheduler::get().schedule(&_max_kernel, Window::DimY);
192  NEScheduler::get().schedule(&_softmax_kernel, Window::DimY);
193 
194  if(_needs_flattening)
195  {
196  NEScheduler::get().schedule(&_reshape_kernel, Window::DimY);
197  }
198 }
static constexpr size_t DimY
Alias for dimension 1 also known as Y dimension.
Definition: Window.h:45
virtual void schedule(ICPPKernel *kernel, const Hints &hints)=0
Runs the kernel in the same thread as the caller synchronously.
static IScheduler & get()
Access the scheduler singleton.
Definition: Scheduler.cpp:96

References Window::DimY, Scheduler::get(), and IScheduler::schedule().

◆ validate()

Status validate ( const ITensorInfo input,
const ITensorInfo output,
float  beta = 1.0f,
size_t  axis = 1 
)
static

Static function to check if given info will lead to a valid configuration of NESoftmaxLayer.

Parameters
[in]inputSource tensor info. Data types supported: QASYMM8/F16/F32.
[in]outputDestination tensor info. Data types supported: same as input
[in]beta(Optional) A scaling factor for the exponent.
[in]axis(Optional) Reduction axis. Defaults to 1. Must be in range [1, input_num_dimensions). It has the purpose of squashing the first axis dimensions together. For instance, given a [4x4x4x4] image, when axis is 2, the Softmax reduction will be applied on each of the [4x4] planes of the input image.
Returns
a status

Definition at line 141 of file NESoftmaxLayer.cpp.

142 {
143  // Perform validation step
145  ARM_COMPUTE_RETURN_ERROR_ON_MSG(input->num_dimensions() > 4, "Only up to 4 dimensions are supported");
146  ARM_COMPUTE_UNUSED(beta);
147  ARM_COMPUTE_RETURN_ERROR_ON(axis < 1 || input->num_dimensions() < axis);
148 
149  // Create intermediate tensor info
150  DataType tmp_data_type = input->data_type();
151  const TensorInfo tensor_info_tmp(input->clone()->set_data_type(tmp_data_type).set_is_resizable(true));
152 
153  TensorShape max_sum_shape = input->tensor_shape();
154  max_sum_shape.set(0, 1);
155  const TensorInfo tensor_info_max_sum(input->clone()->set_tensor_shape(max_sum_shape).set_data_type(tmp_data_type).set_quantization_info(input->quantization_info()).set_is_resizable(true));
156  const TensorInfo dont_care;
157 
158  const bool needs_flattening = (axis != 1);
159 
160  if(needs_flattening)
161  {
162  const TensorShape shape_flatten = misc::shape_calculator::compute_softmax_shape(input, axis);
163  TensorInfo tensor_info_flat(input->clone()->set_tensor_shape(shape_flatten).set_is_resizable(true));
164 
165  if(axis != 3)
166  {
168  }
169  else
170  {
172  }
173  }
174 
175  ARM_COMPUTE_RETURN_ON_ERROR(NELogits1DMaxKernel::validate(input, &tensor_info_max_sum));
176  ARM_COMPUTE_RETURN_ON_ERROR(NELogits1DSoftmaxKernel::validate(&tensor_info_tmp, &tensor_info_max_sum, output, beta, &dont_care));
177 
178  return Status{};
179 }
#define ARM_COMPUTE_RETURN_ON_ERROR(status)
Checks if a status contains an error and returns it.
Definition: Error.h:193
TensorShape compute_softmax_shape(const ITensorInfo *input, size_t axis=1)
Calculate the softmax output shape of a tensor.
#define ARM_COMPUTE_RETURN_ERROR_ON(cond)
If the condition is true, an error is returned.
Definition: Error.h:244
static Status validate(const ITensorInfo *input, const ITensorInfo *max, const ITensorInfo *output, const float beta, const ITensorInfo *tmp)
Static function to check if given info will lead to a valid configuration of NELogits1DSoftmaxKernel.
#define ARM_COMPUTE_UNUSED(...)
To avoid unused variables warnings.
Definition: Error.h:160
#define ARM_COMPUTE_RETURN_ERROR_ON_MSG(cond,...)
If the condition is true, an error is returned.
Definition: Error.h:214
#define ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR(...)
Definition: Validate.h:163
static Status validate(const ITensorInfo *input, const ITensorInfo *output)
Static function to check if given info will lead to a valid configuration of NEReshapeLayerKernel.
static Status validate(const ITensorInfo *input, const ITensorInfo *output)
Static function to check if given info will lead to a valid configuration of NELogits1DMaxKernel.
static Status validate(const ITensorInfo *input, const ITensorInfo *output)
Static function to check if given info will lead to a valid configuration of NEFlattenLayerKernel.
DataType
Available data types.
Definition: Types.h:74

References ARM_COMPUTE_RETURN_ERROR_ON, ARM_COMPUTE_RETURN_ERROR_ON_MSG, ARM_COMPUTE_RETURN_ERROR_ON_NULLPTR, ARM_COMPUTE_RETURN_ON_ERROR, ARM_COMPUTE_UNUSED, arm_compute::test::validation::axis, ICloneable< T >::clone(), arm_compute::misc::shape_calculator::compute_softmax_shape(), ITensorInfo::data_type(), ITensorInfo::num_dimensions(), ITensorInfo::quantization_info(), TensorShape::set(), ITensorInfo::tensor_shape(), NEReshapeLayerKernel::validate(), NELogits1DMaxKernel::validate(), NEFlattenLayerKernel::validate(), and NELogits1DSoftmaxKernel::validate().

Referenced by NESoftmaxLayer::configure().


The documentation for this class was generated from the following files: