21.05
|
Functions | |
TfLiteDelegate * | tflite_plugin_create_delegate (char **options_keys, char **options_values, size_t num_options, void(*report_error)(const char *)) |
Create an ArmNN delegate plugin. More... | |
void | tflite_plugin_destroy_delegate (TfLiteDelegate *delegate) |
Destroy a given delegate plugin. More... | |
Variables | |
std::vector< std::string > | gpu_options |
This file defines two symbols that need to be exported to use the TFLite external delegate provider. More... | |
TfLiteDelegate* tflite::tflite_plugin_create_delegate | ( | char ** | options_keys, |
char ** | options_values, | ||
size_t | num_options, | ||
void(*)(const char *) | report_error | ||
) |
Create an ArmNN delegate plugin.
Available options:
Option key: "backends"
Possible values: ["EthosNPU"/"GpuAcc"/"CpuAcc"/"CpuRef"]
Descriptions: A comma separated list without whitespaces of backends which should be used for execution. Falls back to next backend in list if previous doesn't provide support for operation. e.g. "GpuAcc,CpuAcc"
Option key: "logging-severity"
Possible values: ["trace"/"debug"/"info"/"warning"/"error"/"fatal"]
Description: Sets the logging severity level for ArmNN. Logging is turned off if this option is not provided.
Option key: "gpu-tuning-level"
Possible values: ["0"/"1"/"2"/"3"]
Description: 0=UseOnly(default), 1=RapidTuning, 2=NormalTuning, 3=ExhaustiveTuning. Requires option gpu-tuning-file. 1,2 and 3 will create a tuning-file, 0 will apply the tunings from an existing file
Option key: "gpu-mlgo-tuning-file"
Possible values: [filenameString]
Description: File name for the MLGO tuning file
Option key: "gpu-tuning-file"
Possible values: [filenameString]
Description: File name for the tuning file.
Option key: "gpu-kernel-profiling-enabled"
Possible values: ["true"/"false"]
Description: Enables GPU kernel profiling
Option key: "save-cached-network"
Possible values: ["true"/"false"]
Description: Enables saving of the cached network to a file, specified with the cached-network-filepath option
Option key: "cached-network-filepath"
Possible values: [filenameString]
Description: If non-empty, the given file will be used to load/save the cached network. If save-cached-network is given then the cached network will be saved to the given file. To save the cached network a file must already exist. If save-cached-network is not given then the cached network will be loaded from the given file. This will remove initial compilation time of kernels and speed up the first execution.
Option key: "enable-fast-math"
Possible values: ["true"/"false"]
Description: Enables fast_math options in backends that support it
Option key: "number-of-threads"
Possible values: ["1"-"64"]
Description: Assign the number of threads used by the CpuAcc backend. Default is set to 0 (Backend will decide number of threads to use).
Option key: "reduce-fp32-to-fp16"
Possible values: ["true"/"false"]
Description: Reduce Fp32 data to Fp16 for faster processing
Option key: "reduce-fp32-to-bf16"
Possible values: ["true"/"false"]
Description: Reduce Fp32 data to Bf16 for faster processing
Option key: "debug-data"
Possible values: ["true"/"false"]
Description: Add debug data for easier troubleshooting
Option key: "memory-import"
Possible values: ["true"/"false"]
Description: Enable memory import
[in] | option_keys | Delegate option names |
[in] | options_values | Delegate option values |
[in] | num_options | Number of delegate options |
[in,out] | report_error | Error callback function |
Definition at line 116 of file armnn_external_delegate.cpp.
References DelegateOptions::AddBackendOption(), OptimizerOptions::m_Debug, OptimizerOptions::m_ImportEnabled, OptimizerOptions::m_ModelOptions, OptimizerOptions::m_ReduceFp32ToBf16, OptimizerOptions::m_ReduceFp32ToFp16, armnn::numeric_cast(), DelegateOptions::SetBackends(), DelegateOptions::SetLoggingSeverity(), DelegateOptions::SetOptimizerOptions(), armnnDelegate::TfLiteArmnnDelegateCreate(), and armnnDelegate::TfLiteArmnnDelegateOptionsDefault().
void tflite::tflite_plugin_destroy_delegate | ( | TfLiteDelegate * | delegate | ) |
Destroy a given delegate plugin.
[in] | delegate | Delegate to destruct |
Definition at line 238 of file armnn_external_delegate.cpp.
References armnnDelegate::TfLiteArmnnDelegateDelete().
std::vector<std::string> gpu_options |
This file defines two symbols that need to be exported to use the TFLite external delegate provider.
This is a plugin that can be used for fast integration of delegates into benchmark tests and other tools. It allows loading of a dynamic delegate library at runtime.
The external delegate also has Tensorflow Lite Python bindings. Therefore the dynamic external delegate can be directly used with Tensorflow Lite Python APIs.
See tensorflow/lite/delegates/external for details or visit the tensorflow guide here
Definition at line 29 of file armnn_external_delegate.cpp.