|
23.05
|
|
23.05
|
scons 2.3 or above is required to build the library. To see the build options available simply run scons -h
For Linux, the library was successfully built and tested using the following Linaro GCC toolchain:
To cross-compile the library in debug mode, with Arm® Neon™ only support, for Linux 32bit:
scons Werror=1 -j8 debug=1 neon=1 opencl=0 os=linux arch=armv7a
To cross-compile the library in asserts mode, with OpenCL only support, for Linux 64bit:
scons Werror=1 -j8 debug=0 asserts=1 neon=0 opencl=1 embed_kernels=1 os=linux arch=armv8a
You can also compile the library natively on an Arm device by using build=native:
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv8a build=native scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv7a build=native
For example on a 64bit Debian based system you would have to install g++-arm-linux-gnueabihf
apt-get install g++-arm-linux-gnueabihf
Then run
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv7a build=cross_compile
or simply remove the build parameter as build=cross_compile is the default value:
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=linux arch=armv7a
The examples get automatically built by scons as part of the build process of the library described above. This section just describes how you can build and link your own application against our library.
To cross compile a Arm® Neon™ example for Linux 32bit:
arm-linux-gnueabihf-g++ examples/neon_cnn.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -mfpu=neon -L. -larm_compute -larm_compute_core -o neon_cnn
To cross compile a Arm® Neon™ example for Linux 64bit:
aarch64-linux-gnu-g++ examples/neon_cnn.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -L. -larm_compute -larm_compute_core -o neon_cnn
(notice the only difference with the 32 bit command is that we don't need the -mfpu option and the compiler's name is different)
To cross compile an OpenCL example for Linux 32bit:
arm-linux-gnueabihf-g++ examples/cl_sgemm.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -mfpu=neon -L. -larm_compute -larm_compute_core -o cl_sgemm -DARM_COMPUTE_CL
To cross compile an OpenCL example for Linux 64bit:
aarch64-linux-gnu-g++ examples/cl_sgemm.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -L. -larm_compute -larm_compute_core -o cl_sgemm -DARM_COMPUTE_CL
(notice the only difference with the 32 bit command is that we don't need the -mfpu option and the compiler's name is different)
To cross compile the examples with the Graph API, such as graph_lenet.cpp, you need to link the examples against arm_compute_graph.so too.
i.e. to cross compile the "graph_lenet" example for Linux 32bit:
arm-linux-gnueabihf-g++ examples/graph_lenet.cpp utils/Utils.cpp utils/GraphUtils.cpp utils/CommonGraphOptions.cpp -I. -Iinclude -std=c++14 -mfpu=neon -L. -larm_compute_graph -larm_compute -larm_compute_core -Wl,--allow-shlib-undefined -o graph_lenet
i.e. to cross compile the "graph_lenet" example for Linux 64bit:
aarch64-linux-gnu-g++ examples/graph_lenet.cpp utils/Utils.cpp utils/GraphUtils.cpp utils/CommonGraphOptions.cpp -I. -Iinclude -std=c++14 -L. -larm_compute_graph -larm_compute -larm_compute_core -Wl,--allow-shlib-undefined -o graph_lenet
(notice the only difference with the 32 bit command is that we don't need the -mfpu option and the compiler's name is different)
To compile natively (i.e directly on an Arm device) for Arm® Neon™ for Linux 32bit:
g++ examples/neon_cnn.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -mfpu=neon -larm_compute -larm_compute_core -o neon_cnn
To compile natively (i.e directly on an Arm device) for Arm® Neon™ for Linux 64bit:
g++ examples/neon_cnn.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -larm_compute -larm_compute_core -o neon_cnn
(notice the only difference with the 32 bit command is that we don't need the -mfpu option)
To compile natively (i.e directly on an Arm device) for OpenCL for Linux 32bit or Linux 64bit:
g++ examples/cl_sgemm.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -larm_compute -larm_compute_core -o cl_sgemm -DARM_COMPUTE_CL
To compile natively the examples with the Graph API, such as graph_lenet.cpp, you need to link the examples against arm_compute_graph.so too.
i.e. to natively compile the "graph_lenet" example for Linux 32bit:
g++ examples/graph_lenet.cpp utils/Utils.cpp utils/GraphUtils.cpp utils/CommonGraphOptions.cpp -I. -Iinclude -std=c++14 -mfpu=neon -L. -larm_compute_graph -larm_compute -larm_compute_core -Wl,--allow-shlib-undefined -o graph_lenet
i.e. to natively compile the "graph_lenet" example for Linux 64bit:
g++ examples/graph_lenet.cpp utils/Utils.cpp utils/GraphUtils.cpp utils/CommonGraphOptions.cpp -I. -Iinclude -std=c++14 -L. -larm_compute_graph -larm_compute -larm_compute_core -Wl,--allow-shlib-undefined -o graph_lenet
(notice the only difference with the 32 bit command is that we don't need the -mfpu option)
To run the built executable simply run:
LD_LIBRARY_PATH=build ./neon_cnn
or
LD_LIBRARY_PATH=build ./cl_sgemm
For example:
LD_LIBRARY_PATH=. ./graph_lenet --help
Below is a list of the common parameters among the graph examples :
In order to build for SVE or SVE2 you need a compiler that supports them. You can find more information in the following these links:
An example build command with SVE is:
scons arch=armv8.2-a-sve os=linux build_dir=arm64 -j55 standalone=0 opencl=0 openmp=0 validation_tests=1 neon=1 cppthreads=1 toolchain_prefix=aarch64-none-linux-gnu-
In order to build for SME2 you need to use a compiler that supports SVE2 and enable SVE2 in the build as well.
An example build command with SME2 is:
scons arch=armv8.6-a-sve2-sme2 os=linux build_dir=arm64 -j55 standalone=0 opencl=0 openmp=0 validation_tests=1 neon=1 cppthreads=1 toolchain_prefix=aarch64-none-linux-gnu-
For Android, the library was successfully built and tested using Google's standalone toolchains:
(From 23.02, NDK >= r20b is highly recommended) For NDK r18 or older, here is a guide to create your Android standalone toolchains from the NDK:
Generate the 32 and/or 64 toolchains by running the following commands to your toolchain directory $MY_TOOLCHAINS:
$NDK/build/tools/make_standalone_toolchain.py –arch arm64 –install-dir $MY_TOOLCHAINS/aarch64-linux-android-ndk-r18b –stl libc++ –api 21
$NDK/build/tools/make_standalone_toolchain.py –arch arm –install-dir $MY_TOOLCHAINS/arm-linux-android-ndk-r18b –stl libc++ –api 21
For NDK r19 or newer, you can directly Download the NDK package for your development platform, without the need to launch the make_standalone_toolchain.py script. You can find all the prebuilt binaries inside $NDK/toolchains/llvm/prebuilt/$OS_ARCH/bin/.
CC=clang CXX=clang++ scons toolchain_prefix=aarch64-linux-android21-
CC=aarch64-linux-android21-clang CXX=aarch64-linux-android21-clang++ scons toolchain_prefix=""
export PATH=$PATH:$MY_TOOLCHAINS/aarch64-linux-android-ndk-r18b/bin:$MY_TOOLCHAINS/arm-linux-android-ndk-r18b/bin
To cross-compile the library in debug mode, with Arm® Neon™ only support, for Android 32bit:
CXX=clang++ CC=clang scons Werror=1 -j8 debug=1 neon=1 opencl=0 os=android arch=armv7a
To cross-compile the library in asserts mode, with OpenCL only support, for Android 64bit:
CXX=clang++ CC=clang scons Werror=1 -j8 debug=0 asserts=1 neon=0 opencl=1 embed_kernels=1 os=android arch=armv8a
The examples get automatically built by scons as part of the build process of the library described above. This section just describes how you can build and link your own application against our library.
Once you've got your Android standalone toolchain built and added to your path you can do the following:
To cross compile a Arm® Neon™ example:
#32 bit: arm-linux-androideabi-clang++ examples/neon_cnn.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -larm_compute-static -larm_compute_core-static -L. -o neon_cnn_arm -static-libstdc++ -pie #64 bit: aarch64-linux-android-clang++ examples/neon_cnn.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -larm_compute-static -larm_compute_core-static -L. -o neon_cnn_aarch64 -static-libstdc++ -pie
To cross compile an OpenCL example:
#32 bit: arm-linux-androideabi-clang++ examples/cl_sgemm.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -larm_compute-static -larm_compute_core-static -L. -o cl_sgemm_arm -static-libstdc++ -pie -DARM_COMPUTE_CL #64 bit: aarch64-linux-android-clang++ examples/cl_sgemm.cpp utils/Utils.cpp -I. -Iinclude -std=c++14 -larm_compute-static -larm_compute_core-static -L. -o cl_sgemm_aarch64 -static-libstdc++ -pie -DARM_COMPUTE_CL
To cross compile the examples with the Graph API, such as graph_lenet.cpp, you need to link the library arm_compute_graph also.
#32 bit: arm-linux-androideabi-clang++ examples/graph_lenet.cpp utils/Utils.cpp utils/GraphUtils.cpp utils/CommonGraphOptions.cpp -I. -Iinclude -std=c++14 -Wl,--whole-archive -larm_compute_graph-static -Wl,--no-whole-archive -larm_compute-static -larm_compute_core-static -L. -o graph_lenet_arm -static-libstdc++ -pie -DARM_COMPUTE_CL #64 bit: aarch64-linux-android-clang++ examples/graph_lenet.cpp utils/Utils.cpp utils/GraphUtils.cpp utils/CommonGraphOptions.cpp -I. -Iinclude -std=c++14 -Wl,--whole-archive -larm_compute_graph-static -Wl,--no-whole-archive -larm_compute-static -larm_compute_core-static -L. -o graph_lenet_aarch64 -static-libstdc++ -pie -DARM_COMPUTE_CL
Then you need to do is upload the executable and the shared library to the device using ADB:
adb push neon_cnn_arm /data/local/tmp/ adb push cl_sgemm_arm /data/local/tmp/ adb push gc_absdiff_arm /data/local/tmp/ adb shell chmod 777 -R /data/local/tmp/
And finally to run the example:
adb shell /data/local/tmp/neon_cnn_arm adb shell /data/local/tmp/cl_sgemm_arm adb shell /data/local/tmp/gc_absdiff_arm
For 64bit:
adb push neon_cnn_aarch64 /data/local/tmp/ adb push cl_sgemm_aarch64 /data/local/tmp/ adb push gc_absdiff_aarch64 /data/local/tmp/ adb shell chmod 777 -R /data/local/tmp/
And finally to run the example:
adb shell /data/local/tmp/neon_cnn_aarch64 adb shell /data/local/tmp/cl_sgemm_aarch64 adb shell /data/local/tmp/gc_absdiff_aarch64
For example: adb shell /data/local/tmp/graph_lenet –help
In this case the first argument of LeNet (like all the graph examples) is the target (i.e 0 to run on Neon™, 1 to run on OpenCL if available, 2 to run on OpenCL using the CLTuner), the second argument is the path to the folder containing the npy files for the weights and finally the third argument is the number of batches to run.
The library was successfully natively built for Apple Silicon under macOS 11.1 using clang v12.0.0.
To natively compile the library with accelerated CPU support:
scons Werror=1 -j8 neon=1 opencl=0 os=macos arch=armv8a build=native
For bare metal, the library was successfully built using linaro's latest (gcc-linaro-6.3.1-2017.05) bare metal toolchains:
Download linaro for armv7a and armv8a.
To cross-compile the library with Arm® Neon™ support for baremetal armv8a:
scons Werror=1 -j8 debug=0 neon=1 opencl=0 os=bare_metal arch=armv8a build=cross_compile cppthreads=0 openmp=0 standalone=1
Examples are disabled when building for bare metal. If you want to build the examples you need to provide a custom bootcode depending on the target architecture and link against the compute library. More information about bare metal bootcode can be found here.
Using scons directly from the Windows® command line is known to cause problems. The reason seems to be that if scons is setup for cross-compilation it gets confused about Windows® style paths (using backslashes). Thus it is recommended to follow one of the options outlined below.
The best and easiest option is to use Ubuntu on Windows®. This feature is still marked as beta and thus might not be available. However, if it is building the library is as simple as opening a Bash on Ubuntu on Windows® shell and following the general guidelines given above.
If the Windows® subsystem for Linux is not available Cygwin can be used to install and run scons, the minimum Cygwin version must be 3.0.7 or later. In addition to the default packages installed by Cygwin scons has to be selected in the installer. (git might also be useful but is not strictly required if you already have got the source code of the library.) Linaro provides pre-built versions of GCC cross-compilers that can be used from the Cygwin terminal. When building for Android the compiler is included in the Android standalone toolchain. After everything has been set up in the Cygwin terminal the general guide on building the library can be followed.
Native builds on Windows® are experimental and some features from the library interacting with the OS are missing.
It's possible to build Compute Library natively on a Windows® system running on Arm™.
Windows® on Arm™ (WoA) systems provide compatibility emulating x86 binaries on aarch64. Unfortunately Visual Studio 2022 does not work on aarch64 systems because it's an x86_64bit application and these binaries cannot be exectuted on WoA yet.
Because we cannot use Visual Studio to build Compute Library we have to set up a native standalone toolchain to compile C++ code for arm64 on Windows®.
Native arm64 toolchain installation for WoA:
There are some additional tools we need to install to build Compute Library:
In order to use clang to build Windows® binaries natively we have to initialize the environment variables from VS22 correctly so that the compiler could find the arm64 C++ libraries. This can be done by pressing the key windows + r and running the command:
cmd /k "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvarsx86_arm64.bat"
To build Compute Library type:
scons opencl=0 neon=1 os=windows examples=0 validation_tests=1 benchmark_examples=0 build=native arch=armv8a Werror=0 exceptions=1 standalone=1
Compute Library requires OpenCL 1.1 and above with support of non uniform workgroup sizes, which is officially supported in the Arm® Mali™ OpenCL DDK r8p0 and above as an extension (respective extension flag is -cl-arm-non-uniform-work-group-size).
Enabling 16-bit floating point calculations require cl_khr_fp16 extension to be supported. All Arm® Mali™ GPUs with compute capabilities have native support for half precision floating points.
Integer dot product built-in function extensions (and therefore optimized kernels) are available with Arm® Mali™ OpenCL DDK r22p0 and above for the following GPUs : G71, G76. The relevant extensions are cl_arm_integer_dot_product_int8, cl_arm_integer_dot_product_accumulate_int8 and cl_arm_integer_dot_product_accumulate_int16.
OpenCL kernel level debugging can be simplified with the use of printf, this requires the cl_arm_printf extension to be supported.
SVM allocations are supported for all the underlying allocations in Compute Library. To enable this OpenCL 2.0 and above is a requirement.
In addition to the scons build the repository includes experimental Bazel and CMake builds. These builds currently support a limited range of options. Both are similar to the scons multi_isa build. It compiles all libraries with Neon (TM) support, as well as SVE and SVE2 libraries. The build is CPU only, not including OpenCL support. Only Linux environment is targeted for now. Both were successfully built with gcc / g++ version 10.2.
File structure for all files included in the Bazel build:
.
├── .bazelrc
├── BUILD
├── WORKSPACE
├── arm_compute
│ └── BUILD
├── examples
│ └── BUILD
├── include
│ └── BUILD
├── scripts
│ ├── print_version_file.py
│ └── BUILD
├── src
│ └── BUILD
├── support
│ └── BUILD
├── tests
│ ├── BUILD
│ └── framework
│ └── BUILD
└── utils
└── BUILD
Available build options:
- debug: Enable ['-O0','-g','-gdwarf-2'] compilation flags - Werror: Enable -Werror compilation flag - logging: Enable logging - cppthreads: Enable C++11 threads backend - openmp: Enable OpenMP backend
Build everything (libraries, examples, tests):
bazel build //...
Build libraries:
bazel build //:all
Build arm_compute only:
bazel build //:arm_compute
Build examples:
bazel build //examples:all
Build resnet50 example:
bazel build //examples:graph_resnet50
Build validation and benchmarking:
bazel build //tests:all
File structure for all files included in the CMake build:
.
├── CMakeLists.txt
├── cmake
│ ├── Options.cmake
│ ├── Version.cmake
│ └── toolchains
│ └── aarch64_linux_toolchain.cmake
├── examples
│ └── CMakeLists.txt
├── src
│ └── CMakeLists.txt
└── tests
├── CMakeLists.txt
├── benchmark
│ └── CMakeLists.txt
└── validation
└── CMakeLists.txt
Available build options:
- CMAKE_BUILD_TYPE: "Release" (default) enables ['-O3', '-DNDEBUG'] compilation flags, "Debug" enables ['-O0','-g','-gdwarf-2', '-DARM_COMPUTE_ASSERTS_ENABLED'] - ARM_COMPUTE_WERROR: Enable -Werror compilation flag - ARM_COMPUTE_EXCEPTIONS: If disabled ARM_COMPUTE_EXCEPTIONS_DISABLED is enabled - ARM_COMPUTE_LOGGING: Enable logging - ARM_COMPUTE_BUILD_EXAMPLES: Build examples - ARM_COMPUTE_BUILD_TESTING: Build tests - ARM_COMPUTE_CPPTHREADS: Enable C++11 threads backend - ARM_COMPUTE_OPENMP: Enable OpenMP backend
To build libraries, examples and tests:
mkdir build cd build cmake .. -DCMAKE_BUILD_TYPE=Release -DARM_COMPUTE_OPENMP=1 -DARM_COMPUTE_WERROR=0 -DARM_COMPUTE_BUILD_EXAMPLES=1 -DARM_COMPUTE_BUILD_TESTING=1 -DCMAKE_INSTALL_LIBDIR=. cmake --build . -j32
The GEMM kernels used for convolutions and fully-connected layers in Compute Library employ memory layouts optimized for each kernel implementation. This then requires the supplied weights to be re-ordered into a buffer ready for consumption by the GEMM kernel. Where Compute Library is being called from a framework or library which implements operator caching, the re-ordering of the inputted weights into an intermediate buffer may no longer be desirable. When using a cached operator, the caller may wish to re-write the weights tensor, and re-run the operator using the updated weights. With the default GEMM kernels in Compute Library, the GEMM will be executed with the old weights, leading to incorrect results.
To address this, Compute Library provides a set of GEMM kernels which use a common blocked memory format. These kernels consume the input weights directly from the weights buffer and do not execute an intermediate pre-transpose step. With this approach, it is the responsibility of the user (in this case the calling framework) to ensure that the weights are re-ordered into the required memory format. NEGEMM::has_opt_impl is a static function that queries whether there exists fixed-format kernel, and if so will return in the expected weights format. The supported weight formats are enumerated in arm_compute::WeightFormat.
Fixed format kernels are only available for the CPU backend. To build Compute Library with fixed format kernels set fixed_format_kernels=1:
scons Werror=1 debug=0 neon=1 opencl=0 embed_kernels=0 os=linux multi_isa=1 build=native cppthreads=1 openmp=0 fixed_format_kernels=1
1.8.13