The Micro speech program analyzes an audio input with a voice recognition model that can detect 2 keywords - yes and no. The recognized keywords are then printed into a serial interface. The voice recognition model is implemented using TensorFlow Lite for Microcontrollers.

GitHub repository github.com/arm-software/AVH-TFLmicrospeech contains an implementation supporting Arm Virtual Hardware (AVH) CPUs as well as real hardware boards.

Micro speech demonstrates how to use the processor and peripheral abstraction layers for simpler software portability across different targets. The example repository contains documentation with running instructions.

Getting Started with Micro speech on Arm Virtual Hardware

Table of contents

Prerequisites
Accessing and launching the AMI
Import and build example
- 3.1 Fork and clone example
- 3.2 Build example within the AMI
- 3.3 Run the example in place
- 3.4 Edit example
- 3.5 Submit changes back to GitHub
Automated CI/CD with GitHub Actions
- 4.1 Configure GitHub Actions
- 4.2 Setup runner on AMI
- 4.3 GitHub Actions workflow
- 4.4 Demonstration of a failed workflow
To go further

1. Prerequisites

a valid Github account
a SSH (VNC) client installed: PuTTY, MobaXterm
a valid AWS account

2. Accessing and launching the AMI

2.1. Find AMI on AWS Marketplace

Log into your AWS account and select AWS Marketplace Subscriptions service
Go to Discover products and search for Arm Virtual Hardware (with spaces)

AWS AMI subscription button

Click on Continue to subscribe to the AMI > Continue to configuration > Continue to launch
Choose c5.large instance type. This type has 2 powerful vCPU and 4 GB of RAM which is enough resources to run the AMI and simulation fast.

AWS instance size list

If you don't have a VPC (Virtual Private Cloud network) already, click on Create a VPC in EC2 * in the corresponding section. In the *Your VPCs interface, click on Actions > Create default VPC in the top-right corner.

AWS instance size list

If you don't have a Security Group already, click on Create new security group based on seller settings in the corresponding section. This will allow SSH connection to the instance. Add a name and description to save it.

AWS instance size list

If you don't have a Key Pair already, click on "Create a key pair in EC2" in the corresponding section. A SSH key pair correspond to a public key that will be copied on the server side (the running AVH AMI instance) and a private key needed by a client to connect to it (your local machine). In the Key pairs interface, click on Create a pair in the top-right corner and follow the default steps to download the private key that you will need to connect.

AWS key pair selection menu

Click on Launch. You will see a "Congratulations" message to inform you that the instance was successfully deployed. Click on the link provided "You can view this instance on EC2 Console" to check the instance status and retrieve its Public IPv4 address.

2.2. (Alternative) Launch the AMI from AWS EC2

Log into your AWS account and select Elastic Compute Cloud (EC2) service
Locate Images > AMIs in the sidebar
Search Public Images for ArmVirtualHardware (without spaces) and click on Launch.

AWS key pair selection menu

Follow the steps with the default options. If this has not been configured already, you will need to create a VPC, a new security group similar to what is described in section 2.4.
Click on Review and launch > Launch to select or create a key pair (see section 2.4). You will see a message to inform you that the instance was successfully launched. Click on the View instances link provided to check its status and retrieve its Public IPv4 address.

2.3. Enable AMI console

To connect to the running instance, you will need its Public IPv4 address. If you don't have it already, you can get from the EC2 > Instances.

Use SSH command on Linux, MacOS or Windows Powershell:

    ssh -i <key.pem> ubuntu@<AMI_IP_addr>

Or, if using MobaXterm on Windows:

Add new SSH session
Specify <AMI_IP_addr> as Remote host
Specify ubuntu as username
Enable Use private key and specify path to <key.pem>

MobaXterm SSH configuration

Or, if using PuTTY:

Use PuTTYgen to convert the .pem file to .ppk
Specify ubuntu@<AMI_IP_addr> in Session > Host name

PuTTY username and hostname/IP configuration

Specify path to key.pem in Session > Connection > SSH > Auth > Private key file for authentication

PuTTY private key configuration

2.4. [Optional] Enable <a href="https://github.com/cdr/code-server" >Code Server</a> (Visual Studio Code)

The Arm Virtual Hardware AMI comes with an IDE (Visual Studio Code) which can be accessed with a web browser. To access it, you will need to:

Start a SSH tunnel to the instance and forward port 8080. On Linux, MacOS or Windows Powershell:

ssh -i <key.pem> -N -L 8080:localhost:8080 ubuntu@<AMI_IP_addr>

The -N option holds the SSH tunnel connection and does not allow to execute other remote commands. This is useful for just forwarding ports.

Or, with MobaXterm on Windows:
- Add new SSH tunnel
- Specify 8080 in My computer > Forwarded port
- Specify <AMI_IP_addr> in SSH server
- Specify ubuntu in SSH login
- Specify 22 in SSH port
- Specify localhost in Remote server
- Specify 8080 in Remote port
- Save configuration. Click on the key icon to specify the path to <key.pem>
- Start the tunnel connection

MobaXterm SSH tunnel configuration

Launch a web browser on your local machine and open the following URL: http://localhost:8080

2.5. [Optional] Enable Virtual Network Computing (VNC)

VNC is a protocol to enable remote desktop. The instruction below will securely enable VNC through a SSH tunnel.

In the AMI terminal:

Enable VNC password (no need to enter a view-only password)

vncpasswd
Start the VNC server for the session

sudo systemctl start vncserver@1.service

To restart the VNC server after reboot

sudo systemctl enable vncserver@1.service

On your local machine:

Forward port 5901 on local machine. On Linux, MacOS or Windows Powershell:

ssh -I <key.pem> -N –L 5901:localhost:5901 ubuntu@<AMI_IP_addr>

Then, connect VNC client (e.g. Remmina, TigerVNC) to port 5901. You will be prompted for password.

Or, with MobaXterm on Windows:
- Add new VNC session
- Specify localhost as Remote hostname
- Specify 5901 as Port
- Select the Network settings tab configure the SSH gateway with:
  - <AMI_IP_addr> as Gateway host
  - ubuntu as Username
  - 22 as Port
  - Enable Use SSH key and specify path to <key.pem>

MobaXterm VNC configuration through SSH gateway

3. Import and build example

3.1. Fork and clone example

Open a web browser and enter the following URL: https://github.com/ARM-software/AVH-TFLmicrospeech
Log in to your github account and click on Fork (upper right)

In the AMI terminal:

 git config --global user.name YourGitHubName
 git config --global user.email Your.Email@domain.com
 git config --list
 git clone https://github.com/<YourGitHubName>/AVH-TFLmicrospeech

3.2. Build example within the AMI

In the AMI terminal:

Navigate to build folder

cd AVH-TFLmicrospeech/Platform_FVP_Corstone_SSE-300_Ethos-U55
Use cp_install utility (do once) to install the necessary CMSIS Packs dependencies

cp_install.sh packlist
Use (cbuild)[https://arm-software.github.io/CMSIS_5/Build/html/index.html] to build the software project (this should take about a minute)

cbuild.sh microspeech.Example.cprj

3.3. Run the example in place

In the AMI terminal:

Run script to load application to model and execute
```
 ./run_example.sh
```
This will run the application until it terminates (about a minute). You can terminate the simulation faster by specifying the number of cycles:
```
./run_example.sh --cyclelimit 100000000
```

Observe banner and output log

 Fast Models [11.16.14 (Sep 29 2021)]
 Copyright 2000-2021 ARM Limited.
 All Rights Reserved.

 telnetterminal0: Listening for serial connection on port 5000
 telnetterminal1: Listening for serial connection on port 5001
 telnetterminal2: Listening for serial connection on port 5002
 telnetterminal5: Listening for serial connection on port 5003

 Ethos-U rev afc78a99 --- Aug 31 2021 22:30:42
 (C) COPYRIGHT 2019-2021 Arm Limited
 ALL RIGHTS RESERVED

 Heard yes (146) @1000ms
 Heard no (145) @5600ms
 Heard yes (143) @9100ms
 Heard no (145) @13600ms
 Heard yes (143) @17100ms
 Heard no (145) @21600ms

 Info: Simulation is stopping. Reason: Cycle limit has been exceeded.

 Info: /OSCI/SystemC: Simulation stopped by user.
 [warning ][main@0][01 ns] Simulation stopped by user

 --- cpu_core statistics: ------------------------------------------------------
 Simulated time                          : 23.999999s
 User time                               : 25.804117s
 System time                             : 3.336213s
 Wall time                               : 29.132544s
 Performance index                       : 0.82
 cpu_core.cpu0                           :  26.36 MIPS (   768000000 Inst)
 -------------------------------------------------------------------------------

3.4. Edit example

In the AMI terminal:

Navigate to source folder

cd ../micro_speech/src/
Edit command_responder.cc using the nano text editor for example:

nano command_responder.cc

And change output (e.g. add your name as below)
```
TF_LITE_REPORT_ERROR(error_reporter, “YourName Heard %s (%d) @%dms", found_command, score, current_time);
```

Save (Ctrl+X with nano), rebuild (cbuild will just rebuild the files that have changed) and run to verify change

 cd ../../Platform_FVP_Corstone_SSE-300_Ethos-U55
 cbuild.sh microspeech.Example.cprj
 ./run_example.sh

3.5 Submit changes back to GitHub

In the AMI terminal

mark changed file(s) you wish to submit

 cd ../micro_speech/src/
 git add .

Commit changes, with arbitrary message

 git commit -m "Added my name to output message"

Verify the repository referenced is your forked copy
```
 git remote -v
```
Submit changes back to your repository
```
 git push
```
You will be asked your login and Personal Access Token (password) information.

To enable Personal Access Token in GitHub:
- go to Settings > Developer settings > Personal access tokens > Generate new token
- Enable repo to access the repository from the command-line
- Generate and copy the token to provide
In your own fork on Github, observe the change registered

https://github.com/<YourGitHubName>/AVH-TFLmicrospeech/blob/main/micro_speech/src/command_responder.cc

4. Automated CI/CD with GitHub Actions

4.1. Configure GitHub Actions

In your own fork on GitHub

Navigate to Settings > Actions > Runners
Add New self-hosted runner

GitHub New self-hosted runner button

Select Linux, x64 Runner image and copy the commands to set up

GitHub Self-hosted runner configuration instructions

4.2. Setup runner on AMI

In the AMI terminal

Go to base directory
```
 cd /home/ubuntu
```
Copy the commands from GitHub to configure with default options (no need for a runner group name)
Once configured, use run.sh to start the runner on the AMI
```
 ./run.sh
 Connected to GitHub
 yyyy-mm-dd hh:mm:ss: Listening for Jobs
```
In Github, go the Runner tab to see the runner listed and idle

GitHub runner list

4.3. GitHub Actions workflow

In Github, the file .github/workflows/virtual_hardware.yml defines the list of actions to perform (e.g. code checkout, build, run test script) on the runner and when (e.g. for every push to the repo)

In a new AMI terminal, revert your change in command_responder.cc and push to the repo

 cd AVH-TFLmicrospeech/micro_speech/src
 git add .
 git commit -m "Original message"
 git push

Back to the first AMI terminal where the runner has been started with ./run.sh, the runner reports the status

 <timestamp>: Listening for Jobs
 <timestamp>: Running job: ci_demonstration
 <timestamp>: Job ci_demonstration completed with result: Succeeded

In GitHub, locate the Actions sections and inspect the history of the workflow runs on the AMI instance

GitHub actions reports

4.4. Demonstration of a failed workflow

Change the code to be non-valid C e.g. edit command_responder.cc and remove the semicolon at the end of the line:
```
 TF_LITE_REPORT_ERROR(error_reporter, “YourName Heard %s (%d) @%dms", found_command, score, current_time)
```

Commit change to GitHub

 git add .
 git commit -m "Example of failure"
 git push

Observe in AMI that the runner reports failure

 <timestamp>: Running job: ci_demonstration
 <timestamp>: Job ci_demonstration completed with result: Failed

Same in GitHub's Actions tab

GitHub actions reports

5. To go further

The Machine Learning Group at Arm have developed other examples which can be run in with the Arm Virtual Hardware AMI.

The instructions in the quick start guide are easily reproducible in the AMI. You will need to enable VNC to visualize.

For more information on Arm's solution for IoT: https://www.arm.com/solutions/iot/total-solutions-iot

Have questions, comments, or suggestions? Visit the Arm Virtual Hardware support forum