GetStarted example demonstrates how to setup a Continuous Integration (CI) workflow for testing and debugging embedded applications using Arm Virtual Hardware (AVH). The project is maintained in the GitHub repository github.com/ARM-software/AVH-GetStarted that also contains detailed description of the example.

This chapter provides step-by-step guide through the CI workflow operation and its setup.

Overview

The AVH GetStarted example implements common steps in the CI workflow as shown on the figure below and explained in subsequent sections.

Continuous Integration (CI) workflow

Local development: at first, the software is developed locally using a common embedded toolchain such as Keil MDK and with Arm Fixed Virtual Platforms used for MCU simulation.
A GitHub repository is used as a source code management system for synchronization, storage and version control.
CI pipeline setup: a GitHub Action implements the CI pipeline that gets triggered on every code update in the target repository.
CI execution: automated program build and testing is performed in the cloud with Arm Virtual Hardware and results are reported back to the repository.
Failure analysis and local debug: developer can observe the CI test results in the GitHub Actions of the repository. In case of any failures they can be reproduced and debugged locally.

Prerequisites

Following is required to reproduce operation of the example project:

a GitHub account
an AWS account
a Windows PC with administrator rights

Other necessary software items are available free-of-charge and their installation is described in the related steps.

Develop tests

The AVH GetStarted embedded program implements a set of unit tests for validating operation of a simple function that is expected to return the sum value of two integer arguments. The example uses Unity Framework for test implementation and execution, however, the demonstrated concept is universal and can be applied to a different testing framework as well.

Create repository on GitHub

Initial project repository setup should follow a standard git process for either [creating a new repo] (https://docs.github.com/en/get-started/quickstart/create-a-repo) or forking an existing one. The AVH GetStarted repository is set up as a GitHub template repository so it is very easy to create own repository from it:

Open a web browser and go to the URL: https://github.com/ARM-software/AVH-GetStarted
Verify that you are logged in to your GitHub account.
Click on Use this template button. This opens the Create a new repository from AVH-GetStarted page.
In Repository name field provide the name for the new repository to be created under your account. For simplicity just reuse the original name AVH-GetStarted.
Select whether you want the new repository to be Public or Private. This does not impact example operation, but be aware of limited quotas for automation in private repositories depending on your GitHub account plan.
Check Include all branches. This is important for getting GitHub badges working correctly in your target repo.
Click on Create repository from template.
- The example repo gets copied into `https://github.com/<YourGitHubName>/AVH-GetStarted` repository, where <YourGitHubName> corresponds your GitHub user name.

Setup local project on your PC

Local repository setup

If the repository is present on GitHub, it can be easily copied onto local PC.

Make sure Git Bash is installed on the PC. For example [git for Windows] (https://gitforwindows.org/).
Open the Git Bash terminal in the target directory and execute clone command as:
```
 git clone https://github.com/<YourGitHubName>/AVH-GetStarted
```
This copies the content of the main branch to on your local drive. The badges branch is present in the GitHub repository but is not required for local use.

Project setup

Install Keil MDK and related tools as described in Tools installation.
In the local project repository double-click on the basic/basic.debug.cprj file to get the project imported into µVision IDE.
- Alternatively you can start µVision IDE and import the basic.debug.cprj project via the Project - Import menu.
Verify the project setup as explained in Project Configuration.

Implement tests

The main.c file in the example implements a set of unit tests validating the application function int my_sum(int a, int b). The implementation relies on the Unity Framework that is added to the example as a software component with the Unity software pack. In the example, the test_my_sum_fail demonstrates a test failure and section Analyze failures explains how to analyze CI output for debugging such failed tests.

The implementation of tests in the example can be considered as a template for adding more tests, covering other functions, or setting up unit testing in a custom project. Please refer to the documentation in [Unity GitHub] (https://github.com/ThrowTheSwitch/Unity) for further details and more complex examples.

Redirect stdout

By default Unity uses putchar for print out. Keil MDK does not support semihosting and hence standard output needs to be redirected to become visible during debug session. Redirect I/O component enables several mechanisms for that. In our example stdout output gets redirected to a UART interface.

Build and Run the example in Keil MDK

Build and execute the program in Keil MDK in the same way as any other project. Refer to Program Build and Debug for additional description.

The GetStarted example is configured to open a local Telnet console automatically when the debug session starts. By default the following output shall be observed there, indicating an intentional failure in test_my_sum_fail:

Telnet output with local execution

Export project to CPRJ format

The GetStarted example is also described in basic.debug.cprj file using universal .cprj format that gets used in command-line CI environments.

For correct workflow operation it is important to keep the MDK project uvprojx file and the cprj file synchronized. For that after saving modifications in the MDK project go to the µVision menu Project - Export and select Save project to CPRJ format.

Setup CI pipeline

As common for many projects, the CI pipeline for the AVH Get Started repository is triggered on every code change via push and pull requests. In our example this is explicitly limited to the main branch only.

The CI implementation in this example is implemented with GitHub Actions. AVH Client manages the connection between the GitHub repository and the Arm Virtual Hardware AMI instance in AWS, and configures the actions to be performed on the AMI, such as program build and execution.

Subsections below explain the setup for the AWS and GitHub Actions.

AWS setup

On the AWS side several items shall be setup to enable execution of example CI pipeline on Arm Virtual Hardware AMI.

Arm Virtual Hardware AMI subscription

Enable use of Arm Virtual Hardware AMI in your AWS account with the steps described in Subscribe Arm Virtual Hardware.
- Note the AMI ID value as it will be needed later for GitHub Actions setup.

AWS resources setup

Provision the AWS resources required for using Arm Virtual Hardware AMI.
- The simplest way is explained in Arm Virtual Hardware AWS-CloudFormation and relies on AWS CloudFormation service with the template file specifying default stack configuration.
  - Note the parameters available in the Output tab of the created stack. They will be needed later for GitHub Actions setup.
- Alternatively, for a more customized setup see GitHub-hosted Runners.
Ensure that a Key Pair is available for use with EC2.
By default the AVH GetStarted example expects a key pair with name common to create an EC2 instance (with line ssh_key_name: common in basic.yml file).
- In AWS Management Console type EC2 and go to the EC2 service.
- In the left menu find Network & Security section and click on Key Pairs.
- Verify that the same AWS region is selected as will be used with Arm Virtual Hardware AMI later.
- If no common key pair exists then click on Create key pair button.
- In the Create key pair dialog provide common as Name. Other settings can be kept at default values.
- Click on Create key pair.
- Save the file with the private key locally when corresponding file dialog opens.
- Observe the created Key Pair appears in the list.

GitHub Actions setup

Section Run AMI with GitHub Actions introduces the concept and explains it in details.

The GitHub Action in this example is implemented in the ./.github/workflows/basic.yml file using corresponding YAML syntax for GitHub workflows.

Add GitHub Secrets

Several parameters need to be configured in the repository as GitHub Secrets to enable communication with your AWS account. For that:

Go to the GitHub webpage of your repository.
Go to Settings tab and in the list on the left side select Secrets.
Use button New repository secret and add one by one following secrets:

Secret Name	Value for your AVH-GetStarted repository	Description
AWS_IAM_PROFILE	The value of AVHIAMProfile from the output of AWS resources setup.	The IAM Instance Profile to be used for AWS access.
AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY	The values of AVHAccessKeyId and AVHSecretAccessKey respectively from the output of AWS resources setup.	Access key pair for the AWS account (as IAM user) that shall be used by the CI workflow for AWS access.
AWS_S3_BUCKET_NAME	The value of AVHS3BucketName from the output of AWS resources setup.	The name of the S3 storage bucket to be used for data exchange between GitHub and Arm Virtual Hardware AMI.
AWS_SECURITY_GROUP_ID	The value of AVHEC2SecurityGroupId from the output of AWS resources setup.	The id of the VPC security group to add the EC2 instance to. Shall have format `sg-xxxxxxxx`.
AWS_DEFAULT_REGION	Use the same region as was used for AWS setup.	The data center region the Arm Virtual Hardware AMI will be run on. For example `eu-west-1`.
AWS_SUBNET_ID	Obtain a value as described in View your subnet.	The id of the VPC subnet to connect the EC2 instance to. Shall have format `subnet-xxxxxxxx`.

Execute CI

CI pipeline gets executed automatically on every code change in the main branch and execution results can be observed on the repository GitHub page.

Go to the GitHub webpage of the repository.
Go to Actions tab.
- In Workflows area you can see the list of GitHub workflows defined in the repository. In our example there is just one defined by *./.github/workflows/basic.yml* file in the repository.
Select the workflow and its runs will be displayed on the page. Note that right after copying the repository the list is empty because no workflows are executed yet.
- Use Run workflow to manually trigger execution of this workflow on the initial codebase.
- Wait until workflow execution is completed.
  In this example it shall have success status indicated with a green circle icon. This only means that the CI pipeline execution was successful, but the actual test results are not considered (where one unit test is intentionally failing).
Click on the title of the workflow run to analyze its execution as explained in next section.

Analyze failures

GitHub Documentation gives an overview about monitoring and troubleshooting options available for GitHub Actions.

Steps below guide through the analysis for AVH GetStarted example.

Observe the three CI jobs executed in the selected CI workflow as shown in the Jobs area on the left side.
- ci_test job executes the CI pipeline without doing any analysis of unit test results. It is considered as successful (green circle) when the CI pipeline was executed fully to the end.
- badge job generates GitHub badges for the README.md file to make the CI workflow status easily visible. The badge job is shown as successful when badge generation worked correctly.
- Test results job analyses actual results of the unit test execution in the CI. Green checkmark indicates that all executed tests pass.
By default the execution result of three CI jobs defined in this GetStarted example appears as follows:
In the Artifacts area click on results to download an archive file with additional details about the test run. It contains:
- basic.axf binary image that was built and tested in the CI run.
- .log file with program execution output.
- .xunit file with unit test results.
Click on ci_test or badge jobs to open corresponding run log that can be explored for execution details. For example for ci_test:
Click on Test results job that analyses the results of the executed tests. It can be seen that in default setup 1 out of 4 tests fails. The specific location (check on line 48 in main.c) and the reason (Expected 2 Was 0) for the failure are shown as well.

This allows to find the failure quickly. In our example it is a trivial one, introduced on purpose in the example code.

Return to the AVH GetStarted project in Keil MDK on your PC.
Open main.c file.
In line 48 replace the incorrect value 2 with the correct 0:
```
  TEST_ASSERT_EQUAL_INT(0, sum);
```
Rebuild the project.
Start debug session and observe in Telnet client that all tests pass:
Although this specific change only impacts main.c file, it is a good practice to also export the project to .cprj format to ensure it is synchronized with the MDK project.
Open the Git bash in your local repository and execute following commands that upload only the updated main.c file to your main repository:
```
  git commit basic/main.c -m "Fixed test_my_sum_fail"
  git push
```
Go to GitHub Actions page of your repository.
Observe that the CI workflow was automatically started with the code change. When it gets completed the Test results job shall indicate that there were no test failures.