Theory of Operation

The SDS Framework enables recording and playback of one or more data streams for an application under development, as shown in the diagram below. With the SDSIO interface, the data streams are connected to SDS data files. The file storage can be either embedded within the system and accessed via a file system, or external (on a host computer) and accessed via a communication interface such as Ethernet or USB.

The DSP or ML algorithms that are tested operate on blocks and are executed periodically. This documentation uses these terms:

• Data Block: is a set of input or output data which is processed in one step by a DSP or ML compute node.
• Block size: is the number of bytes of a data block.
• Interval: is the periodic time interval at which the compute node executes.

Using the SDS Stream Interface functions (sds.c/h), the data stream under development may read and write as shown above. The Stream Interface functions store data streams in a circular buffer (sds_buffer.c/h). This circular buffer is the I/O queue for the SDSIO-Client (sdsio_x.c / sdsio.h).

Usage

The following diagram shows the usage of the SDS Stream Interface functions (executed in sdsThread). The sdsControlThread controls the overall execution. AlgorithmThread is the thread that executes Signal Conditioning (SC) and ML Model.

When AlgorithmThread starts, it first calls the InitInputData function, which initializes the input interfaces (for example, camera, microphone, accelerometer, etc.). It then calls InitAlgorithm, which is responsible for initializing the ML algorithm.

After initialization, AlgorithmThread enters a loop in which it repeatedly calls GetInputData. This function provides a block of input data used for a single inference. The inference itself is executed by the ExecuteAlgorithm function.

Two additional functions are also provided:

• DiscardInputData: used during playback to discard incoming live input data, since the actual input is read from the SDS file.
• ResetAlgorithm: called before each playback run to reset internal states, memory buffers, and results from previous inferences, ensuring a clean start for the new playback session.

SDS Data Files

Each data stream is stored in a separate SDS data file. In the diagram below SCinput.0.sds is the input to Signal Conditioning, SCoutput.0.sds is the output of Signal Conditioning, and MLoutput.0.sds is the output of the ML Model. Each execution of the algorithm is represented in a data block with a timeslot. The timeslot allows correlating blocks from different streams. In the example above, all blocks of one algorithm execution have the same timeslot value.

• Each call to the function sdsWrite writes one complete data block.
• Each call to the function sdsRead reads one complete data block.

Filenames

The sdsOpen function takes <name> for the stream and the opening mode as input parameters. Opening a stream in sdsModeRead mode is used for playback and opening stream in sdsModeWrite is used for recording.

The actual files used when opening a stream depend on the presence of the *.sdsio.yml control file and its play node.

Recording:

<label> is a sequential integer starting at 0. The value is incremented until no file exists with the corresponding name, at which point a new file is created.

Each subsequent recording session uses the next <label> value in the sequence. If a file with the selected name already exists, it is preserved by renaming it with a .bak extension before a new file is created with the original name.

Playback:

When a *.sdsio.yml control file is used and contains a play: node, the filename follows the pattern <stream-name>.<label>.sds, where <label> is specified in the corresponding step:.

When *.sdsio.yml control file is not used, the <label> is a sequential number starting at 0. If the corresponding file does not exist, the open operation fails. After playback session completes, the process repeats with the <label> incremented by one.

Timeslot

The timeslot is a 32-bit unsigned value and is used for:

• Alignment of different data streams that have the same timeslot value.
• Order of the SDS data files captured during execution.
• Combining multiple SDS file records with the same timeslot value.

The same timeslot connects different SDS file records. It is therefore useful to use the same timeslot for the recording of one iteration of a DSP or ML algorithm. In most cases the granularity of an RTOS tick (typically 1ms) is a good choice for a timeslot resolution.

File Format

The SDS Framework uses a binary data file format to store the individual data streams. It supports recording and playback of multiple data streams that may have jitter. Therefore, each stream contains timeslot information that allows correlating data streams, as required, for example, in sensor fusion applications.

The binary data format (stored in *.sds data files) has a record structure with a variable size. Each record has the following format:

1. timeslot: record timeslot in tick-frequency (32-bit unsigned integer, little endian)
2. data size: number of data bytes in the record (32-bit unsigned integer, little endian)
3. binary data: SDS stream (little endian, no padding) as described with the *.sds.yml file.

YAML Metadata Format

The content of each data stream may be described in a YAML metadata file that is created by the user. The following section defines the YAML format of this metadata file. The file schema/sds.schema.json is a schema description of the SDS Format Description.

For image, exactly one of stride_bytes or planes must be provided.

Example

This example defines a data stream with the name "sensorX" that contains the values of a gyroscope, temperature sensor, and additional raw data (that are not further described).

The binary data coming from these sensors are stored in data files with the following file format: <sensor-name>.<index>.sds. In this example, the filenames could be:

   sensorX.0.sds   # capture 0
   sensorX.1.sds   # capture 1

The following sensorX.sds.yml provides the format description of the SDS sensorX binary data files and may be used by data conversion utilities and data viewers.

sds:                   # describes a synchronous data stream
  name: sensorX        # user defined name
  description: Gyroscope stream with 1KHz, plus additional user data
  frequency: 1000
  content:
  - value: x           # Value name is 'x'
    type:  uint16_t    # stored using a 16-bit unsigned int
    scale: 0.2         # value is scaled by 0.2
    unit: dps          # base unit of the value
  - value: y
    type: uint16_t
    scale: 0.2
    unit: dps
  - value: z
    type: uint16_t
    unit: dps          # scale 1.0 is default
  - value: temp
    type: float
    unit: degree Celsius
  - value: raw
    type: uint16_t     # raw data, no scale or unit given
  - value: flag
    type: uint32_t:1   # a single bit stored in a 32-bit int

Image Metadata Format

The pixel_format values listed in sds.schema.json map to the following template files located in schema/image_format and use Linux V4L2 references.

Code Example

The following code snippets show the usage of the SDS for recording of sensor data. In this case an accelerometer data stream is recorded.

#include "sds.h"

// variable definitions
struct {                        // sensor data stream format
  uint16_t x;
  uint16_t y;
  uint16_t z;
} accelerometer [30];           // number of samples in one data stream record

sdsId_t *accel_id;              // data stream id
uint8_t  accel_buf[(sizeof(accelerometer)*2)+2048];     // data stream buffer for circular buffer handling
uint32_t timeslot;
int32_t  n;                     // number of bytes written to data stream
     :
// *** function calls ***
   sdsInit(NULL);               // init SDS
     :
   // open data stream for writing (recording)
   accel_id = sdsOpen("Accel", sdsModeWrite, accel_buf, sizeof(accel_buf));
     :
   // write data in accelerometer buffer with timeslot from RTOS kernel.
   timeslot = osKernelGetTickCount();
   n = sdsWrite(accel_id, timeslot, accelerometer, sizeof(accelerometer));
   if (n != sizeof(accelerometer)) {
     ... // unexpected size returned, error handling
   }
     :
  sdsClose(accel_id);           // close data stream

Buffer Size

The size of the data stream buffer depends on several factors such as:

• the communication interface technology that may impose specific buffer size requirements to maximize data transfer rates.
• the frequency of the algorithm execution. Fast execution speeds may require a larger buffer.

As a guideline, the buffer size should be at least (2 × block size) + 2 KB.
The minimum recommended buffer size is 4 KB.

SDSIO-Server Protocols

The SDSIO-Server communicates with the firmware using the SDSIO-Server Firmware Protocol over supported physical interfaces, including Ethernet, Wi-Fi, USB, debugger, and serial (USART) connections. It communicates with the monitor program through the Monitor Interface over a network connection, typically via the loopback (localhost) interface.

SDSIO-Server Firmware Protocol

The SDSIO-Server uses a simple protocol for data exchange between a host computer and the embedded target that integrates an SDSIO Interface. The protocol assumes that communication to the SDSIO-Server is ensured by the underlying transport technology (TCP/IP or USB); therefore, no additional checks are implemented.

The following conventions describe the command semantics used in the following documentation:

Commands:

Commands are sent from the embedded target to the host computer running the SDSIO-Server.

Each Command starts with a Header (4 Words = 16 bytes) followed by optional data of variable length. Depending on the Command, the SDSIO-Server replies with a Response that includes a Header with the same ID as the Command and may contain additional data.

SDSIO_CMD_OPEN

The Command with ID = 1 (SDSIO_CMD_OPEN) opens an SDS data file on the host computer. Mode defines read (value = 0) or write (value = 1) operation. Len of Stream Name is the size of the string in bytes.

SDS data filenames use the following file format: <name>.<label>.sds, where <name> is the stream name used as the base filename of the SDS data file and <label> is a user label or index depending on usage (for details see section Filenames).

| WORD |  WORD  | WORD | WORD **************|+++++++++++++|
>  1   |   0    | Mode | Len of Stream Name | Stream Name |
|******|********|******|********************|+++++++++++++|

The Response with ID = 1 (SDSIO_CMD_OPEN) provides a Handle that is used to identify the file in subsequent Commands.

| WORD |  WORD  | WORD | WORD *******|
<  1   | Handle | Mode | 0           |
|******|********|******|*************|

SDSIO_CMD_CLOSE

The Command with ID = 2 (SDSIO_CMD_CLOSE) closes an SDS data file on the host computer. The Handle is the identifier obtained with SDSIO_CMD_OPEN. There is no Response from the SDSIO-Server to this Command.

| WORD |  WORD  | WORD | WORD |
>  2   | Handle |  0   |  0   |
|******|********|******|******|

SDSIO_CMD_WRITE

The Command with ID = 3 (SDSIO_CMD_WRITE) writes data to an SDS data file on the host computer. The Handle is the identifier obtained with SDSIO_CMD_OPEN. Size specifies the size of Data in bytes. There is no Response from the SDSIO-Server to this Command.

| WORD |  WORD  | WORD | WORD |++++++|
>  3   | Handle |  0   | Size | Data |
|******|********|******|******|++++++|

SDSIO_CMD_READ

The Command with ID = 4 (SDSIO_CMD_READ) reads data from an SDS data file on the host computer. The Handle is the identifier obtained with SDSIO_CMD_OPEN. Size specifies the number of bytes to read.

| WORD |  WORD  | WORD | WORD |
>  4   | Handle | Size |  0   |
|******|********|******|******|

The Response with ID = 4 (SDSIO_CMD_READ) provides the data read from an SDS data file on the host computer. Size specifies the size of Data in bytes that was read and Status with nonzero = end of stream, else 0.

| WORD |  WORD  |  WORD  | WORD |++++++|
<  4   | Handle | Status | Size | Data |
|******|********|********|******|++++++|

SDSIO_CMD_PING

The Command with ID = 5 (SDSIO_CMD_PING) verifies if the SDSIO-Server is active and reachable on the host.

| WORD | WORD | WORD | WORD |
>  5   |  0   |  0   |  0   |
|******|******|******|******|

The Response with ID = 5 (SDSIO_CMD_PING) returns the Status with nonzero = server active, else 0.

| WORD | WORD |  WORD  | WORD |
<  5   |  0   | Status |  0   |
|******|******|********|******|

SDSIO_CMD_FLAGS

The asynchronous Response with ID = 6 (SDSIO_CMD_FLAGS) contains the SDS control flags update information from the host. This Response can arrive at any time when the host wants to update the SDS control flags. It can also precede a Response to any other command (e.g., SDSIO_CMD_OPEN or SDSIO_CMD_READ), but it cannot be sent by the host while a Response to another Command is in progress. The Set Mask specifies the bits to set in the sdsFlags and the Clear Mask specifies the bits to clear in the sdsFlags.

| WORD | WORD     | WORD       | WORD |
<  6   | Set Mask | Clear Mask |  0   |
|******|**********|************|******|

SDSIO_CMD_INFO

The Command with ID = 7 (SDSIO_CMD_INFO) sends control information (sdsFlags, sdsIdleRate and error information) to the host. There is no Response from the SDSIO-Server to this Command.

• sdsFlags is the current value of that global variable.
• sdsIdleRate is the current value of that global variable, value 0xFFFFFFFF indicates that idle rate information is not valid.
• Error Len specifies the size of the Error Data, value 0 indicates that no error occurred.

| WORD | WORD     | WORD        | WORD      |++++++++++++|
>  7   | sdsFlags | sdsIdleRate | Error Len | Error Data |
|******|**********|*************|***********|++++++++++++|

Error Data contains the information from sdsError global structure and has the following format.

| WORD   | WORD |+++++++++++++++++++|
| Status | Line | Filename (string) |
|********|******|+++++++++++++++++++|

SDSIO-Server Monitor Interface

The SDSIO-Server provides an additional TCP socket that may be used by a monitor program to observe SDS file activity and control sdsFlags in the firmware. The monitor interface is enabled with command line option --mon-port <port>.

The following conventions describe the command and message semantics used in the following documentation:

Commands and Messages:

Commands are sent to the SDSIO-Server which replies with a response (depending on command). SDSIO-Server can also send asynchronous messages at any time when not processing a command.

Each Command or Message starts with a Header (4 Words = 16 bytes) followed by optional data of variable length. Depending on the Command, the SDSIO-Server replies with a Response that includes a Header with the same ID as the Command and may contain additional data.

SDSIO_MON_OPEN

The Message with ID = 1 (SDSIO_MON_OPEN) is sent whenever an SDS data file is opened by the SDSIO-Server. Mode defines read (value = 0) or write (value = 1) operation. Filename Len is the size of the string in bytes and Filename is the name of the file (including path).

| WORD | WORD | WORD | WORD | WORD ********|++++++++++|
<  1   |  0   | Mode |  0   | Filename Len | Filename |
|******|******|******|******|**************|++++++++++|

SDSIO_MON_CLOSE

The Message with ID = 2 (SDSIO_MON_CLOSE) is sent whenever an SDS data file is closed by the SDSIO-Server. Filename Len is the size of the string in bytes and Filename is the name of the file (including path).

| WORD | WORD | WORD | WORD | WORD ********|++++++++++|
<  2   |  0   |  0   |  0   | Filename Len | Filename |
|******|******|******|******|**************|++++++++++|

SDSIO_MON_FLAGS

The Command with ID = 6 (SDSIO_MON_FLAGS) is used by the Monitor program to request an update of the SDS control flags in the firmware. The Set Mask specifies the bits to set in the sdsFlags and the Clear Mask specifies the bits to clear in the sdsFlags. This command does not generate a response from the SDSIO-Server.

| WORD | WORD     | WORD       | WORD |
>  6   | Set Mask | Clear Mask |  0   |
|******|**********|************|******|

SDSIO_MON_INFO

The Message with ID = 7 (SDSIO_MON_INFO) contains status information received by the SDSIO-Server from the firmware, including sdsFlags, sdsIdleRate, and error information. The message is sent to the Monitor program whenever the SDSIO-Server receives updated information from the firmware, as well as upon the initial connection to the Monitor program.

• sdsFlags is the current value of that global variable in the firmware.
• sdsIdleRate is the current value of that global variable in the firmware, value 0xFFFFFFFF indicates that idle rate information is not valid.
• Error Len specifies the size of the Error Data, value 0 indicates that no error occurred in the firmware.

| WORD | WORD     | WORD        | WORD      |++++++++++++|
<  7   | sdsFlags | sdsIdleRate | Error Len | Error Data |
|******|**********|*************|***********|++++++++++++|

Error Data contains the information from sdsError global structure and has the following format.

| WORD   | WORD |+++++++++++++++++++|
| Status | Line | Filename (string) |
|********|******|+++++++++++++++++++|

SDSIO_MON_SHUTDOWN

The Command with ID = 8 (SDSIO_MON_SHUTDOWN) is used by the Monitor program to request the SDSIO-Server to complete any ongoing operations, signal to the firmware that it will stop operating, and shut down gracefully.

| WORD | WORD | WORD | WORD |
>  8   |  0   |  0   |  0   |
|******|******|******|******|

SDSIO Message Sequence

This section describes the states and the message sequence of the SDS framework when using the SDSIO-Server. It contains the following threads that execute on the target.

• sdsControlThread: Control thread that organizes the overall execution.
• AlgorithmThread: Executes the algorithm under test.
• sdsThread: SDS worker thread.
• SDSIO-Server: SDSIO-Server running on the host computer.

The sdsControlThread handles a state machine with the following states:

The following flowcharts show the state transition in context with the messages that are exchanged with the SDSIO-Server.

Connect flowchart

Recording start flowchart

Recording stop flowchart

Playback flowchart