Group SDS_Stream_Interface

Modules > SDS_Stream_Interface

sds.h : Synchronous Data Stream Interface for record (write) and playback (read) of SDS data streams via a SDSIO interface.More...

Modules

Type	Name
module	Event Codes SDS Event Codes.
module	sdsFlags Bitmasks SDS Flag Bitmasks.
module	Function Return Codes SDS Function Return Codes.
module	sdsState Codes SDS State Codes.

Public Types

Type	Name
typedef void(*	sdsEvent_t Callback function for SDS stream events.
typedef void *	sdsId_t Handle to SDS stream.
enum	sdsMode_t SDS stream open mode.

Public Attributes

Type	Name
sdsError_t	sdsError Error information.
volatile uint32_t	sdsFlags Configuration options and control information.
volatile uint32_t	sdsIdleRate Idle rate information.
volatile uint32_t	sdsState State information.

Public Functions

Type	Name
int32_t	sdsClose (sdsId_t id) Close SDS stream.
int32_t	sdsExchange (void) Exchange information with the host.
void	sdsFlagsModify (uint32_t set_mask, uint32_t clear_mask) Modify sdsFlags control flags (atomic operation).
int32_t	sdsGetSize (sdsId_t id) Get data block size from an SDS stream opened in read mode.
int32_t	sdsInit (sdsEvent_t event_cb) Initialize SDS.
sdsId_t	sdsOpen (const char * name, sdsMode_t mode, void * buf, uint32_t buf_size) Open SDS stream.
int32_t	sdsRead (sdsId_t id, uint32_t * timeslot, void * buf, uint32_t buf_size) Read entire data block along with its timeslot information from the SDS stream opened in read mode.
int32_t	sdsUninit (void) Uninitialize SDS.
int32_t	sdsWrite (sdsId_t id, uint32_t timeslot, const void * buf, uint32_t buf_size) Write entire data block along with its timeslot information to the SDS stream opened in write mode.

Detailed Description

The SDS Interface manages recording (write) and playback (read) of SDS data streams through the SDSIO Interface. Data streams represent real-world I/O data for DSP and machine learning algorithms.

The SDS system uses a dedicated worker thread (sdsThread) to handle data stream I/O asynchronously. User-facing APIs interact only with internal circular buffers, allowing efficient, non-blocking data operations.

Each SDS stream is identified by a handle of type sdsId_t, which is returned by the sdsOpen function and is required for all subsequent operations on that stream.

Thread Safety

The SDS Interface functions are thread-safe during normal operation:

• A single thread may read from or write to a specific data stream at a time.
• Multiple data streams can be used concurrently by separate threads without conflict.

While operational calls are thread-safe, reuse of closed streams can lead to data corruption:

• When a stream is closed via sdsClose, its internal control block may be reallocated if another stream is opened.
• If a read or write operation is still pending on a handle after it has been closed, and a new stream is opened that causes control block reuse, the pending operation may unexpectedly complete on the newly opened stream.

To prevent these issues:

• Avoid opening a new stream immediately after closing another unless you can guarantee that all references and asynchronous operations related to the previous stream have completed or have been canceled.

Public Types Documentation

typedef sdsEvent_t

Callback function for SDS stream events.

typedef void(* sdsEvent_t) (sdsId_t id, uint32_t event);

Parameters:

• id sdsId_t handle of SDS stream
• event event code (see Event Codes)

This callback function is registered by passing a pointer to it as a parameter to the sdsInit function. It is triggered when an event such as insufficient data or space, or an error, occurs during the stream operation.

---

typedef sdsId_t

Handle to SDS stream.

typedef void* sdsId_t;

This pointer defines the handle to an SDS stream. It is used to identify a data stream across the different functions for the SDS system.

---

enum sdsMode_t

SDS stream open mode.

enum sdsMode_t {
    sdsModeRead = 0,
    sdsModeWrite = 1
};

This enum identifies the opening mode of an SDS stream: read or write. It is a parameter of the sdsOpen function.

---

Public Attributes Documentation

variable sdsError

Error information.

sdsError_t sdsError;

This global structure stores SDS diagnostic information, including status code, source file, line number, and an occurrence flag.

This information is relayed to the host in real time as it occurs (see sdsExchange).

---

variable sdsFlags

Configuration options and control information.

volatile uint32_t sdsFlags;

The sdsFlags variable stores configuration and diagnostic information in a single 32-bit word. This global variable can be directly read by the SDS application. To update the value from the SDS application use the function sdsFlagsModify.

The highest 8 bits are reserved for SDS control (see sdsFlags Bitmasks), and the lower 24 bits are available to the user. User bits can be used to configure algorithms in A/B tests (e.g., bypassing a filter) and to report status information to the SDSIO-Server.

The SDSIO-Server running on a host computer receives the sdsFlags value periodically (see sdsExchange). The SDSIO-Server can also update the sdsFlags variable, for example to start recording or configure algorithms.

---

variable sdsIdleRate

Idle rate information.

volatile uint32_t sdsIdleRate;

This global variable stores the current idle rate as a percentage. The value 0xFFFFFFFF indicates an unknown value when idle time measurement is not available.

This information is relayed to the host in real time periodically (see sdsExchange).

---

variable sdsState

State information.

volatile uint32_t sdsState;

This global variable contains current state code (see sdsState Codes) of the firmware application.

This information is not relayed to the host; it is available only in the firmware.

---

Public Functions Documentation

function sdsClose

Close SDS stream.

int32_t sdsClose (
    sdsId_t id
) 

Parameters:

• id sdsId_t handle to SDS stream

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Description:

Closes an SDS stream and releases any internal resources associated with the stream handle.

For closing a stream used in write mode:

• Prior to closing, any remaining data in the internal circular buffer is flushed to the SDS file. The function blocks until all data transfers are complete or a timeout occurs.

For closing a stream used in read mode:

• The function waits for all pending data transfers to complete or until a timeout occurs.

Upon successful closure, the stream handle becomes invalid.

---

function sdsExchange

Exchange information with the host.

int32_t sdsExchange (
    void
) 

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Description:

Exchanges SDS control information with the host. Updates sdsFlags if requested by the host, and sends the current sdsFlags value along with sdsIdleRate and optional error information sdsError to the host.

---

function sdsFlagsModify

Modify sdsFlags control flags (atomic operation).

void sdsFlagsModify (
    uint32_t set_mask,
    uint32_t clear_mask
) 

Parameters:

• set_mask bits to set in sdsFlags (see sdsFlags Bitmasks)
• clear_mask bits to clear in sdsFlags (see sdsFlags Bitmasks)

Description:

Atomically sets and clears bits in the global sdsFlags control flags. Bits present in set_mask are set and bits present in clear_mask are cleared in the update, applied in that order.

---

function sdsGetSize

Get data block size from an SDS stream opened in read mode.

int32_t sdsGetSize (
    sdsId_t id
) 

Parameters:

• id sdsId_t handle to stream

Returns:

number of bytes in next available data block, or a negative value on error or SDS_EOS (see Function Return Codes)

Description:

The function verifies that the entire header and the complete data block specified by the header are both present in the SDS circular buffer. It returns the size, in bytes, of the next available data block in the stream. If either the header is incomplete or the corresponding data block is not yet fully available, the function returns SDS_NO_DATA. If the end of the stream has been reached and no further data is available, the function returns SDS_EOS.

---

function sdsInit

Initialize SDS.

int32_t sdsInit (
    sdsEvent_t event_cb
) 

Parameters:

• event_cb pointer to sdsEvent_t callback function

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Description:

Initializes the SDS system. This function allocates resources, initializes underlying SDSIO interface, and creates the sdsThread worker thread. An optional callback function can be registered to receive notifications (e.g., I/O errors). This function must be called once before opening SDS streams.

---

function sdsOpen

Open SDS stream.

sdsId_t sdsOpen (
    const char * name,
    sdsMode_t mode,
    void * buf,
    uint32_t buf_size
) 

Parameters:

• name SDS stream name (pointer to NULL terminated string)
• mode SDS stream opening mode (see sdsMode_t)
• buf pointer to buffer for SDS stream
• buf_size buffer size in bytes

Returns:

sdsId_t handle to SDS stream, or NULL if operation failed

Description:

Opens an SDS stream for reading or writing timeslot information and data blocks to or from the SDS file. The buf parameter specifies a user-allocated memory region that serves as an internal circular buffer. The buffer must be large enough to hold at least the largest expected data block plus 8 bytes for header information.

To open a stream in write mode (for recording):

• The name parameter defines the base name for the SDS output file and is used to construct the full filename in the format name.index.sds. The index is an auto-incrementing value that ensures a unique filename is generated. If a file with the generated name already exists, the index is incremented until an unused name is found.

To open a stream in read mode (for playback):

• The name parameter specifies the base name of the SDS input file. When SDSIO uses a file system, the function attempts to locate and open the file in the format name.index.sds, where index is an auto-incrementing value. If no matching file is found, the function returns an error. When SDSIO uses an SDSIO-Client in conjunction with the SDSIO-Server, the filename is constructed by the SDSIO-Server according to the configuration specified in the sdsio.yml steering file.

For details, refer to Filenames section.

This function returns a handle that uniquely identifies the stream. The handle is used as a reference in subsequent function calls to perform operations on the stream.

---

function sdsRead

Read entire data block along with its timeslot information from the SDS stream opened in read mode.

int32_t sdsRead (
    sdsId_t id,
    uint32_t * timeslot,
    void * buf,
    uint32_t buf_size
) 

Parameters:

• id sdsId_t handle to SDS stream
• timeslot pointer to buffer for a timeslot value
• buf pointer to the data block buffer to be read
• buf_size size of the data block buffer in bytes

Returns:

number of bytes successfully read, or a negative value on error or SDS_EOS (see Function Return Codes)

Description:

Reads a data block along with its associated timeslot from the internal SDS circular buffer.

The sdsThread worker thread asynchronously reads the data from the SDS file using the underlying SDSIO interface. For details on how the specific SDS file is selected, refer to Filenames section. The retrieved data is then written to the internal SDS circular buffer. This asynchronous design enables efficient, non-blocking data handling and ensures optimal performance.

Before attempting to read, the function verifies that the entire header and the complete data block specified by the header are both present in the SDS circular buffer. If either the header is incomplete or the corresponding data block is not yet fully available, the function aborts and returns SDS_NO_DATA.

The function verifies that the user-provided buffer buf, with size buf_size, is large enough to accommodate the entire data block. If it is too small, the function aborts and returns SDS_ERROR_PARAMETER.

If the end of the stream has been reached and no further data is available, the function returns SDS_EOS.

On success, the function reads the data block from the circular buffer, stores it in the user-provided buffer, and returns the size of the data block in bytes. The associated timeslot is returned via the output parameter timeslot.

Thread safety is ensured by allowing only a single thread to read from a given stream at a time. However, multiple threads can concurrently read from different streams, enabling parallel operations across multiple streams.

---

function sdsUninit

Uninitialize SDS.

int32_t sdsUninit (
    void
) 

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Description:

De-initializes the SDS system. This function terminates the sdsThread worker thread, and releases the internal resources. All open SDS streams must be closed by the user before calling this function. After de-initialization, the system must be re-initialized before further use.

---

function sdsWrite

Write entire data block along with its timeslot information to the SDS stream opened in write mode.

int32_t sdsWrite (
    sdsId_t id,
    uint32_t timeslot,
    const void * buf,
    uint32_t buf_size
) 

Parameters:

• id sdsId_t handle to SDS stream
• timeslot timeslot
• buf pointer to the data block buffer to be written
• buf_size size of the data block buffer in bytes

Returns:

number of bytes successfully written or a negative value on error (see Function Return Codes)

Description:

Writes a data block, including a header containing the timeslot information and data block size, to the internal circular buffer.

The sdsThread worker thread asynchronously writes the data to the SDS file via the underlying SDSIO interface. For an explanation of how the SDS system selects and names the target SDS file, refer to Filenames section. This asynchronous design enables efficient, non-blocking data handling and optimized performance.

Before attempting to write, the function verifies that the entire header and the complete data block, provided via the buffer pointer buf and its size buf_size, can fit within the available space in the internal SDS circular buffer. If insufficient space is available, the operation is aborted and the function returns SDS_NO_SPACE.

On success, the function writes the header and data block to the SDS circular buffer and returns the number of data bytes written, excluding the header.

Thread safety is ensured by allowing only a single thread to write to a given stream at a time. However, multiple threads can concurrently write to different streams, enabling parallel operations across multiple streams.

---

---