Group SDS_Interface

Modules > SDS_Interface

sds.h : Synchronous Data Stream Interface for writing and reading SDS files via communication or file I/O interface.More...

Modules

Type	Name
module	Event Codes SDS Event Codes.
module	sdsFlags Bitmasks SDS Flag Bitmasks.
module	Function Return Codes SDS Function Return Codes.
module	sdsState Codes SDS State Codes.

Public Types

Type	Name
typedef void(*	sdsEvent_t Callback function for SDS stream events.
typedef void *	sdsId_t Handle to SDS stream.
enum	sdsMode_t SDS stream open mode.

Public Attributes

Type	Name
sdsError_t	sdsError Error information.
volatile uint32_t	sdsFlags Configuration options and control information.
volatile uint32_t	sdsIdleRate Idle rate information.
volatile uint32_t	sdsState State information.

Public Functions

Type	Name
int32_t	sdsClose (sdsId_t id) Close SDS stream.
int32_t	sdsExchange (void) Exchange information with the host.
void	sdsFlagsModify (uint32_t set_mask, uint32_t clear_mask) Modify sdsFlags control flags (atomic operation).
int32_t	sdsGetSize (sdsId_t id) Get data block size from an SDS stream opened in read mode.
int32_t	sdsInit (sdsEvent_t event_cb) Initialize SDS.
sdsId_t	sdsOpen (const char * name, sdsMode_t mode, void * buf, uint32_t buf_size) Open SDS stream.
int32_t	sdsRead (sdsId_t id, uint32_t * timeslot, void * buf, uint32_t buf_size) Read entire data block along with its timeslot information from the SDS stream opened in read mode.
int32_t	sdsUninit (void) Uninitialize SDS.
int32_t	sdsWrite (sdsId_t id, uint32_t timeslot, const void * buf, uint32_t buf_size) Write entire data block along with its timeslot information to the SDS stream opened in write mode.

Detailed Description

The SDS system manages writing to and reading from SDS files through communication or file I/O interfaces. It supports recording and playback of real-world data for applications such as machine learning and data analysis. For details on I/O interfaces, refer to chapter SDS IO Interface.

The system uses a dedicated worker thread (sdsThread) to handle file I/O asynchronously. User-facing APIs interact only with internal circular buffers, allowing efficient, non-blocking data operations.

Each SDS stream is identified by a handle of type sdsId_t, returned by sdsOpen function, and is required for all subsequent operations on that stream.

Thread Safety

The SDS Interface functions are thread-safe for regular operation:

• A single thread may read from or write to a specific stream at a time.
• Multiple streams can be used concurrently by separate threads without conflict.

While operational calls are thread-safe, improper reuse of closed streams can lead to data corruption:

• When a stream is closed via sdsClose, its internal control block may be reallocated if another stream is opened.
• If a read or write operation is still pending on a handle after it has been closed, and a new stream is opened that causes control block reuse, the pending operation may unexpectedly complete on the newly opened stream.

To prevent such issues:

• Avoid opening a new stream immediately after closing another unless you can guarantee that all references and asynchronous operations related to the previous stream have been fully completed or canceled.

Public Types Documentation

typedef sdsEvent_t

Callback function for SDS stream events.

typedef void(* sdsEvent_t) (sdsId_t id, uint32_t event);

Parameters:

• id sdsId_t handle of SDS stream
• event event code (see Event Codes)

This callback function is registered by passing a pointer to it as a parameter to the sdsInit function. It is triggered when an event such as insufficient data or space, or an error, occurs during the stream operation.

---

typedef sdsId_t

Handle to SDS stream.

typedef void* sdsId_t;

This pointer defines the handle to an SDS stream. It is used to identify a data stream across the different functions for the SDS system.

---

enum sdsMode_t

SDS stream open mode.

enum sdsMode_t {
    sdsModeRead = 0,
    sdsModeWrite = 1
};

This enum identifies the opening mode of an SDS stream: read or write. It is a parameter of the sdsOpen function.

---

Public Attributes Documentation

variable sdsError

Error information.

sdsError_t sdsError;

This global structure stores SDS diagnostic information, including status code, source file, line number, and an occurrence flag.

This information is relayed to the host in real time as it occurs (see sdsExchange).

---

variable sdsFlags

Configuration options and control information.

volatile uint32_t sdsFlags;

The sdsFlags are used to store configuration and diagnostic information in a single word. The sdsFlags value can be modified by the SDS application (using the function sdsFlagsModify) or by the SDSIO-Server.

The sdsFlags are a 32-bit global variable that contains a set of bits used for firmware control and feedback. The highest 8 bits are used by the SDS system (see sdsFlags Bitmasks) for control, and the lower 24 bits are available to the user. The user bits are useful for configuring algorithms in A/B tests (i.e. bypass a filter), but can be also used to communicate status information to the host.

The host that runs the SDSIO-Server can update the sdsFlags on demand. The sdsFlags value is also sent to the host periodically (see sdsExchange).

---

variable sdsIdleRate

Idle rate information.

volatile uint32_t sdsIdleRate;

This global variable stores current value of idle rate in percents, value 0xFFFFFFFF is reserved for unknown value in case that idle time measurement is not available.

This information is relayed to the host in real time periodically (see sdsExchange).

---

variable sdsState

State information.

volatile uint32_t sdsState;

This global variable contains current state code (see sdsState Codes) of the firmware application.

This information is not relayed to the host; it is available only in the firmware.

---

Public Functions Documentation

function sdsClose

Close SDS stream.

int32_t sdsClose (
    sdsId_t id
) 

Parameters:

• id sdsId_t handle to SDS stream

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Closes an SDS stream and releases any internal resources associated with the stream handle.

For closing a stream used in write mode:

• Prior to closing, any remaining data in the internal circular buffer is flushed to the SDS file. The function blocks until all data transfers are complete or a timeout occurs.

For closing a stream used in read mode:

• The function waits for all pending data transfers to complete or until a timeout occurs.

Upon successful closure, the stream handle becomes invalid.

---

function sdsExchange

Exchange information with the host.

int32_t sdsExchange (
    void
) 

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Exchanges SDS control information with the host. Update sdsFlags if requested by the host, and send current sdsFlags value along with sdsIdleRate and optional error information sdsError to the host.

---

function sdsFlagsModify

Modify sdsFlags control flags (atomic operation).

void sdsFlagsModify (
    uint32_t set_mask,
    uint32_t clear_mask
) 

Parameters:

• set_mask bits to set in sdsFlags (see sdsFlags Bitmasks)
• clear_mask bits to clear in sdsFlags (see sdsFlags Bitmasks)

Atomically sets and clears bits in the global sdsFlags control flags. Bits present in set_mask are set and bits present in clear_mask are cleared in the update, applied in that order.

---

function sdsGetSize

Get data block size from an SDS stream opened in read mode.

int32_t sdsGetSize (
    sdsId_t id
) 

Parameters:

• id sdsId_t handle to stream

Returns:

number of bytes in next available data block, or a negative value on error or SDS_EOS (see Function Return Codes)

Function verifies that the entire header and the complete data block specified by the header are both present in the SDS circular buffer. It returns the size, in bytes, of the next available data block in the stream. If either the header is incomplete or the corresponding data block is not yet fully available, the function returns SDS_NO_DATA. If the end of the stream has been reached and no further data is available, the function returns SDS_EOS.

---

function sdsInit

Initialize SDS.

int32_t sdsInit (
    sdsEvent_t event_cb
) 

Parameters:

• event_cb pointer to sdsEvent_t callback function

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

Initializes the SDS system. This function allocates resources, initializes underlying SDS I/O interface and creates the sdsThread worker thread. An optional callback function can be registered to receive notifications (e.g., I/O errors). This function must be called once before opening SDS streams.

---

function sdsOpen

Open SDS stream.

sdsId_t sdsOpen (
    const char * name,
    sdsMode_t mode,
    void * buf,
    uint32_t buf_size
) 

Parameters:

• name SDS stream name (pointer to NULL terminated string)
• mode SDS stream opening mode (see sdsMode_t)
• buf pointer to buffer for SDS stream
• buf_size buffer size in bytes

Returns:

sdsId_t handle to SDS stream, or NULL if operation failed

Opens an SDS stream for reading or writing timeslot information and data blocks to or from the SDS file. The buf parameter specifies a user-allocated memory region that serves as an internal circular buffer. The buffer must be large enough to hold at least the largest expected data block plus 8 bytes for header information.

For opening stream in write mode:

• The name parameter defines the base name for the SDS output file and is used to construct the full file name in the format name.index.sds. The index is an auto-incrementing value that ensures a unique file name is generated. If a file with the generated name already exists, the index is incremented until an unused name is found.

For opening stream in read mode:

• The name parameter specifies the base name of the SDS input file. When SDS I/O is a file system then the function attempts to locate and open the file in the format name.index.sds, where index is an auto-incrementing value. If no matching file is found, the function returns an error. When SDS I/O uses SDS I/O Client in conjunction with the SDS I/O Server, the file name is constructed by the SDS I/O Server according to configuration specified in the sdsio.yml steering file.

For details, refer to Filenames section.

This function returns a handle that uniquely identifies the stream. The handle is used as a reference in subsequent function calls to perform operations on the stream.

---

function sdsRead

Read entire data block along with its timeslot information from the SDS stream opened in read mode.

int32_t sdsRead (
    sdsId_t id,
    uint32_t * timeslot,
    void * buf,
    uint32_t buf_size
) 

Parameters:

• id sdsId_t handle to SDS stream
• timeslot pointer to buffer for a timeslot value
• buf pointer to the data block buffer to be read
• buf_size size of the data block buffer in bytes

Returns:

number of bytes successfully read, or a negative value on error or SDS_EOS (see Function Return Codes)

Reads a data block along with its associated timeslot from the internal SDS circular buffer.

The sdsThread worker thread asynchronously reads the data from the SDS file using the underlying SDS I/O interface. For details on how the specific SDS file is selected, refer to Filenames section. The retrieved data is then written to the internal SDS circular buffer. This asynchronous design enables efficient, non-blocking data handling and ensures optimal performance.

Before attempting to read, the function verifies that the entire header and the complete data block specified by the header are both present in the SDS circular buffer. If either the header is incomplete or the corresponding data block is not yet fully available, the function aborts and returns SDS_NO_DATA.

The function verifies that the user-provided buffer buf, with size buf_size, is large enough to accommodate the entire data block. If it is too small, the function aborts and returns SDS_ERROR_PARAMETER.

If the end of the stream has been reached and no further data is available, the function returns SDS_EOS.

On success, the function reads the data block from the circular buffer, stores it in the user-provided buffer, and returns the size of the data block in bytes. The associated timeslot is returned via the output parameter timeslot.

Thread safety is ensured by allowing only a single thread to read from a given stream at a time. However, multiple threads can concurrently read from different streams, enabling parallel operations across multiple streams.

---

function sdsUninit

Uninitialize SDS.

int32_t sdsUninit (
    void
) 

Returns:

SDS_OK on success or a negative value on error (see Function Return Codes)

De-initializes the SDS system. This function terminates the sdsThread worker thread, and releases the internal resources. All open SDS streams must be closed by the user before calling this function. After de-initialization, the system must be re-initialized before further use.

---

function sdsWrite

Write entire data block along with its timeslot information to the SDS stream opened in write mode.

int32_t sdsWrite (
    sdsId_t id,
    uint32_t timeslot,
    const void * buf,
    uint32_t buf_size
) 

Parameters:

• id sdsId_t handle to SDS stream
• timeslot timeslot
• buf pointer to the data block buffer to be written
• buf_size size of the data block buffer in bytes

Returns:

number of bytes successfully written or a negative value on error (see Function Return Codes)

Writes a data block, including a header containing the timeslot information and data block size, to the internal circular buffer.

The sdsThread worker thread asynchronously writes the data to the SDS file via the underlying SDS I/O interface. For an explanation of how the SDS system selects and names the target SDS file, refer to Filenames section. This asynchronous design enables efficient, non-blocking data handling and optimized performance.

Before attempting to write, function verifies that the entire header and the complete data block, provided via the buffer pointer buf and its size buf_size, can fit within the available space in the internal SDS circular buffer. If insufficient space is available, the operation is aborted and the function returns SDS_NO_SPACE.

On success, the function writes the header and data block to the SDS circular buffer and returns the number of data bytes written, excluding the header.

Thread safety is ensured by allowing only a single thread to write to a given stream at a time. However, multiple threads can concurrently write to different streams, enabling parallel operations across multiple streams.

---

---