 |
File System Component
MDK Middleware for Devices with Flash File System
|
|
|
File System Component
MDK Middleware for Devices with Flash File System
|
|
The File System Component provides functions to create, save, read, and modify files. The Retarget Interface of the C/C++ compiler run-time library allows the usage of ANSI C standard file I/O functions. The File System Component is designed for Cortex-M processor-based devices and requires an CMSIS-RTOS compliant real-time operating system for thread-safe operation.
The File System Component consists of two different file systems, placed on top of different layers, depending on the media type:
The File System Component integrates with the Standard C/C++ Library and requires a CMSIS-RTOS2 compliant RTOS. It cannot be used with the Standard C/C++ Library that does not provide the (retarget) hooks for I/O file handling.
The following picture shows the File System's structure from a developer's perspective.
The configuration steps and files that are needed to create applications for the different devices are explained here:
The File System Component supports various memory and storage devices that are assigned to a drive.
All available drives are defined with the RTE Component Selection. A drive is passed as string to System Routines and may be part of a filename. The system allows you to configure a current drive that is used when no drive is specified as part of the filename.
The file system type is automatically chosen depending on the drive type (which correlates to a certain memory device). Some drives may use different interfaces that will be covered by appropriate CMSIS-Drivers. The following table shows the relationship between drive - memory device - and driver/interface:
| Drive | File System | Drive Type | CMSIS-Driver/Interface |
|---|---|---|---|
| "F:" or "F0:" | EFS | NOR Flash drive 0 | (Memory Bus or ) |
| "F1:" | EFS | NOR Flash drive 1 | (Memory Bus or ) |
| "R:" or "R0:" | FAT | RAM drive | N/A |
| "M:" or "M0:" | FAT | Memory Card drive 0 | or |
| "M1:" | FAT | Memory Card drive 1 | or |
| "U:" or "U0:" | FAT | USB Memory Stick drive 0 | |
| "U1:" | FAT | USB Memory Stick drive 1 | |
| "N:" or "N0:" | FAT | NAND Flash drive 0 | (Memory Bus) |
| "N1:" | FAT | NAND Flash drive 1 | (Memory Bus) |
Note
- NOR Flash is typically connected over Memory Bus or SPI. For using a NOR Flash drive, a Flash driver is required which encapsulates either the Memory Bus interface or SPI (in which case an SPI driver required in addition).
- The Middleware Component contains example drivers for Flash devices that can be used to implement other device drivers if required. The examples are: AM29x800BB/M29W640FB (Flash with 16-bit memory bus interface), S29GL064Nx2 (Flash with 32-bit memory bus interface), and AT45DB642D (Flash with SPI interface using an SPI driver).
- Other drives (memory card, USB memory device, NAND Flash) require appropriate CMSIS-Driver for the microcontroller device. If no suitable CMSIS-Driver is available for your selected device, you may create the required Flash interface driver. Refer to the for details.
- Currently, using NAND Flash devices with SPI interface (Serial NAND) is not supported.
The Current Drive defines the drive used when the filename itself does not specify the drive or when functions that expect a drive are called with an empty ("") string.
If there is only one drive enabled in the system then it automatically becomes the "Current Drive". Alternatively, when there are multiple drives enabled, the one with the drive letter that is first in alphabetical order is set as the "Current Drive". The Current Drive can be changed during run-time using the function fchdrive.
Note
- Define
FS_INITIAL_CDRIVEcan be used to define initial current drive at build time.- Set
FS_INITIAL_CDRIVEto one of the following values: 0(F0), 1(F1), 2(M0), 3(M1), 4(N0), 5(N1), 6(R0), 9(R1), 7(U0), 8(U1)
The FAT File System was initially used on PC operating systems such as MS-DOS and early versions of Microsoft Windows. Still, it is widely used on USB memory sticks or memory cards for file storage. It is simple, robust and offers good performance especially in embedded applications.
FAT is an acronym for File Allocation Table. This table provides the index of the files in the system and is statically allocated at the time of formatting the drive. It contains an entry for each cluster (a data storage area). The FAT's entries contain either the number of the next cluster in the file, or a marker indicating EOF (end of file), unused disk space, or other areas of the drive that are specially reserved. The drive's root directory contains the number of the first cluster of each file in the directory; the operating system traverses the FAT table, looking up the cluster number of each successive part of the file, until the end of the file is reached. Likewise, sub-directories are implemented as special files containing the directory entries of their respective files.
Since the first implementation, the maximum number of clusters of a drive has increased dramatically, and so the number of bits used to identify each cluster has grown. The versions of the FAT format are named after the number of table element bits: FAT12, FAT16, and FAT32. Each of these variants is is supported by the File System Component.
The actual FAT file system is made up of four different sections:
| Section | Description |
|---|---|
| Boot sector | Contains machine startup code |
| File Allocation Table | Contains the data region map; multiple entries might exist for redundancy purposes |
| Root Directory | Stores information about the files and directories located in the root directory (FAT12/FAT16 only) |
| File Data Region | Actual stored file data |
Volume label for FAT12, FAT16 or FAT32 volume must be maximum 11 characters long and cannot contain the * ? / \ | , ; : + = < > [ ] " characters.
Path name format used to specify either a drive, directory or file:
Depending on the usage case, each specifier can be optional:
The table below explains how to specify either absolute or relative path.
| Path name | Type | Description |
|---|---|---|
| \ | Absolute | Root directory |
| \ .. | Absolute | Root directory |
| . | Relative | Current directory |
| .. | Relative | Parent directory |
| M:\file.txt | Absolute | A file in root directory of the drive M0 |
| M:file.txt | Relative | A file in current directory of the drive M0 |
| \file.txt | Absolute | A file in the root directory of the current drive |
| file.txt | Relative | A file in the current directory of the current drive |
| folder1\ .. | Relative | Current directory |
File System Component is provided with the long (LFN) and short (SFN or 8.3) filename support. Since there is no compulsory algorithm for creating the 8.3 name from an LFN, File System Component uses convention described below.
The File System Component fully supports time stamping for files and directories:
To be able to use the time information provided by an on-chip RTC, the user application needs to implement the function fs_get_time(). The fsTime structure must be filled with the data returned from the RTC driver. If the fs_get_time() function is not implemented in the user application (which is the default), predefined time and date stamps are used.
The File System Component supports a proprietary journaling mechanism for FAT which makes file I/O functions robust against system crashes or power failures. When FAT Journaling is enabled, the file allocation table and directory entries are protected.
FAT Journaling keeps track of the changes in the file system using a journal before committing them to the main file system. In the event of a system failure, file system does not become corrupted. Only content of files that are opened in writing mode may be lost. Other files are not affected.
Journaling can be enabled individually for each drive that is using the FAT File System. For information on how to enable Journaling refer to:
There requirements for creating a journal are:
If journaling is enabled, mounting a drive triggers the File System to search for the journal.
When journal is found the journaling system is initialized and the journal content is checked for any incomplete data transactions that may be the result of a system crash. When incomplete data transactions are found the file system restores the drive to the last known valid state.
When no journal is found, the journal is automatically created. The required space (32kB) for journal is reserved by marking the adequate amount of clusters in the last 1% of the drive partition. This reduces the reported amount of free space on the drive accordingly.
No special drive formatting is required. When a drive is formatted on a different computer (running Window or Linux) free space is typical available at the end of a partition. Creation of a journal may fail on removable media when above requirements are not meet. This may happen if the drive is fragmented or there is not enough space. In this case the drive should be defragmented or some files should be deleted.
When using the fformat function of the File System Component, the journal is automatically created on drives with enabled option "Use FAT Journal".
When File System Component experiences a power loss or crashes before a file opened for writing is successfully closed the content of that file is lost. This results in lost data clusters which can be restored using a file system repair utility tool such as chkdsk on Windows or fsck on Linux.
By design, FAT carries a few limitations:
The actual FAT implementation of the File System Component has the following limitations:
The Memory Card (MC) Control Layer is used when connecting to external memory cards or eMMC devices. It supports data transfer via MCI or SPI, but the data rates are much better using MCI.
In general, the following devices are supported by the File System Component:
| Memory Card | Specification |
|---|---|
| Standard Capacity SD Memory Card (SDSC) | Up to and including 2 GB |
| High Capacity SD Memory Card (SDHC) | More than 2GB and up to and including 32GB |
| Extended Capacity SD Memory Card (SDXC) | More than 32GB and up to and including 2TB |
| MultiMediaCard (MMC) | Up to specification version 4.2 |
| Embedded MultiMediaCard (eMMC) | From specification 4.3 |
MC Control Layer uses when memory device is connected via MCI. Depending on the , File System Component can take advantage of certain features. The following table briefly describes how enabled capabilities affect the functionality of the MC Control Layer:
| Capability | Functionality |
|---|---|
| cd_state | Enables device presence detection by pooling the Card Detect (CD) switch state |
| cd_event | Enables device presence detection using MCI driver events |
| wp_state | Enables write protection detection by pooling Write Protect (WP) switch state |
| vdd | Power consumption can be controlled by turning VDD (VCC) power supply on or off |
| vccq | Power consumption can be controlled by turning VCCQ power supply on or off |
| data_width_4 | Data transfer rate can be increased by using 4-bit data bus |
| data_width_8 | Data transfer rate can be increased by using 8-bit data bus |
| high_speed | Data transfer rate can be increased by using MCI bus frequencies up to 52MHz |
| rst_n | eMMC device can be reset from software using RST_n pin |
With appropriately designed hardware and support from the corresponding MCI driver, MC Control Layer can control memory device power supply voltage:
Depending on the memory device type, appropriate data bus width will be selected:
The MC Control Layer is able to automatically switch to high speed mode for SD cards supporting SD specification version 1.10 or higher (3.3V signaling, bus clock frequency up to 50MHz) and MMC/eMMC devices supporting MMCA specification version 4.2 or higher (3.3V signaling, bus clock frequency up to 52MHz). In order to do so, the high speed mode needs to be supported by the corresponding .
The password protection feature enables the host to lock a card while providing a password, which later will be used for unlocking the card. The password is stored to device internal non-volatile registers and can consist of up to 16 bytes, where each byte can be arbitrary 8-bit value.
Password can be managed by using the function fs_ioc_device_ctrl, with control code fsDevCtrlCodeLockUnlock and structure fsLockUnlock as control code argument. When executing management operation, members of the fsLockUnlock structure must be set as follows:
Setting the password:
.password.length.flags to FS_DEVICE_SET_PASSWORDWhen operation executes successfully, new password is stored into device registers.
Reset the password:
.password.length.flags to FS_DEVICE_CLR_PASSWORDWhen operation executes successfully, password is cleared from device registers.
Lock the card:
.password.length.flags to FS_DEVICE_LOCK_UNLOCKWhen operation executes successfully, data access protection is activated. All read/write accesses are rejected.
Note It is possible to set password and lock the card in a single step:
- define set password and lock the card operation by setting member
.flagsto (FS_DEVICE_SET_PASSWORD | FS_DEVICE_LOCK_UNLOCK)
Unlock the card:
.password.length.flags to 0 (zero)When operation executes successfully, data access protection is deactivated. All read/write accesses are allowed.
Forcing erase:
.password and .length are ignored.flags to FS_DEVICE_ERASEWhen operation executes successfully, all the card data content is erased along with the current password.
Note
- SD and MMCA specification also allows to replace the password in a single step. This is not supported by the FileSystem library, since it is known that various devices have issues with the interpretation of this operation. Therefore, when changing the password, old one must be first cleared and then new password can be set.
- Is is also recommended to unlock the device before resetting the password.
To successfully mount locked memory card and gain read/write data access, password management information must be provided from the user application:
User application should only implement callback function fs_mc_lock_unlock when support for memory card password management is required. This function will be called only when mounting a memory card with active password protection.
Many industrial grade eMMC and SD memory devices support health status monitoring system known as S.M.A.R.T. (Self-Monitoring, Analysis and Reporting Technology). Health Status monitor data is used to anticipate and detect imminent memory failures.
eMMC and SD memory devices use common command (CMD56) to access vendor specific information about memory health status. In order to retrieve relevant information, function fs_ioc_device_ctrl can be used together with control code fsDevCtrlCodeHealthStatus and structure fsHealthStatus as control code argument.
Health status access command behavior is vendor specific but in general the following rules apply:
When function fs_ioc_device_ctrl is called, general command (CMD56) is executed and variable fsHealthStatus::arg is used as command argument. The transfered data gets copied into or from the buffer specified by fsHealthStatus::buf.
File System Component tends to reduce overall power consumption as much as possible, therefore the MC Control Layer will automatically:
eMMC devices (MMCA specification version 4.3 and higher) support CMD5 (Sleep state). To switch between Sleep state and Standby state, use the function fs_ioc_device_ctrl. While in the Sleep state, VCC power supply may be switched off to further minimize the power consumption.
SD memory cards (up to SD specification version 4.10) and MMC devices (up to MMCA specification version 4.2) also support Serial Peripheral Interface (SPI) bus mode as alternative way of communication. When SPI mode is used, MC Control Layer will use to use the underlying hardware.
The advantage of using the memory card in SPI mode is in simplified hardware requirements, but at the cost of performance loss and limited command set. Since SPI consists of only SCLK, MOSI, MISO and nSS communication lines, File System Component provides following callback routines for interfacing Card Detect (CD) and Write Protect (WP) switches:
Callback functions fs_mc_read_cd and fs_mc_read_wp are optional. If they are not implemented by user application, File System Component will assume that memory device in SPI mode is always present and without write protection.
Since software controlled Slave Select line is not always available an alternative way is provided in such case:
User application should only implement callback function fs_mc_spi_control_ss when SPI driver does not provide software controlled Slave Select line.
Reliable communication at high frequencies is a common problem when using SPI bus, since maximum bus frequency can be limited by PCB design. To overcome this problem, FileSystem component implements simple bus frequency adaptation algorithm in order to determine the maximum bus frequency at which it is still possible to communicate with the memory card.
FileSystem component will always attempt to set the highest bus frequency supported by the memory card (usually 25MHz) but if after that the memory card is not accessible anymore I will lower the bus frequency. The adaptation is performed only during memory card initialization procedure.
Features
NAND Flash Overview
NAND Flash memory is used in USB drives, secure digital (SD) cards, memory sticks, CompactFlash cards and multimedia cards (MMC). It is a non-volatile memory with 60 percent smaller memory cells comparing to NOR memory cells thus offers high data density at a lower cost. It is organized into a series of blocks, which are the smallest erasable entities in a NAND Flash device. The smallest entity that can be programmed is a byte. Due to a limited number of times NAND Flash blocks can be reliably programmed and erased (nominally, each NAND block will survive 100000 program/erase cycles) it is critical to implement a NAND Flash Translation Layer (NFTL) to maximize the lifespan of the device. This layer comprises of wear leveling algorithms, bad block management and error control algorithms and thus maximizes the lifespan of a device by redirecting new writes to different physical blocks. It only applies to NAND Flash devices directly attached to the microcontroller.
NAND Flash Architecture
A NAND Flash array is organized in blocks with multiple pages per block. Each page consist of user data area and spare data area. The spare data area is physically the same as user data area and is typically used for error correction information (ECC), wear leveling and other software system functions. The user data area is available to the user for data storage. The size of the user data area depends on the page size and is typically of one or multiple sector size.
Due to its architecture, data can be written to an address on a NAND flash memory only a finite number of times. NAND flash memory wears out if data is written too often to the same address, so wear leveling is used to help prolong the life of the NAND flash device. It ensures that data erasures and writes are distributed evenly across the NAND Flash storage medium, so that NAND memory blocks don't fail prematurely due to a high number of erase cycles.
Dynamic wear leveling addresses the issue of repeated writes to the same logical blocks by redirecting new writes to different physical blocks. Dynamic wear leveling works on data blocks that are being written to dynamically.
Static wear leveling addresses the issue of static data that is kept unchanged for a long time in content and location. Some static data can stay in the same data block until the NAND flash memory is broken. To evenly distribute write and erase cycles across NAND memory blocks, an algorithm is implemented to move the static data from one location to the other in the Flash.
The File System Component implements wear leveling for all kinds of NAND Flash devices (SLC, MLC, TLC).
It is normal for a NAND flash memory to contain bad or invalid blocks of memory. Invalid blocks are blocks that contain one or more invalid bits so that the reliability can no longer be guaranteed. Blocks can become invalid during erasing and programming at run-time or during the manufacturing process. Bad blocks generated during manufacturing process are marked bad by the manufacturer. Bad blocks detected at run-time are marked as bad by monitoring the status register in the NAND flash chip during an erase and program operation. A block that is marked as bad is skipped and no longer used.
The File System Component implements bad block management for all kinds of NAND Flash devices (SLC, MLC, TLC):
Error detection and correction codes are used in flash memory to protect data from corruption. All types of error correction codes add redundant information to the data being stored to the flash memory. The redundant information is calculated before the data is written, added to the spare area of the page and then stored in flash memory. When the data is read, the redundant information is recalculated and compared to those stored in the flash.
Error correction codes (ECC) used in the NAND flash memory are block codes. This means that the redundant data bits are calculated for a fixed size block of used data. NTFL is implementing a Hamming ECC algorithm which is able to correct 1-bit and detect 2-bit errors for a fixed size of one sector or 512 bytes for SLC NAND Flashes. The redundant information is calculated in a way that a balance of correction power and efficiency is achieved.
FS_Config_NAND_n.h file, otherwise the File System will not function correctly. Hardware driver or on-chip ECC calculation must be provided in such case.On-Chip ECC is supported on EZ NAND compliant devices. Various ECC layouts are supported by using flexible ECC codeword configuration.
ECC algorithms typically operate on smaller sections of the NAND physical page size. This section can be described as ECC virtual page, with its main and spare part. Typically main part consists of 512 bytes while spare part can be of different sizes (16, 28, 32, etc. bytes).
Amount of protected data is defined by codeword size, which is typically 512 bytes in main area and of various sizes in spare. Spare codeword may also not be aligned to start of spare area, but is located in spare with some offset. Spare codewords may also follow each other in arbitrary order, therefore codeword gap defines the gap between each codeword. It is similar with generated ECC data - its size, offset from start of spare and gap between each other may be arbitrary.
The Embedded File System (EFS) is a proprietary file system used on NOR flash devices. Basic features are:
A NOR flash device memory array is physically divided into sectors or blocks. The File System Component designates them as blocks. Typically, a block's size is 64 kB which is also the smallest erasable unit. Blocks can be further divided, down to memory cells. The memory cell size depends on the device architecture and is 8- (byte), 16- (half word) or 32-bit wide (word). The memory cell architecture also defines smallest programmable unit, which must be maximum 32-bit for use with the Embedded File System.
Embedded File System organizes each block into three regions:
Allocation information region is located on top of a block and describes the block's content. It consists of block signature and file allocation information records, which are written in descending order. Each file has at least one record associated with it. Multiple records belong to files with content and to fragmented files. A file is fragmented when it is modified or its content size exceeds a single block size and must be stored across several blocks. Several small files are stored into a single block.
Each block contains a signature, consisting of 4 bytes and determines if block is:
The file allocation information record consists of 8 bytes and has the following components:
The file allocation information is written when:
The file names & content region is located at the bottom of a block and is fully defined through the file allocation information records. It consists of file names and file content, which can both be fragmented. The first file fragment always starts at the beginning of a block (at offset 0) and is written in ascending order.
In the Embedded File System, a file name consists of maximum 31 characters. Directories are not supported, therefore any file name which contains a directory separator character, such as slash (/) or backslash(\), is rejected as invalid. Other characters are allowed.
Since file fragments are of variable size, create big file fragments in order to reduce the total number of file fragments and make the best use of a block. Writing or appending small amounts of data to a file is not optimal, since such an approach creates a large number of allocation information records. They consume free space and the required processing time results in a slow file access time.
When the file content is modified, the old file content is invalidated and a new file fragment is allocated. a block is erased when all the data stored within has been invalidated.
The following restrictions are applicable to the EFS: