read/prog sizes for SPI NAND flash with internal HW-buffer #277
Comments
|
I can't remember where, but I saw geky saying that smaller read/prog size can improve performance. |
|
So this is where the difference between cache_size vs read_size becomes useful. littlefs reads in multiples of the read_size (the same is true for writing). So it's more an alignment field than an upper-limit. What this means is if you have cache_size=2048, read_size=4, and you write to a large file, littlefs will send the data to the block device in 2048 byte chunks aligned to 4 bytes. Small files and metadata will be written out in smaller chunks, but still aligned to 4 bytes. So what you can do, is set cache_size=2048, read_size=1, and write a greedy read function. This is one of the best ways to take advantage of these small buffers without limiting the block device. int my_read(const struct lfs_config *c, lfs_block_t block,
lfs_off_t off, void *buffer, lfs_size_t size) {
uint8_t *data = buffer;
while (size > 0) {
if (off % MY_PAGE_SIZE == 0 && size >= MY_PAGE_SIZE) {
// read a page
do_page_read(block, off, buffer);
buffer += MY_PAGE_SIZE;
off += MY_PAGE_SIZE;
size -= MY_PAGE_SIZE;
} else {
// read a byte
do_byte_read(block, off, buffer);
buffer += 1;
off += 1;
size -= 1;
}
}
return 0;
}
Does the NAND device support rewriting some bits that way? Sometimes flash parts only support a single write after erase. May be worth checking. If not it's find for read_size != prog_size. As for as performance: The smallest prog size improves wear on the device, as less bytes are wasted on alignment. In theory, a smaller prog size also means fewer garbage collection cycles, which improves performance. It also means a smaller amount of bytes sent over the wire, though this may be inconsequential. However, we are finding scalability issues with the number of commits in a directory, and smaller prog size = more commits (#203). This is especially an issue for NAND devices, where blocks are very large. So if you need performance, the best answer is to experiment. You may find that small prog size turns into too many commits for fetching to run efficiently. It mostly depends on your use case. |
|
I have implemented the read according to your proposed "greedy" function, and it works well. Lfs seems to either read only a few bytes, or a complete cache-size of 2kb. So far I have only tested it with small files, but eventually I want to use up to about 1MB per file (but only a small number of files).
According to the datasheet it supports up to 4 partial writes per page. So I guess this could be forced by setting prog-size to a quarter page. For now I have just left prog-size == page-size.
Is there any chance that this is enforced by lfs? Or maybe also just working by chance? So far I have not run into an issue... |
That's a clever solution. 👍
This is guaranteed by littlefs. 👍 Because of how the erase -> prog two step write works, it's actually kinda challenging to make a filesystem that writes to prog pages in an erase block out of order. I don't know of any filesystems that don't write pages sequentially. |
|
Thank you to everyone that has contributed here. This information has helped me create an SPI NAND flash driver. There is an edge case that I am uncertain about and would like to seek some input. Here are the device specific parameters: And here are the LFS configuration parameters: At the moment, my driver commits the data from the LFS "prog" callback to the on chip hw buffer. If the "prog" spans across a page boundary of the device, each page is committed to flash every time "prog" crosses the page boundary. The last partial page of data is not committed to flash until the LFS "sync" callback. I think I understand that LFS will "sync" after a "prog". Is there ever a case where LFS could "read" from a different page between the "prog" and the "sync"? Do I need to check for this case during "read" and sync the hw buffer before loading a new page if there is a "pending sync"? |
|
I've put the read boot count working, but it's not working well. It counts well until a non fixed number (73, 79, 80, 84 or another one around theses ones) and then, the LFS erases Block 0 (where the file is being accessed). After erasing it, it count number is not read correctly. Is it normal? Here are the logs:
|
|
littlefs can manage the bad blocks of nand? |
|
Just a small comment here. It may be worth seeing what effects I got from modifying write_size for an even larger NAND flash with 4096 byte pages and 64x4096 byte erase-blocks. #661 I also want to warn about the "internal programming buffer" and sync, as it appears sync is only littlefs synch, and non flash-sync. Code does not show any calls to any sync after calling lfs_bd_flush() which relies on the .prog configured for the block device. |
|
|
I have not looked very hard at it but to me the idea of calling synch here seems reasonable. Also I did make a mistake as I have found the call to lfs->cfg->sync in lfs_bd_sync() which is the underlying block device sync. This is what is required to make sure any caching in partial program buffers etc is committed to memory. |
Hi,
I am trying to use littlefs on a SPI nand flash device. This device has an internal buffer of 1 page (2048 bytes), while the erasable block size is 64 pages.
Reading/Writing from the flash memory to the hw-buffer is done in full 2048 byte pages, so my first guess for read_size and prog_size was 2048.
However, SPI-access to the hw-buffer can be done one byte at a time. This means that I could set the read_size to 1. In the same way I could set the program size to 1 as well. If lfs tried to write less than a full page, I could first reset the hw-buffer, write only some part, and then program the buffer content to flash.
So I guess technically I could choose any value from 1 to 2048 for the read_size and prog_size, is there any guidelines on what would be an useful value?
The text was updated successfully, but these errors were encountered: