stat() function on SPIFFS is too slow (IDFGH-934) #3277
Comments
github-actions
bot
changed the title
|
SPIFFS traverses the entire filesystem when doing
Since SPIFFS does have a cache, order of files also matters. Consider 2 cases:
Code used: test_setup();
int file_count = 100;
for (int i = 0; i < file_count; ++i) {
char name[32];
snprintf(name, sizeof(name), "%s%d.txt", filename, i);
test_spiffs_create_file_with_text(name, "foo\n");
}
test_teardown();
test_setup();
spi_flash_reset_counters();
for (int i = 0; i < file_count; ++i) {
char name[32];
snprintf(name, sizeof(name), "%s%d.txt", filename, i);
struct stat st;
TEST_ASSERT_EQUAL(0, stat(name, &st));
TEST_ASSERT(st.st_mode & S_IFREG);
TEST_ASSERT_FALSE(st.st_mode & S_IFDIR);
}
spi_flash_dump_counters();
test_teardown();and in the second test the loop is
In the context of IDF repository, we can't change the logic of SPIFFS. You can increase SPIFFS logical page size, which will result in fewer pages in the filesystem, and fewer read operations. With 512 byte page size, above "worst case" test results in 23k read operations. This is not orders of magnitude of improvement you are probably looking for, though. Flash read operations are expensive on the ESP32 in dual core mode, and even more so if PSRAM support is enabled. It is possible to do some caching of flash reads, e.g. by reading whole 4kB sectors when SPIFFS only asks for 256 bytes. However this approach doesn't bring performance improvement, unfortunately. In the "worst case" test, such modification results in 13k flash read operations, but because each operation reads more data, the total time is increased to 10 seconds. In the end, if you need better |
|
OK, I understand, I will try to use FATFS, but... ...my partition is about 2 Mbytes, only 5 files and only 9538 bytes used: If I'm right stat() needs to read only 5 blocks (maximum) to find a file and all information to fill stat structure. I'm really confused that reading of 5 blocks may long 15 seconds. If it is so there no ways to use SPIFFS on ESP32 at all for any purposes? Thanks |
@no1seman Please try enabling |
…f#3277) * Add Added unimplemented getCharacteristicsByHandle function Defined in BLERemoteService.h But, Not included in BLERemoteService.cpp * Delete log output
Problem Description
I my program logic I have to check file existance during config checks processing (see code in Code to reproduce this issue section). When I call is_file_exist() function during boot (before setting up Network, BLE, MQTT, etc. I've got 1,333 seconds delay for each check of file existance and when I call this function later from http handler - delay becomes terrible, about 14 seconds.
It seems to me that 1,33 for stat() - is inadmissible delay for such a function and 14 seconds - is out of my understanding.
Expected Behavior
stat() on SPIFFS must work rather faster in all circumstances.
Actual Behavior
stat() on SPIFFS tackes from 1,33 to 15 seconds acording to MCU payload or if there any other way to check file existance faster?
Steps to repropduce
call is_file_exist() after mounting SPIFFS and enabling WiFi and/or Ethernet, BLE, MQTT ...
sdkconfig.zip
Code to reproduce this issue
bool is_file_exist(char *filename)
{
ESP_LOGI(TAG, "Enter is_file_exist()");
if (is_filesystem_mounted())
{
ESP_LOGI(TAG, "Exit is_file_exist()");
return false;
}
Debug Logs
Other items if possible
The text was updated successfully, but these errors were encountered: