Feature request: support multiple SPIFFs partitions

[robcazzaro]

I'm working on a project where I need to store webpages and settings/logs in a permanent storage. And over time I need to update the webpages, but not the settings and logs (which are device-dependent).

If I use the current implementation of LittleFS, when I perform a web pages OTA update of the SPIFFS, I will lose the settings. I can work around it by temporarily storing all settings in memory, unmounting the SPIFFS, performing OTA update, remounting the SPIFFS and finally restoring the settings. But would be problematic to do the same for the logs.

Would be great if LittleFS provided a way to support multiple SPIFFS partitions, so that I could independently update them

[lorol]

Hi,
I will consider for future.
For your needs, you can keep the settings in different place, see:
https://github.com/espressif/arduino-esp32/tree/master/libraries/Preferences

[robcazzaro]

Thanks for considering the suggestion. And thanks for the pointer to the preferences nvs, which I had long forgotten about. I might be able to partition my data and save the portion of data that doesn't change frequently into nvs. I still believe that for things like logs, having dual SPIFFS partitions would be a nice-to-have for the future.

[lorol]

For multiple data partitions, it should be worked at the core (FS) side. Especially for Arduino it will need some changes to work with more data partitions at same time. Have you seen similar projects with SPIFFS or FAT?

[robcazzaro]

There's an ESP32 Arduino library for multiple SPIFFS partitions https://github.com/lukaswagner/SPIFFS. Plus some information floating on how to do it using the Espressif SDK, e.g. https://esp32.com/viewtopic.php?t=12331.

It requires changing partitions.csv for the device, based on the size of flash memory (in that library, the needed partitions.csv is in the examples folder). It's also simplifying things by only allowing one mounted partition at a time which, if needed, is an acceptable tradeoff.

And I know it's going to sound hopelessly naive, but "it doesn't look like a lot of work" to make it work (says the clueless manager to the developer :). Just kidding... what I mean is that it doesn't seem to require a lot of changes at the system level, outside of having to change partitions.csv, which can be tricky. I think that asking the user to figure out partitions.csv based on their needs is an acceptable tradeoff: the people who need multiple partitions will have to figure out how to partition the SPIFFS portion of flash.

[lorol]

@robcazzaro you can try yourself, this should be very similar to:
espressif/arduino-esp32@3cbfa2f#diff-d033864b5ea50ef1ee43c07dfc1ec7c828f4eff57dee791348fd66e6d588ac78

Just instead of NULL, use LFS_NAME as default name.
Also since LittleFS cannot work with NULL (the name is mandatory) you may skip few if's to check.

This way you should be able to pass different name so different partition.
In your sketch, keep the begin - end fs for only one "active" partition at a time.

If you succeed, just test it and do a PR :)

[robcazzaro]

Thanks! I'm finishing another part of the code right now, but as soon as I'm done, I'll definitely look into this and do a full writeup

[lorol]

@robcazzaro and @tsctrl - Done (multiple partitions, custom name of partition)
Please test and let me know your thoughts.
For now, closing ...

[robcazzaro]

EDIT: I re-run the test with the previous version of LittleFS, and I get the same behavior, I'll open a new issue #27

First of all, thanks for adding this!
Second, apologies for taking this long to test it...

Good news is that it works very well!

I'm running a test, with a ESPAsyncWebserver on the main partition and data/logs on the second. I noticed a problem which I'm not sure it's caused by the new code

After resizing the partitions, the following code

// initalize second partition
  if (!LITTLEFS.begin(true, "/lfs2", 5, "part2"))
  {
    Serial.println("part2 failed! AutoFormatting.");
    return;
  }

results in an error message for both partitions

lib\LittleFS_esp32\lfs.c:1076:error: Corrupted dir pair at {0x0, 0x1}
E (234) esp_littlefs: mount failed,  (-84)
E (238) esp_littlefs: Failed to initialize LittleFS

I expected to see "part2 failed! AutoFormatting." instead (and the same for partition 1). When executing that code again without changing anything, it works as expected, since the partitions are now valid. It looks as if the code correctly formats an un-intialized partition, but doesn't return the correct error code

for what it's worth, here's my partition file

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x120000,
app1,     app,  ota_1,           ,0x120000,
spiffs,   data, spiffs,          ,0xD6000,
part2,    data, spiffs,          ,0xD6000,
#1179648

(yes, I know, the number after # on the last line is wrong, but doesn't seem to be used. And that I'm wasting some flash space, that was just a test)