How to use LittleFS with ESP-IDF?

[dumarjo]

The LittleFS library seams not to be compiled in when using platform=arduino,espidf.

This result in link error.

d:/.../packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio/build/esp32/src/main.o:(.literal._Z5setupv+0x14): undefined reference to `LittleFS'
d:/.../packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio/build/esp32/src/main.o:(.literal._Z5setupv+0xa0): undefined reference to `fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'
d:/.../packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio/build/esp32/src/main.o: in function `setup()':
D:\.../src/main.cpp:39: undefined reference to `fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'

#include "LittleFS.h"

void setup() {
   LittleFS.begin();
}

void loop(){
}

I have started the project from the espidf-arduino-blink example.

regards,

[Jason2866]

You have to add as component, it is not included in IDF. This one is used https://github.com/joltwallet/esp_littlefs.git

[dumarjo]

How to add it with platformio ?

[Jason2866]

Git clone in components folder. I have done this way https://github.com/tasmota/esp-idf/tree/Tasmota/v4.4

[dumarjo]

I try to find a solution to do it with platformio and not forking the ESPIDF and make modification. For now, we will using the SPIFFS since we don't need to write the FS often (once in the life of the product).

Thanx @Jason2866 for your input !

[wijnsema]

I have this problem too. My way of reproducing this problem:

• Start with: https://github.com/espressif/arduino-esp32/tree/master/libraries/LittleFS/examples/LITTLEFS_PlatformIO
• This project will build OK
• Change platformio.ini file line framework = arduino to framework = arduino, espidf
• Fix some minor problem with this platform (CONFIG_FREERTOS_HZ 1000)
• Now build the project and the error

undefined reference to `LittleFS'

will appear, amongst similar messages.

[Jason2866]

The littlefs component needs to be added as component in the Platformio setup.
Component LittleFS is not included in IDF!

[wijnsema]

Thank you Jason. I'm not sure how to this though. LittleFS may not be included in IDF, but it is included in Arduino framework, which confuses me.

[Jason2866]

Yes, but when you compile Arduino as component in IDF no precompiled lib from Arduino is used (where the LittleFS part is included). This is the reason why doing this compile variant.
The support of LittleFS in Arduino is there because it is added as component in IDF in the Arduino Lib Builder. Seen here https://github.com/espressif/esp32-arduino-lib-builder/blob/master/tools/update-components.sh
where all extra IDF components are added which are later in Arduino

[wijnsema]

Just to be practical: what should one do to use LittleFS in an arduino, espidf project in PlatformIO?

[Jason2866]

Make components folder in root of your project and place LittleFS there.
In the LittleFS folder, edit CMakeLists.txt to be included in the project.

[Jason2866]

Will do an example setup (the next days) and will provide a link too.

[dumarjo]

HI, I think this is the new feature that is supported. check here https://docs.platformio.org/en/latest/frameworks/espidf.html#esp-idf-components

[Jason2866]

@dumarjo not new ;-)

[wijnsema]

Again starting with: https://github.com/espressif/arduino-esp32/tree/master/libraries/LittleFS/examples/LITTLEFS_PlatformIO

Change platformio.ini file line framework = arduino to framework = arduino, espidf

Now add components/littlefs folder and add source files from https://github.com/littlefs-project/littlefs

Add CMakeList.txt file to this folder, with following content:

idf_component_register(SRCS "lfs.c" "lfs.h" "lfs_util.c" "lfs_util.h")

The CMakeList.txt in the root of the project:

cmake_minimum_required(VERSION 3.16.0)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
list(APPEND EXTRA_COMPONENT_DIRS littlefs)
project(LITTLEFS_PlatformIO)

The result:

Executing task: C:\Users\wijns\.platformio\penv\Scripts\platformio.exe run 

Processing esp32 (platform: espressif32; board: esp32dev; framework: arduino, espidf)
----------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (6.0.1) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 @ 3.20006.221224 (2.0.6)
 - framework-espidf @ 3.40403.0 (4.4.3)
 - tool-cmake @ 3.16.4
 - tool-esptoolpy @ 1.40400.0 (4.4.0)
 - tool-idf @ 1.0.1
 - tool-mconf @ 1.4060000.20190628 (406.0.0)
 - tool-ninja @ 1.9.0
 - toolchain-esp32ulp @ 1.23500.220830 (2.35.0)
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Warning! Arduino framework as an ESP-IDF component doesn't handle the `variant` field! The default `esp32` variant will be used.
Reading CMake configuration...
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 0 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Compiling .pio\build\esp32\components\littlefs\lfs.o
Compiling .pio\build\esp32\components\littlefs\lfs_util.o
Archiving .pio\build\esp32\esp-idf\littlefs\liblittlefs.a
Generating project linker script .pio\build\esp32\sections.ld
Linking .pio\build\esp32\firmware.elf
c:/users/wijns/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32\src\main.o:(.literal._Z5setupv+0x14): undefined reference to `LittleFS'  
c:/users/wijns/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32\src\main.o:(.literal._Z5setupv+0x4c): undefined reference to `fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'
c:/users/wijns/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32\src\main.o: in function `setup()':
c:\users\wijns\.platformio\packages\toolchain-xtensa-esp32@8.4.0+2021r2-patch5\xtensa-esp32-elf\include\c++\8.4.0\ext/atomicity.h:49: undefined reference to `fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\esp32\firmware.elf] Error 1
=============================================== [FAILED] Took 23.85 seconds ===============================================

I have to admin that I have no idea what I'm doing since I know little about CMake.

What am I doing wrong here?

[Jason2866]

@wijnsema Working dummy LittleFS IDF/Arduion example https://github.com/Jason2866/Arduino_IDF_LittleFS

[wijnsema]

Thanks @Jason2866

This project is indeed working properly.

A couple of things to note when integrating this into your running project:

• I had to 'upgrade' to the development version of the PlatformIO Espressif platform in my current project. So you need
  platform = https://github.com/platformio/platform-espressif32.git
  instead of
  platform = espressif32

• If you just drop in the components/esp_littlefs folder you might need to force a recompile by deleting the build folder first

[Jason2866]

There are no changes needed. The platformio.ini from my example project just looks like this

[wijnsema]

Sure Jason, your project runs fine. It's just a reminder that people with existing projects that want to use LittleFS should realize that the development branch of 'platform' espressif32 is needed.

[Jason2866]

No, it does work fine with standard release. There is no issue.

[nelfata]

Hi @Jason2866, I am trying to build your latest code and I keep on getting the following error:

Linking .pio\build\esp32dev\firmware.elf c:/users/nef/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\src\main.o:(.literal._Z5setupv+0x14): undefined reference to LittleFS'
c:/users/nef/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\src\main.o:(.literal._Z5setupv+0x4c): undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)' c:/users/nef/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\src\main.o: in function setup()':
c:\users\nef.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\ext/atomicity.h:49: undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)' collect2.exe: error: ld returned 1 exit status *** [.pio\build\esp32dev\firmware.elf] Error 1

I can't seem to figure it out. Can you please take a look and let me know? I would like to have LittleFS working with: framework = arduino, espidf

Thanks.

[Jason2866]

@nelfata Here is an working example setup

[nelfata]

Thanks for the prompt reply. I did try to clone your latest repository and built it as it is (pro run), but I mentioned that error I am having in my previous post. Am I missing something?

[Jason2866]

@nelfata For some strange reasons the idf component manager does not work with official platform-espressif32
It does work well with my fork?! Anyways this branch does work

[nelfata]

OK great, that worked. Thanks again.

[Jason2866]

@nelfata Found the reason for not working components manager :-) Branch main is working now too

[nelfata]

Yes that worked too, thanks for the follow up.
I did run the tests that your provided on ESP32-S3 and here is the output (just FYI):
`
Listing directory: /
Creating Dir: /mydir
Dir created
Writing file: /mydir/hello2.txt

• file written
  [ 454][E][vfs_api.cpp:105] open(): /littlefs/mydir/newdir2/newdir3/hello3.txt does not exist, no permits for creation
  Create missing folders of: /mydir/newdir2/newdir3/hello3.txt
  Writing file to: /mydir/newdir2/newdir3/hello3.txt
• file written
  Listing directory: /
  DIR : mydir LAST WRITE: 1969-12-31 23:59:59
  Listing directory: mydir
  [ 655][E][vfs_api.cpp:29] open(): mydir does not start with /
• failed to open directory
  Deleting file: /mydir/hello2.txt
• file deleted
  Deleting file and empty folders on path: /mydir/newdir2/newdir3/hello3.txt
• file deleted
  Removing all empty folders on path: /mydir/newdir2/newdir3/hello3.txt
  Removing Dir: /mydir
  [ 799][E][vfs_api.cpp:250] rmdir(): /mydir does not exists or is a file
  rmdir failed
  Listing directory: /
  Writing file: /hello.txt
• file written
  Appending to file: /hello.txt
• message appended
  Reading file: /hello.txt
• read from file:
  Hello World!
  Renaming file /hello.txt to /foo.txt
• file renamed
  Reading file: /foo.txt
• read from file:
  Hello World!
  Deleting file: /foo.txt
• file deleted
  Testing file I/O with /test.txt
• writing................................................................
• 1048576 bytes written in 14534 ms
• reading................................................................
• 1048576 bytes read in 515 ms
  Deleting file: /test.txt
• file deleted
  Test complete
  `

[nelfata]

Just FYI. You may want to mention that in order to upload the files in the data directory it is required to add to platformio.ini the following: board_build.filesystem = littlefs
And to perform the upload: pin -t uploadfs
Otherwise the upload defaults to SPIFSS and no files will show up even though the upload was successful.

[borisqbf]

Hi @Jason2866
I am using ESP Mail Client (mobizt/ESP Mail Client@^3.4.14) and getting the same LittleFS linker error:

.pio\build\esp32dev\libf72\libESP Mail Client.a(ESP_Mail_Client.cpp.o):(.literal._ZN5MB_FS10createDirsE9MB_String22mb_fs_mem_storage_type[_ZN5MB_FS10createDirsE9MB_String22mb_fs_mem_storage_type]+0x8): undefined reference to LittleFS' .pio\build\esp32dev\libf72\libESP Mail Client.a(ESP_Mail_Client.cpp.o):(.literal._ZN5MB_FS13openFlashFileERK9MB_String15mb_fs_open_mode[_ZN5MB_FS13openFlashFileERK9MB_String15mb_fs_open_mode]+0x14): undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'
.pio\build\esp32dev\libf72\libESP Mail Client.a(ESP_Mail_Client.cpp.o): in function MB_FS::flashReady()': /.pio/libdeps/esp32dev/ESP Mail Client/src/./extras/MB_FS.h:331: undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'
.pio/libdeps/esp32dev/ESP Mail Client/src/./extras/MB_FS.h:331: undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)' .pio/libdeps/esp32dev/ESP Mail Client/src/./extras/MB_FS.h:331: undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)'
.pio/libdeps/esp32dev/ESP Mail Client/src/./extras/MB_FS.h:1058: undefined reference to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)' .pio\build\esp32dev\libf72\libESP Mail Client.a(ESP_Mail_Client.cpp.o):C:\Working\mcu\IrrigationController/.pio/libdeps/esp32dev/ESP Mail Client/src/./extras/MB_FS.h:331: more undefined references to fs::LittleFSFS::begin(bool, char const*, unsigned char, char const*)' follow

I have followed suggestions from this posts but so far no luck. My platform.ini is like this:

[env]
platform = https://github.com/tasmota/platform-espressif32/releases/download/2023.10.02/platform-espressif32.zip
framework = arduino, espidf
monitor_speed = 115200

[env:esp32dev]
board = esp32dev
extra_scripts = ./littlefsbuilder.py
board_build.partitions = partitions_custom.csv
debug_tool = esp-prog
debug_init_break = tbreak setup
upload_protocol = esp-prog
lib_deps =
psychogenic/Chronos@^1.2.0
mobizt/ESP Mail Client@^3.4.14
jchristensen/Timezone@^1.2.4
paulstoffregen/Time@^1.6.1

Any ideas?

[Jason2866]

Link to repo with the project?

[borisqbf]

https://github.com/borisqbf/mcu/tree/master/IrrigationController

managed_components folder (not under source control) includes keep.me file and littlefs folder, which is a clone of main branch of https://github.com/littlefs-project/littlefs

[Jason2866]

@borisqbf Sorry, as long the project code does have compile errors in code, i will not look into.

Processing esp32dev (board: esp32dev; platform: https://github.com/tasmota/platform-espressif32/releases/download/2023.10.02/platform-espressif32.zip; framework: arduino, espidf)
------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (2023.10.2) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 2.0.13 
 - framework-espidf @ 3.40406.0 (4.4.6) 
 - tool-cmake @ 3.21.3 
 - tool-esptoolpy @ 1.40700.0+dev (4.7.0) 
 - tool-mklittlefs @ 3.2.0 
 - tool-mkspiffs @ 2.230.0 (2.30) 
 - tool-ninja @ 1.9.0 
 - tool-openocd-esp32 @ 2.1100.20220706 (11.0) 
 - toolchain-esp32ulp @ 1.23500.220830 (2.35.0) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Warning! Arduino framework as an ESP-IDF component doesn't handle the `variant` field! The default `esp32` variant will be used.
Reading CMake configuration...
Generating assembly for certificate bundle...
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 34 compatible libraries
Scanning dependencies...
Dependency Graph
|-- Chronos @ 1.2.0
|-- ESP Mail Client @ 3.4.14
|-- Timezone @ 1.2.4
|-- Time @ 1.6.1
|-- ESPmDNS @ 2.0.0
|-- WebServer @ 2.0.0
|-- WiFi @ 2.0.0
Building in release mode
Compiling .pio/build/esp32dev/src/Irrigation.o
Compiling .pio/build/esp32dev/src/IrrigationController.o
Compiling .pio/build/esp32dev/src/WebController.o
Compiling .pio/build/esp32dev/src/WiFiController.o
Generating LD script .pio/build/esp32dev/memory.ld
Compiling .pio/build/esp32dev/app_trace/app_trace.o
Compiling .pio/build/esp32dev/app_trace/app_trace_util.o
Compiling .pio/build/esp32dev/app_trace/host_file_io.o
Compiling .pio/build/esp32dev/app_trace/gcov/gcov_rtio.o
Compiling .pio/build/esp32dev/app_update/esp_ota_ops.o
Compiling .pio/build/esp32dev/app_update/esp_app_desc.o
Compiling .pio/build/esp32dev/asio/asio/asio/src/asio.o
Archiving .pio/build/esp32dev/esp-idf/app_update/libapp_update.a
In file included from .pio/libdeps/esp32dev/Chronos/src/chronosinc/DateTime.h:34,
                 from .pio/libdeps/esp32dev/Chronos/src/Chronos.h:79,
                 from src/WebController.cpp:2:
.pio/libdeps/esp32dev/Chronos/src/chronosinc/../chronosinc/timeTypes.h:46:9: error: 'tmElements_t' does not name a type; did you mean 'eth_event_t'?
 typedef tmElements_t TimeElements;
         ^~~~~~~~~~~~
         eth_event_t

[borisqbf]

Fair enough. The issue is caused by ambiguity in time.h header used by Chronos library. Looks like this lib is out of maintenance and the authors have not replaced time.h with TimeLib.h after Arduino's Time library header Time.h has been renamed to TimeLib.h to avoid a conflict with standard time.h on Case Sensitive operating systems. Excuses aside, I have removed Chronos dependency and added modified Chronos lib source to the source control. Hope it gives you something to work with, in a meantime I'll think about better ways to resolve the issue

[Jason2866]

@borisqbf With this sdkconfig.defaults

CONFIG_AUTOSTART_ARDUINO=y
# CONFIG_WS2812_LED_ENABLE is not set
CONFIG_FREERTOS_HZ=1000
CONFIG_MBEDTLS_PSK_MODES=y
CONFIG_MBEDTLS_KEY_EXCHANGE_PSK=y

CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_SIZE=y
CONFIG_COMPILER_OPTIMIZATION_SIZE=y

it does compile and links successfully

To avoid such lib issues we have decided in our project Tasmota not to use any external dependencies.
We place all needed libs (many are modified) in the folder lib in the root of the Platformio project folder

[borisqbf]

the problem still evident even after adding these two lines to sdkconfig.defaults. I guess the problem is in the content of managed_components folder, because I have nuked my project folder and did fresh pull from the repo before attempting to re-build.
I am adding managed_components to the repo, perhaps you could spot what is missing there. Also I can't see any traces of lfs.c, etc being compiled. Looks like managed_components folder is ignored by the compiler...
And thank you for a tip on how to handle ext deps

[Jason2866]

Add managed components by addingidf_component.yml See as example https://github.com/Jason2866/Arduino_IDF_LittleFS/blob/main/src/idf_component.yml
So nothing in the folder. Only the file .keep.me

[borisqbf]

Yes! Adding idf_component.yml works. Also a reminder to myself (and may be others) When changing something in project configuration/CMake options, delete build folder to force recompile. Thank you @Jason2866

[garudaonekh]

@Jason2866 Your solution works. I spent several days tackling this issue.

[hitecSmartHome]

Yes! Adding idf_component.yml works. Also a reminder to myself (and may be others) When changing something in project configuration/CMake options, delete build folder to force recompile. Thank you @Jason2866

But where is the build folder?

[hitecSmartHome]

I have added idf_component.yml to my src directory, added managed_components folder with .keep.me file.
If I modify almost anything in my project, I have to compile -> get the reference LittleFSFS error -> clean project -> compile again.
If I clean first and compile after, it does not work. Is there a workaround? I often modify my pio.ini file and things like that because I'm experimenting and toggling stuff in the menuconfig. I always have to do this round trip to compile my sketch with LittleFS.
Really annoying. Also the compile time grow so much when I switched to Arduino as a component of IDF I can't even belive it. Sometimes the compile takes minutes!

[valeros]

@hitecSmartHome IDF is a large framework with hundreds of files (not to mention Arduino). Each time you modify platformio.ini the build folder is cleaned and hence IDF is build again from scratch. To speed things up, you can cache your build folder via the build_cache_dir option (similarly to ccache) so that files won't be compiled again, for example:

[platformio]
; Set a path to a cache folder
build_cache_dir = ./cache

[env:esp32dev]
platform = espressif32
framework = espidf
board = esp32dev

[hitecSmartHome]

Wow. Thank you I will definitely try this out!

[hitecSmartHome]

Well, it doesn't seem to work.

Warning! Ignore unknown configuration option build_cache_dir in section [env:esp-wrover-kit]

[valeros]

build_cache_dir is a global option and should be set in the platformio section, see my example above.

[hitecSmartHome]

Oh I see. Sorry for my stupidity. WIll try.