httpUpdateSPIFFS - how to get and save SPIFFS

[odilonafonso]

Basic Infos

Hardware

Hardware: ESP-12 NodeMCU / Wemos D1
Core Version: 2.3.0-rc2

Description

httpUpdate is working well; now I want to update the SPIFFS too.
I have read the httpUpdateSPIFFS from the samples and look simple. But I do not know how to load and save the "spiffs.bin" file.

In the IDE we have:
Tools -> "ESP8266 Sketch Data Upload" - to load data from a file to the SPIFFS
and
Sketch -> Export Compiled Binary- to save the sketch bin file for upload to my web server

I do not know if there is a tool to copy the contents of my spiffs ESP or I'll have to do it myself.
It is also unclear if copying only the data recorded on spiffs or if I provide a "spiffs.bin" with the total size of the available area for spiffs on my ESP.

Someone please know how to do this?
@krzychb, you being an expert in OTA, you know how to do this?
Thanks in advance !!

[WereCatf]

There is a command-line tool called "mkspiffs" that is used to make those SPIFFS-images. I have never manually used it, so I can't give much more info on that, but call it and check what arguments it supports. The resulting image will have to be the size of your whole SPIFFS, not just the size of the files on it, so e.g. on a Nodemcu with 1MB/3MB split you'll end up with a 3MB SPIFFS-image.

[odilonafonso]

Hi @WereCatf, thank you !!

Yes, I gave it a read and seems to be the right tool to use !!

I'll test it and post the results here.

[odilonafonso]

Unfortunately I could not understand the workings of "mkspiffs" to make a copy of my ESP SPIFFS área to a file.

Please @igrr,

It is possible to use "mkspiffs" to load an SPIFFS image of my ESP in a file?

For a next step using httpUpdateSPIFFS () function from ESP8266HttpUpdate library?

I read mkspiffs help, but don't know how to make this.

Thank you for help

[WereCatf]

To create a 3MB SPIFFS-image you'd use the following:
mkspiffs -c directory_whose_contents_you_want_in_the_SPIFFS_image -b 8192 -s 3125248 image.spiffs

[odilonafonso]

Ahhhh OK @WereCatf.

But that's not what I want to do.

I would like to access the ESP and bring its SPIFFS contents in a directory.

Like: Sketch --> Export compiled Binary.

Nevertheless it may be useful; I can for example create some other way a content equal to my ESP and (assuming I do it in the data directory of my sketch):

mkspiffs -c data -b 8192 -s 3125248 image.spiffs

OK, thank you very much for the help!

[WereCatf]

OTA-updates and httpUpdateSPIFFS are for sending content to the ESP, not for receiving content from it. If you want to receive the SPIFFS-filesystem that is already on your ESP you will have to write the code all by yourself.

[odilonafonso]

OK

[tonu42]

It is an interesting proposition to read the files back, as that could be used to get recorded data back, 3 MB is a lot a of data depending on frequency of data recording.

If you know the file names you can read them back by serving them up to an HTTPServer. The esp is not like a computer where you can have files existing unless you actually made them so I think it would be feasible. Could even make it a form to get back only what you want.

[mrhurless]

@WereCatf said:

To create a 3MB SPIFFS-image you'd use the following:
mkspiffs -c directory_whose_contents_you_want_in_the_SPIFFS_image -b 8192 -s 3125248 image.spiffs

I was curious if that is the actual file you would point the .updateSpiffs() to, or does it specifically need to be a .bin file (output of mkspiffs is .spiffs)? Thanks in advance for any help!

[HugoML]

I have the same puzzle.
mkspiffs creats image.spiffs, but how to create the .bin file?
I couldn't find the verbose mode line showing the process command. Please advice.
And, how to calculate the block size?
Using NodeMCU 4M(3M SPIFFS):
-b block_size 8192
8192 = 81024 byte (can we change this to smaller size , such as 2K = 2048 bytes?)
-s image_size 3125248
3125248 != 31024*1024 byte (3MB) . Why is this?

for 4M(1M SPIFFS): the spiffs makes the file size (on PC) is 1,028,096 bytes. (!= 1024*1024)

[captainreptile]

@HugoML did you figure it out how to make the bin file?

[mrcodetastic]

@captainreptile I have never got Igor's mkspiffs tool to work properly, even with the correct parameters, the bin file when used with the ESPhttpUpdate.updateSpiffs function has resulted in a garbage SPIFFS file being loaded and the ESP then re-creating a blank file system.

A much simpler workaround is to simply use Arduino on Windows.

When you run the ESP8266 Sketch Data Upload tool, you will quickly see in the Arudino terminal output, something along the lines of:

/file.html

[SPIFFS] upload : C:\Users\USER\AppData\Local\Temp\build87a71177855ef0141b9b3758467a3f0f.spiffs/Project.spiffs.bin
[SPIFFS] reset : nodemcu
[SPIFFS] port : COM5
[SPIFFS] speed : 921600
[SPIFFS] address: 0x300000

Take note of that file path (C:\Users\USER\AppData\Local\Temp etc.) and go immediately to it to copy the .bin file that works!

[HugoML]

@mrfaptastic This works quite well.
@captainreptile Command line I successfully tried:
For nodeMCU with 4M flash (configured 3M File System)
esptool –cd nodemcu –cb 115200 –cp COM24 –ca 0x100000 –cf yourArduino.spiffs.bin
For generic ESP8266 with 4M flash: (configured 3M File System)
esptool –cd ck –cb 115200 –cp COM24 –ca 0x100000 –cf yourArduino.spiffs.bin
Note: 0x100000 is the starting address of your file system. Depends on how you allocate it, the address is different.

[HugoML]

And, according to document on file system, there is a reserved sector for EEPROM (4KB). So the calculation for SPIFFS starting address is better to be observed in Arduino output window.