File LFS Lua module initial commit

[vsky279]

Fixes #3329.

Make sure all boxes are checked (add x inside the brackets) when you submit your contribution, remove this sentence before doing so.

• This PR is for the dev branch rather than for the release branch.
• This PR is compliant with the other contributing guidelines as well (if not, please describe why).
• I have thoroughly tested my contribution.
• The code changes are reflected in the documentation at docs/*.

Support for arbitrary files stored in LFS

Usage

The module is intended to overlay standard file module. If file is present in SPIFFS then this version is preferred. If it is not present then read only access (of course) to the file in LFS is provided.

The following functions for LFS files are supported:

• file.exists()
• file.open()
• file.getcontents()
• file.read(), file.obj:read()
• file.readline(), file.obj:readline()
• file.seek(), file.obj:seek()

Other functions are just passed-through to the standard file module.

When file is embedded in LFS, no standard userdata file structure is created. Instead a Lua object is created.

Example

file = require("file_lfs")

local idx = file.open("index.html") -- file not in SPIFFS but in resource.lua in LFS
print(idx:readline())
idx:seek("set", 0)
print(file.readline(idx))

idx = file.open("favicon.ico", "r") -- file not in SPIFFS but in resource.lua in LFS
print(file.read(512))
file.seek(idx, "cur", 100)
print(idx:read(16))
idx:close()

local i = file.open("init.lua") -- files in SPIFFS are accessible using same routines
print(i:readline())

Also test file for the testing framework (#2983) is provided.

[pjsg]

I like this -- I built something similar (though not so general purpose) for an application that I am working on. This is definitely a better approach!

[HHHartmann]

@vsky279 could you please not force push? We can squash the commits when merging and the reviewer will have to read everything over again and again.

[vsky279]

Sure, no problem. So I will pile up commits that we squash once ready to be merged.

[nwf]

I'm delighted to see NTest catching on! :D

[HHHartmann]

Please also review my comments in #3329
I feel that a file enhancement should work as usual and throw errors where normal operation can not be guaranteed.
Given the following code

if file.exists("myfile") then file.rename("myfile", "myotherfile") end
print(file.getcontents("myotherfile"))

should work, regardless of myfile being in SPIFFS or LFS.
An alternative would be to throw a meaningful error that the operation is not possible.
Atm copy from a non existing file throws an error (I hope) but with puzzeling message.

Maybe some code to smoothly access the resource like a file might be more useful without blending with file.

[vsky279]

Thanks Gregor for all these comments!

Please also review my comments in #3329

I'm sorry I missed these ones.

open modes

I suggest the following LFS file behavior:

• "r": standard mode
• "w": opens file of the same name on SPIFFS
• "a", "r+", "w+", "a+": copy LFS file content to SPIFFS and opens with given open mode

file.getcontents()

We already have file.getcontents() - this needed to be implemented for the test suite.

file.rename()

Renaming non existing file returns false rather than raising error. So I guess an attempt to rename LFS file should return false too.
It's implemented now. But the implementation is a bit useless as calling directly file.rename with LFS filename (as if file_lfs.rename was not implemented) would return false anyway.

file.stat()

 :thumbsup:

file.open("index.html")
print(file.readline())

Oups. I didn't know this approach is still possible. Implemented.

[HHHartmann]

Sorry for the long delay.
Now again reviewed and tested and found some issues but then we should be done I think.

[vsky279]

Gregor, thanks for this in-deep review. 👍
Everything should be handled now.

[vsky279]

I can't find anywhere what is the limitation for the string length in LFS.
53kB string is too long - LFS image that includes such string cannot be flashed.

[HHHartmann]

Looking forward to this working. Will use it in my current project to speed up startup.

[vsky279]

Can we merge this? I think everything has been adressed.

[HHHartmann]

Yes. Sorry for the delay