Problem with the new Storage module

[espruino-discuss]

Posted at 2019-01-16 by fanThomas

I am trying out the Sorage module. The example in the documentation works fine on my board. However a small change in the code brought me to a problem:
I would start with an empty file of the required size and then populate the file with content:

var f = require("Storage");
f.write("a", "", 0, 20);
console.log("File-Size: ", f.read("a").length);
f.write("a","Hello",0);
f.write("a"," ",5);
f.write("a","World!!!",6);
print(f.read("a"));

This results in an error:

File-Size:  14
Uncaught Error: Too much data for file size
 at line 1 col 18
f.write("a"," ",5);
                 ^
Uncaught Error: Too much data for file size
 at line 1 col 25
f.write("a","World!!!",6);
                        ^
Hello

And another question: How to determine the size of a file without reading it? The file might be bigger than the available memory and the "list()" method just lists the file name.

[espruino-discuss]

Posted at 2019-01-16 by @gfwilliams

It looks like your issue is f.write("a","Hello",0); -Storage assumes that if the offset is 0 you intend to write a new file, so it just creates a new file with 5 characters.

Instead, just combine the first write and you're sorted:

f.write("a","Hello",0,20);

How to determine the size of a file without reading it?

http://www.espruino.com/Reference#l_Storage_read says:

This function returns a String that points to the actual memory area in read-only memory, so it won't use up RAM.

Basically running f.read("a").length is just as efficient as if there were a special-purpose length function, so we just saved some space and left it out.

---

Posted at 2019-01-16 by fanThomas

Does the size parameter always determine the total file size even when I use it with a offset > 0? E.g.

f.write("a","World!!!",6, 14);

---

Posted at 2019-01-16 by @gfwilliams

It looks like the size gets ignored once the file itself is created.

---

Posted at 2019-01-16 by fanThomas

I found out that it does not get ignored. You can specify always the total file size.

---

Posted at 2019-01-17 by @gfwilliams

Do you have some example code? I did some tests before I wrote that reply and found that it didn't reallocate the file if the size was different.

---

Posted at 2019-01-17 by fanThomas

This is my module saver program. It works now and can be used to save modules stored on a webserver onto esp8266 flash memory. Start the program and then key in on the left side:
s.save('MyModule');

class StModule {
  constructor () {
    this.st = require('Storage');
    this.wifi = require('Wifi');
    this.http = require('http');
    this.server = null;
  }
  
  init (ssid, pw, server) {
    this.wifi.connect(ssid, {password: pw}, (err) => {
      if (err) {
        console.log('Connection error, cannot connect to ', ssid);
      } else {
        console.log('Connected to ', ssid);
        this.server = server;
      }
    });
  }
  
  save(name) {
    var url = 'http://'+this.server+'/'+name+'.js';
    this.st.erase(name);
    this.http.get(url, (res) => {
      var offset = 0;
      var sz = res.headers['Content-Length'];
      res.on('data', (data) => {
        this.st.write(name, data, offset, sz);
        offset += data.length;
      });
      res.on('close', () => {
        console.log('Module '+name+' saved, size: '+this.st.read(name).length);
        console.log(this.st.list());
      });
    });
  }
  
  show(name) {
    print(this.st.read(name));
  }
}

exports = StModule;

var s = new StModule();
s.init('MyWlan', 'mypassword', '192.168.88.117:8081');
//        SSID        Password       webserver's url

I had to find out the size of the module. It was stored in the header of the received code.
then it needs to specify the total module size for each chunk to be written to the flash:

this.st.write(name, data, offset, sz);

Altough some times i get an error which might be from a buffer overrun in the http.get module.

---

Posted at 2019-01-18 by fanThomas

I have found the problem of the mentioned error. After calling Storage.compact() the error vanished.