Modules: are they loaded into RAM?

[espruino-discuss3]

Posted at 2022-01-08 by Andreas_Rozek

I'm currently experimenting with Espruino modules (on a Bangle.js 2).

From the documentation it looks as if modules are written to Flash as source code - non-minified, not pre-tokenized. This sounds as if modules will have to be parsed (and perhaps tokenized) when "require"d for the first time.

Does that mean that modules consume RAM while other code may be run from Flash?

(by the way: is it intended that module file names do not end with a ".js" suffix? Would those three characters also count as the eight which are allowed?

Amendment: a (probably important) detail of my intention is: I want to load modules at run-time(!) and not as part of the right-hand side of the Web IDE! The idea is to load modules as required by a configuration file

[espruino-discuss3]

Posted at 2022-01-08 by @fanoush

by default no, espruino interpreter runs the source directly as per https://www.espruino.com/Performance , that's why it is important to minify and pretokenize before upload. there is however flag pretokenise set by E.setFlags which behaves like this when loading file from storage (?) see https://www.espruino.com/Reference#l_E_getFlags , also there is "ram" attribute in function body that will copy it to ram (for speed critical code).

---

Posted at 2022-01-08 by @fanoush

(by the way: is it intended that module file names do not end with a ".js" suffix? Would those three characters also count as the eight which are allowed?

The limit for Storage is around 20 now and yes whole filename is stored as one string, . or / are not special in any way.

The only exception was added recently if multiple storage areas are in use for nrf52, then there is one optional letter drive name, c: is internal nrf52 flash and any other letter or no drive prefix at all is external larger SPI flash storage. This is for now not used except in custom builds.

Advantage of (smaller) internal flash is that it is directly mapped to CPU memory so it runs in place like from RAM.

---

Posted at 2022-01-08 by Andreas_Rozek

Thank you very much for your quick response!

Documentation wants me to write a module to storage in the following way:

require('Storage').write('SmokeTest',`
  exports.getAnswer = function () { return 42; }
`);

i.e., obviously un-minified and not pre-tokenized.

Could I use the following approach instead?

let exports = {};
  exports.getAnswer = function () {
    for (let i = 0, j = 0; i < 1000; i++) {
      j++; // just consume some time...
    }
    return 42;
  };

let ModuleSource = '';
  for (let Key in exports) {
    if (exports.hasOwnProperty(Key)) {
      ModuleSource += 'exports.' + Key + '=' + exports[Key] + ';';
    }
  }
require('Storage').write('SmokeTest',ModuleSource);

If I just print the contents of ModuleSource, I get

exports.getAnswer=function () {for(let i=0,j=0;i<1000;i++){j++;}return 42;}

which looks quite promising (in terms of minification, at least)...

---

Posted at 2022-01-10 by @gfwilliams

If you upload modules with the IDE (or use the app loader to do it) by default they will be minified.

If you use that method, when required the modules do use up some RAM, but only enough to hold the declarations for every function that you export or that is defined at module scope. So... if your module just exports a single function but that function is huge, the RAW usage will actually be very low and the function is executed from flash.

---

Posted at 2022-01-10 by Andreas_Rozek

That's good to know - thank you very much!

---

Posted at 2022-01-10 by Mark_M

the function is executed from flash

Am I understanding it right? A function is loaded into from flash into RAM line by line while executed? Or is loaded as a whole and then executed and then unloaded? Or CPU is capable to work with flash same as with RAM just slower?

Please do not reply. I am reading https://www.espruino.com/Performance

---

Posted at 2022-01-10 by @fanoush

A function is loaded into from flash into RAM line by line while executed? Or is loaded as a whole and then executed and then unloaded?

no and no. Except when you put "ram" marker into function body or use `E.setFlags({pretokenise:true}), then it is parsed into copy in ram

function is not 'loaded', it gets first parsed when the part with definition is executed like

function x(){
//whatever
}

and later when you call it, it already knows where the string of the body is located (ram or flash)

Or CPU is capable to work with flash same as with RAM just slower?

yes for internal nrf52 flash, that's also where the whole native espruino code is, no for SPI flash, but that one also has special support so the string is streamed from spi flash in small chunks. So in both cases RAM size is not a problem. And in both cases it works mostly the same.

---

Posted at 2022-01-10 by @fanoush

no for SPI flash

or in fact nrf52840 (unlike 52832) can directly map SPI flash to CPU memory and read it transparently like internal flash but it is currently not used and may not improve things much as the interpreter can do it on its own quite easily too when needed

---

Posted at 2022-01-10 by Mark_M

thank you fanoush