What is process.memory giving ?

[espruino-discuss3]

Posted at 2014-01-28 by JumJum

process.memory() gives a total memory of 2000 and free of 1988 (after reset)
After sending a script with about 490 chars it gives free of 1841
Minifyied code has a length of about 270 and free returns 1875

How should I interprete this numbers, bytes seem to be a wrong, and memory units(20 bytes) would be a lot (3000 bytes for 490 chars of code for example) .
dump() did not help either

[gfwilliams]

Posted at 2014-01-28 by @gfwilliams

Yes, memory units of 20 bytes is right. Just for anyone else:

1988 (after reset) - 1841 = 147
147 * 20 = 2940 bytes

Note: I've discovered a bug here, which is that process.memory().history (which is the amount of memory used for the command history) is being calculated wrong, and so that knocks all the calculation of 'free' memory out. My info below is with a development version that now has this fixed. When 1v48 comes out that should have the fix in.

What I think is happening is that the script is being executed, and is generating variables/functions which are using up more space than just the individual characters. If you just had one function that wasn't executed then the situation would be a lot better.

For instance:

>process.memory() 
={"free":1988,"usage":12, ...

function foo() {
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXX
}

>process.memory()
={"free":1953,"usage":47, ...

So the code above (which, incidentally is 490 characters) uses 35 blocks in total which is 700 bytes.

Of that:

blocks	use
1	The function name foo
1	A Block saying 'I'm a function'
1	A Block saying 'I'm a function's code'
31	String data, as Espruino uses 4 bytes out of each 20 byte block for other stuff (490*/16=31)
1	When we typed process.memory() to query the memory, it added one block named process because we referenced it and it wasn't there before. This is something that could be fixed later though

You can type trace() (a debug tool for developers) which outputs the raw structure of the interpreter's data, and that might help to explain where your memory is going...

[espruino-discuss3]

Posted at 2014-01-28 by JumJum

Thanks a lot for this interesting answer, its very late, I appreciate this.
What I wanted to find with this question is the best way to write code.
There are at least two parameter, speed and size.
BTW, my next test used a module.
Base is an object called Blinker, which uses Interval.
Having LED1 to LED3 blinking in different frequency costs 175 blocks.
Same with function in modules takes more than 300 blocks.
Do you have a guide, something like "best architecture to save memory" ?

[gfwilliams]

Posted at 2014-01-29 by @gfwilliams

Hi,

At the moment all the info I have is in the Performance page.

I think to sum it up:

• Put comments in the 'root scope', outside the functions (that way they get thrown away on load)
• Use Uint8Array/etc for storing arrays of values wherever possible
• Try and keep variable/function names less than 8 chars (as in some cases it might cause another var to be used)

You can always look at the result of trace() to get an idea of where your memory is going.

Can't think of much else right now... But if you post up your code then I might notice some way you could save space?

[espruino-discuss3]

Posted at 2014-01-29 by JumJum

Hello,
I tried it (see source) and got memory usage of 199, which is close to 4k or 3k by subtracting 4 bytes/block. Sourcecode is less than 400 bytes only.
Trace is not a help for me, my knowledge about that is too poor.
Checked memory-usage:

1. function Blinker only: usage is 60
2. function Blinker with LED1: usage is 132
3. function Blinker with LED1, LED2: usage is 199
4. function Blinker with LED1, LED2, LED3: usage is 265
5. used Blinker as module and required for LED1: usage is 178
6. used Blinker as module and required for LED1, LED2: usage is 252
7. used Blinker as module and required for LED1, LED2, LED3: usage is 326

function Blinker(pin,frequency){
  var me = this;
  me.iv=-1;
  me.status = true;
  me.dur = 1000 / frequency;
  me.stop = function(){
    pin.write(false);clearInterval(me.iv);
  };
  me.start = function(){
    iv = setInterval(function(){
      me.status = !me.status;pin.write(me.status);
    },me.dur);
  };
  this.start();
}
var b = new Blinker(LED1,2);
var c = new Blinker(LED2,5);

[gfwilliams]

Posted at 2014-01-29 by @gfwilliams

Did you try all this with a version you compiled yourself from Git?

As I said at the start, it turns out that the current version 1v47 has a bug where it reports the used memory incorrectly.

When I try your function in a newer version it reports 26 memory units (=520 bytes) for Blinker.

However: Looking at what's in Blinker, what happens is a new version of start and stop are created each time you instantiate it. If you put start and stop into Blinker.prototypeI think memory usage would be a lot better when blinking multiple LEDs.

[espruino-discuss3]

Posted at 2014-01-29 by JumJum

I have 1v47 from espruino/binaries on Espruino Board.
My understanding was the bug is for history only, sorry.
Already tested prototyping, this works fine and saves memory(about 20% less) even with the buggy calculation.
Anyway I will put any testing for memory consumption on hold, until next version is available, or we get a nightly build.
BTW, would it be possible to send mails around if new version is available ?

[gfwilliams]

Posted at 2014-01-29 by @gfwilliams

I'm trying to tweet when new versions are available... Otherwise the next time you plug the board into the Espruino web IDE it should tell you :)

[espruino-discuss3]

Posted at 2014-02-06 by JumJum

Gave it a new try and got these numbers:
blinker object only, usage 50, history 37
3 blinker with object, usage 158, history 49
3 blinker with object, minified from 504 to 386, usage 149, history:31
3 blinker with require, usage 192, history:55
From my best understanding

• the blinker object itself takes a few bytes only
• each instance with 4 variables(pin,duration,interval,status) takes around 600 bytes
• minifying reduces some bytes
• require takes more memory than an object
  Is there anything wrong with my interpretation ?

function Blinker(pin,frequency){
  this.interval = 0;
  this.status = true;
  this.duration = 1000 / frequency;
  this.pin = pin;
  this.start();
}
Blinker.prototype.start = function(){
  var me = this;
  me.interval = setInterval(function(){
    me.status = !me.status;
    me.pin.write(me.status);
  },me.duration);
};
Blinker.prototype.stop = function(){
  this.pin.write(false);
  clearInterval(this.interval);
};
var b = new Blinker(LED1,2);
var c = new Blinker(LED2,5);
var d = new Blinker(LED3,7);

[gfwilliams]

Posted at 2014-02-06 by @gfwilliams

Hi. You can basically ignore 'history' now - process.usage is all you need.

If you look at trace, each blinker is allocating:

A Blinker instance	12 blocks
An interval	8 blocks
A newly instantiated callback function (including the scope that Blinker.prototype.start was executed in)	~16 blocks

So each new Blinker is a total of 36 blocks = 720 bytes.

When you use require, you could try calling http://www.espruino.com/Reference#l_Modules_removeAllCached after instantiating all Blinkers? That'll probably free the cached module.

IMO Espruino could do significantly better. If we added a Function.prototype.bind function you could skip defining the function inside start() and that'd save you a lot of memory.

[espruino-discuss3]

Posted at 2014-02-06 by user6350

To me is now becoming more clear how the Javascript interpreter works looking at how this simple example code has been parsed and put into memory structures.
I am now asking myself which should be the best way to use Espruino.
In example adding a simple toFixed function would require adding a prototype to Number Class, which is too complex with respect using an existing native C sprintf function, even a big piece of C code itself in the real world.
Probably the best use of Espruino could be to act as a scripted shell coprocessor of a traditional CPU, doing more intensive computations ( i.e. another STm32Fx micro controller ).
It seems that there are pro and cons using Espruino.
Is there a place where to find good books about the theory of a Javascript interpreter or handy manuals about how to build one.
Looking inside the Espruino code should give enough explanation, but to a novice this is the hardest way.

var Circle = function(radius) {
  this.radius = radius;
};

Circle.prototype.area = function() {
  return Math.PI * this.radius* this.radius;
};

var a = new Circle(3);
var b = new Circle(4);
var c = new Circle(1);

a.area(); // .toFixed(2);
b.area(); // .toFixed(2);
c.area();

#1[r1,l1] Object {
  #4[r1,l0] Name: '>timers'  #5[r2,l0] Array [
  ]
  #6[r1,l0] Name: '>watches'  #7[r2,l0] Array [
  ]
  #9[r1,l0] Name: '>history'  #10[r1,l0] Array [
    #11[r1,l0] Name: int 0  #8[r1,l0] String echo(0);
    #24[r1,l0] Name: int 1  #16[r1,l0] String var Circle = function(radius) {\n  this.radius = radius;\n};
    #41[r1,l0] Name: int 2  #30[r1,l0] String Circle.prototype.area = function() {\n  return Math.PI * this.radius* this.radius;\n};
    #50[r1,l0] Name: int 3  #43[r1,l0] String var a = new Circle(3);
    #61[r1,l0] Name: int 4  #54[r1,l0] String var b = new Circle(4);
    #72[r1,l0] Name: int 5  #65[r1,l0] String var c = new Circle(1);
    #75[r1,l0] Name: int 6  #77[r1,l0] String a.area();  
   #82[r1,l0] Name: int 7  #83[r1,l0] String b.area();  
    #86[r1,l0] Name: int 8  #76[r1,l0] String c.area();
    #88[r1,l0] Name: int 9  #89[r1,l0] String echo(1);
  ]

  #12[r4,l0] Name: 'Circle'  #13[r1,l0] Function {
    #14[r1,l0] Param Name: 'radius'  undefined
    #23[r1,l0] Name: '>code'  
    #15[r1,l0] String {\n  this.radius = radius;\n}
    #26[r4,l0] Name: 'prototype'  #25[r1,l0] Object {
      #28[r1,l0] Name: 'area' 
      #29[r1,l0] Function {
        #40[r1,l0] Name: '>code'  
        #36[r1,l0] String {\n  return Math.PI * this.radius* this.radius;\n}
      }
    }
  }
  #42[r1,l0] Name: 'a'  #45[r1,l0] Object {
    #46[r1,l0] Name: '__proto__'  #26[r4,l0] ...
    #52[r1,l0] Name: 'radius'     #49[r1,l0] Integer 3
    #48[r1,l0] Name: 'constructor'  #12[r4,l0] ...
  }
  #53[r1,l0] Name: 'b'  #56[r1,l0] Object {
    #57[r1,l0] Name: '__proto__'  #26[r4,l0] ...
    #63[r1,l0] Name: 'radius'  #60[r1,l0] Integer 4
    #59[r1,l0] Name: 'constructor'  #12[r4,l0] ...
  }
  #64[r1,l0] Name: 'c'  #67[r1,l0] Object {
    #68[r1,l0] Name: '__proto__'  #26[r4,l0] ...
    #74[r1,l0] Name: 'radius'  #71[r1,l0] Integer 1
    #70[r1,l0] Name: 'constructor'  #12[r4,l0] ...
  }
  #81[r1,l0] Name: 'Math'  #80[r1,l0] Function {
  }
}

process.memory()

=
{"free" :5394,
 "usage":56,
 "total":5450,
 "history":40,
 "stackEndAddress" :536983744,
 "flash_start"     :134217728,
 "flash_binary_end":134404416,
 "flash_code_start":135135232,
 "flash_length"    :  1048576}

[espruino-discuss3]

Posted at 2014-02-06 by user6350

Possibly ESPRUINO software internal architecture is better than what can be expected after a first look.
I am now beginning to understand that having a local storage (SD card ) and a "huge SDRAM " amount for variables and adding to that the possibility to add or remove cached modules gives a program the ability to work as in the old fashioned overlapped memory concept.
Not the same as a virtual memory but a really effective technique on embedded system.

Assuming those SDRAM memory cost figures:

expr 20 * 5450 = 109000 100KB/192KB SDRAM 92KB free STM32F4x7, x=0,1,2
expr 20 * 8200 = 164000 164KB/256KB SDRAM 92KB free STM32F4x9, x=2,3

a lot of possibility arise for the Javascript interpreter, whose memory allocation penalty is bigger for simple variables such a int, float but not for objects or functions.
The STm32F429 can use an external 133mhz SDR SDRAM so memory cost of allocating variables will be minimized at all.

Has the capability of adding and removing cache modules been designed for such purposes or is there something more behind the scenes ?

[gfwilliams]

Posted at 2014-02-07 by @gfwilliams

I'm not sure I understand. The Module class exists to handle node-style require commands (and to allow the Web IDE to dynamically load modules), and it is not a way to page in/page out data unless your program is specifically designed for it.

Of course you could always just execute files directly with: eval(require('fs').readFile('foo.js')

Espruino was also designed with the possibility of automatically storing less-used data in slower memory - for instance the SD card. That would make it massively flexible - however it is yet to be implemented as for the uses that Espruino was designed for, 48kB of RAM actually goes a long way.