10k limit on esp8266

[espruino-discuss]

Posted at 2018-01-31 by kameas

hi, i use espruino_1v95, and if u upload file > 10kb - i have error:

>Uncaught SyntaxError: Got UNFINISHED STRING expected EOF
 at line 1 col 15
E.setBootCode("Modules.removeAllCached();Modules.addCached(\...
              ^
New interpreter error: LOW_MEMORY,MEMORY
=undefined

if i send file <10kb - it's ok, maybe I'm doing something wrong?

process.memory() print:

>process.memory()
={ "free": 1660, "usage": 40, "total": 1700, "history": 773,
  "gc": 3, "gctime": 1.542 }

and process.memory(), with file almost 10 kb in size

>process.memory()
={ "free": 1155, "usage": 545, "total": 1700, "history": 0,
  "gc": 23, "gctime": 1.66 }

[espruino-discuss]

Posted at 2018-01-31 by kameas

Espruino write on start:

>
 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v95 Copyright 2017 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:512/512, manuf 0xe0 chip 0x4016

---

Posted at 2018-02-01 by @gfwilliams

Yes, that's what you'd expect.

Turn off Save on send and you may well be able to manage more.

edit: it's something that'll hopefully change soon, but right now you need to load your code into RAM in one go before writing it to flash.

---

Posted at 2018-02-01 by kameas

thanks Gordon, the file success send, but if i want save it - i have error:

>save()
=undefined
Erasing Flash.....
Writing............................
ERROR: Too big to save to flash (26151 vs 12284 bytes)
Deleting command history and trying again...
Erasing Flash.....
Writing......................
ERROR: Tooàþ
 ets Jan  8 2013,rst cause:4, boot mode:(3,7)
wdt reset
load 0x40100000, len 2408, room 16
tail 8
chksum 0xe5
load 0x3ffe8000, len 776, room 0
tail 8
chksum 0x84
load 0x3ffe8310, len 632, room 0
tail 8
chksum 0xd8
csum 0xd8
2nd boot version : 1.6
  SPI Speed      : 80MHz
  SPI Mode       : QIO
  SPI Flash Size & Map: 32Mbit(512KB+512KB)
jump to run user1 @ 1000
�
Disconnected

esp12 have 4mb flash memory, why it's say what i have 12284 bytes?

---

Posted at 2018-02-01 by kameas

thanks, I have one more question, did not notice the "Reply" button

---

Posted at 2018-02-01 by Wilberforce

The esp8266 has limited ram, and only 12k is left after the interpreter and operating system.

If you minimize the code before uploading, you may be able to get it to fit.

What is the code you are trying to load? It seems quite large.

---

Posted at 2018-02-01 by kameas

Code here, it's management of climate monitoring via telegram api, at this moment - work wine, but i want keep subscribers and notify them when necessary.

At this moment free space in esp for additional methods is end, and i dont have idea how to fix it.

Here MaBe proposes to use "eval" and keep code inside unused flash memory:

>require('Flash').getFree()
=[
  { "addr": 487424, "length": 4096 },
  { "addr": 524288, "length": 4096 },
  { "addr": 1011712, "length": 36864 },
  { "addr": 1048576, "length": 3129344 }
 ]

is it real for me?

---

Posted at 2018-02-01 by @MaBecker

Flash map 4MB:512/512, manuf 0xe0 chip 0x4016

You have a ESP8266 with a 4MB flash, but flashed firmware for a ESP 01 512K.

Check ESP8266 page, section Build Content

---

Posted at 2018-02-02 by kameas

ok, i apologize for the stupid questions and thank you for the explanation.

Спасибо, товарищи!

---

Posted at 2018-03-08 by CMSpooner

So is there a way to get around the 10k source code size limit?

---

Posted at 2018-03-08 by @gfwilliams

On 1v96 firmware and later, yes. You can use require("Storage").write to create a large file called ".bootcde", and can then write the data in chunks (see the example).

Ideally it'd be something built into the IDE but I haven't got around to that yet.

---

Posted at 2018-03-08 by CMSpooner

Excellent! Is there anyway you could link to the example? I did a little searching but didn't see how to write executable code using the .write() command

---

Posted at 2018-03-08 by @gfwilliams

http://www.espruino.com/Reference#l_Storage_write - it's linked from require("Storage").write but is hard to see

---

Posted at 2018-03-08 by @gfwilliams

Just to add - so what you want to do is make a bit of code that takes your JS file and splits it up into ~1kB chunks, then creates a bunch of .write calls one after the other with the correct offsets. The very first one needs to have the file's length in it.

---

Posted at 2018-03-08 by CMSpooner

I saw that...I was hoping for an example of saving an entire javascript program into storage and then loading it and executing it. Would I create a helper program to save the .js file as a really long string then onInit() read it back out and then somehow execute the string as Javascript?

Edit: I think I'm on the same page as you now....I will try playing with that tonight.

---

Posted at 2018-03-08 by @gfwilliams

Ok, I've just changed the Web IDE so Save on Send now does it properly if it detects 1v96. If you use the latest from GitHub: https://github.com/espruino/EspruinoWebIDE

Or use the command-line tools with npm install espruino then you should get something that works.

---

Posted at 2018-03-08 by CMSpooner

Thanks! That seems to have done the trick!...I'm not sure how much more stuff I can stuff into it; but I should be able to finish a nice demo program that shows the capabilities of this board.

---

Posted at 2018-03-08 by @gfwilliams

Which board are you using? If it's the ESP8266_4MB it looks like you can put about 200kB's worth of code in there!

---

Posted at 2018-03-08 by CMSpooner

I'm using this ESP8266_4MB board with battery and oled screen . I am able to install my codebase with basic minification on, but get Low memory messages. I guess I will have to optimise a bit.

I plan to give these out at my schools summer STEM camp. Bought in bulk we can get it to around $10 a student with a battery and 3d print the case. I am writing a comprehensive demo to show them what they can do. It also makes sure I understand the platform; so I can answer questions. I just picked up JavaScript recently (Fitbit Ionic Development), so I am still getting comfortable with the language. I'll attach my demo so far.

---

Attachments:

• MainCodeMenu.js

---

Posted at 2018-03-08 by @gfwilliams

Wow, I'm surprised that kills your memory.

One thing I'd say is if it doesn't have to be super fast, make a function getBootImg that returns the structure. That way the image will stay in flash, but will only be loaded into RAM just in time to be displayed and will then be unloaded again.

---

Posted at 2018-03-08 by CMSpooner

I'll try that.

Here is the response from my device on load:

 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 1v96.5 (c) 2018 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:1024/1024, manuf 0xc8 chip 0x4016
>
=undefined
Loading 14495 bytes from flash...
Execution Interrupted during event processing.
New interpreter error: LOW_MEMORY,MEMORY

On Boot:

>process.memory()
={ "free": 603, "usage": 997, "total": 1600, "history": 1,
  "gc": 485, "gctime": 2.459 }

On connect to wifi:

>process.memory()
={ "free": 624, "usage": 976, "total": 1600, "history": 4,
  "gc": 73, "gctime": 2.527 }
>process.memory()

On start of ball bounce demo:

={ "free": 611, "usage": 989, "total": 1600, "history": 4,
  "gc": 146, "gctime": 2.581 }
> 

---

Posted at 2018-03-08 by Wilberforce

The other option you have is to have a bootstrap script that you run first, that saves the large btoa image as an array into with Storage.write. This can be run once to save the data, and then the main script just pulls a reference to this in flash and won’t consume any valuable ram.

---

Posted at 2018-03-08 by Wilberforce

Another option is to build a custom firmware that compiles in the SSD1306 and graphical_menu as js sources, then these modules will be in flash instead of ram...

---

Posted at 2018-03-08 by CMSpooner

Interesting...Saving the image to flash is definitely something that would be easy. I'm not sure if I want to build a custom image; since I would then have to keep parity with the official branch after I distribute them to the kids.

---

Posted at 2018-03-09 by @gfwilliams

Actually what happens if you turn on Modules uploaded as functions in Communications?

@wilberforce do you see any reason I shouldn't just remove the option and turn it on automatically for boards with firmware 1v90 and later? That'd save a whole bunch of memory.

---

Posted at 2018-03-09 by CMSpooner

That definitely helped with ram usage:

>process.memory()
={ "free": 833, "usage": 767, "total": 1600, "history": 7,
  "gc": 571, "gctime": 2.573 }

I was able to load the unminimized code (it previously failed to load where different levels of minimization would load; so i've been using it to test. I still get a message:

 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 1v96.5 (c) 2018 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:1024/1024, manuf 0xc8 chip 0x4016
>
=undefined
Loading 14495 bytes from flash...
Execution Interrupted during event processing.
New interpreter error: LOW_MEMORY,MEMORY

But the code is loaded and stored.

---

Posted at 2018-03-09 by CMSpooner

So I was starting to have some problems uploading code. Code looked like it went over; but it wasn't changing. I ran the following code:

>s = require("Storage")
=function () { [native code] }
>s.eraseAll()
=undefined

And then re-loaded my code over usb and no more memory errors. I am wondering if it is not cleaning itself out properly between loads with the newest version of the IDE (pulled from Git today)

---

Posted at 2018-03-10 by @allObjects

In case you still struggle with MainCodeMenu.js code, no surprise to me: upload invokes at the end setup(). This means that before the upload can complete, things get executed... setup() is also called in E.on("init",...);.

Even though I cannot validate the following change, give it a try: put setup() at the end of the code into a setTimeout(setup,1000);.

What I generally suggest and do is just using function onInit() { setup(); } at the end of the code. While I'm still developing it is cumbersome to every time enter setup() in the console after code upload, I add an extra line containing setTimeout(onInit,1000); and remove it after I'm done with developing. I then upload the code with this last line removed and save it. That way I have a plain vanilla code on the chip and on power on it will do what I'm expecting: run the complete setup();. Level 0 code gets right away interpreted/executed while in upload... see the (often) referenced conversation about simple explanation how to save code that espruino run on start?.

---

Posted at 2018-03-10 by @gfwilliams

@allObjects when you upload code via the 'Save on send' method it's all executed after upload, so I don't think that's such a big deal.

Uploading with 'save on send', function code can be kept in flash, but variables can't. So for anything big it's best to write a function that just returns the data - then it'll be loaded into RAM only when needed. (or you could use Storage).

@cmspooner the issue you hit is probably because you were also experimenting with using Storage as well, or perhaps the 2 different types of 'Save on Send'? Espruino can get in a position where there are multiple bits of information in storage, but there's not enough RAM available to read all the stored data in and compact it - so it can't do anything without potentially losing things. In that case you'd have to manually erase like you did.

---

Posted at 2018-03-10 by CMSpooner

That makes a lot of sense! I pulled from github and wanted to make sure I had the most recent version of the IDE and couldn't remember which save on send I had. Thanks for the great support. I can't wait to have the kids at STEM camp playing with these!

---

Posted at 2018-03-10 by @allObjects

@gfwilliams, thanks for the update.

---

Posted at 2018-03-10 by Wilberforce

Rather than remove - what about changing the default to be on if greater the v90, so that people still have the option of turning off?

Perhaps it should be relabeled or a sub note saying that this means modules go to flash?

---

Posted at 2018-03-12 by @gfwilliams

Yes, could be an idea - however I can't think of any reasons you'd ever want to turn it off?

But yes, more info would be helpful. The modules only actually go to flash if you've enabled the 'Save on Send' as well.