Memory Optimization

According to this site, in our NodeMCU is "~80kB DRAM (Data RAM), and ~35kB of high speed IRAM (Instruction RAM)". Processor in ESP8266 uses a Harvard architecture. "The IRAM is loaded at boot with whatever the user wants to keep on-processor, though the processor can run code directly off of the external flash at a lower speed".

"The ESP8266 use onchip RAM and offchip Flash memory connected using a dedicated SPI interface. Both of these are very limited (when compared to systems than most application programmer use). The SDK and the Lua firmware already use the majority of this resource (the later build versions keep adding useful functionality, and unfortunately at an increased RAM and Flash cost)." [source]

Size of RAM seems to be even quite good. However, it's very very little while using LUA language. Therefore we must save RAM on each step. Each loaded module take a lot of RAM. How to manage with it? There are multiple ways to ommit this obstacle.

1. How memory is managed in NodeMCU (optional)

• If you want to learn how Lua events and callbacks work - read this topic in documentation, it's quite good explained how it's working "under the hood". However, it may be quite complicated and too long...

• In a nutshell...

  • Under the LUA API (provided by NodeMCU) there is another SDK API (provided by ESP8266).

  • SDK API provides own set of functions (for declaring functions written i C) called "Callbacks" - just to associate tasks with specific hardware and timer events to queue pending events.

  • The registered callback routines are invoked sequentially with the associated C task running.

  • Example: tmr.alarm(id, interval, repeat, callback). This calls a function in the tmr library which registers a C function for this alarm using the SDK, and when this C function is called then it invokes the Lua callback.

  • Some libraries (net, tmr, wifi, ...) use the SDK callback mechanism to bind Lua processing to individual events (e.g. timer alarm).

  • Non-Lua processing (e.g. network functions) will only take place once the current Lua has completed execution. So any network calls should be viewed as an asynchronous requests.

    • Uncorrect approach is e.g. if two socket:send() are alongside in code and the first has completed by the time the second is executed.

    • Correct approach: each socket:send() request simply queues the send operation for dispatch. Second request won't start to process until the Lua code returns from first C-function called by first query. Difficult? Nope - everything is done inside and we don't have to know how it is done. It's behaves as asynchronous Active Object design-pattern. And under the hood it's not quite simple...

    • Stacking up requests in a single Lua task function uses a lot of RAM and can trigger a PANIC. It is even while requesting restart (node.restart(); for i = 1, 20 do print("not quite yet -- ",i); end).

  • Callbacks (written in Lua) - are stored in the Lua registry.

    "Note that we have identified a number of cases where library code does not correctly clean up Registry content when closing out an action, leading to memory leaks."

  • If registered Lua callback routines are invoked, they run to completion uninterrupted.

  • Therefore long-running functions can cause other system-functions and services to timeout and can cause the system to reboot.

• See an example in this section.

2. How it connects with our code

• It is important that any event callback task is associated with a single Lua function. Even dofile("...") can be treated as a special case of it.

  • Let's look at variables - any Global will persist until we reassign it to nil. This is how we can recover some RAM! As some libraries don't correctly do dereference, it's good to use Globals and nil manually.

  • On the other side, there are Upvalues, which are built-in LUA language (just closure feature). They are taking a lot of RAM, because of technical background connected with references and garbage collector... Upvalues are variables created in a functions and available in its' inner scopes (in inner functions). Therefore there is a huge overhead of RAM while creating applications.

• Example how to avoid running out of RAM in our code.

• Avoid tmr.delay() ! - Great explanation "When and why should i avoid using tmrdelay".

• How to avoid a PANIC loop in "init.lua".

3. How to optimize RAM and Flash memory

Everything about it in the Documentation.

1. Flash the newest firmware

2. Build and use firmware with only modules you really need

3. Reduce size of compiled code

   Instead of dofile("file.lua") use node.compile("file.lua) and then require("file") which will firstly run compiled version of file ("file.lc").

4. Reduce flash footprint

   Use LuaDrcDiet** - to minimize file size.

5. Don't use debug information

6. Use alternative memory optimization

   You can use "Massive memory optimization". However, it causes that our program will operate not only in RAM (loaded once from flash) but also it will use a flash memory a lot. Therefore, lifetime of our flash memory will be quickly reduced. It's the mechanism of last resort.

7. Use Globals

   ...and reassign them to nil after using them.

8. Use volatile modules

   Modules also can be reassigned to nil, but inside of them. Read how to do it - at the bottom of this paragraph.

###Read more Tip 1 | Tip 2 | Tip 3 | Tip 4 | Tip 5

last updated: 2017-01-23