Memory not released after dofile.

[saitejal]

Hello. I am new to lua and NodeMCU. I dont know if you can classify this as a issue, but there they are.
I am trying to extract data from an xml file.
Here is my xml file:

<?xml version="1.0" encoding="UTF-8"?>
<netconfig>
<mode>0</mode>
<stamac>18-FE-34-A4-4B-05</stamac>
<staip>XXX.XXX.XXX.XXX</staip>
<stanetmask>XXX.XXX.XXX.XXX</stanetmask>
<stagateway>XXX.XXX.XXX.XXX</stagateway>
<apmac>1A-FE-34-A4-4B-05</apmac>
<apip>192.168.4.1</apip>
<apnetmask>255.255.255.0</apnetmask>
<apgateway>192.168.4.1</apgateway>
<port>80</port>
<dns>XXX.XXX.XXX.XXX</dns>
<dhcp>1</dhcp>
<stacustomconfig></stacustomconfig>
<timezone>10</timezone>
<serial>0x00000001</serial>
<connssid>ESP-10767109</connssid>
<ssid></ssid>
<passwd></passwd>
<hostname>ESP-10767109</hostname>
<reboot></reboot>
<message></message>
</netconfig>

Here is my xmlparser:

return function (xmlfile, xmlword)
    file.open(xmlfile,"r")
    local eofflag = 0
    local i, j, k, l, xmloutput
    while(eofflag < 1) do
        local m = file.readline()
        if(m == nil) then
            eofflag = eofflag + 1
        elseif (string.find(m, xmlword) ~= nil) then
            i, j = string.find(m, xmlword, 1)
            i = i - 1
            j = j + 2
            k, l = string.find(m, xmlword, j)
            k = k - 3
            l = l + 1
            xmloutput = string.sub(m, j, k)
            eoffile = 1
        end
    end
    file.close()
    return xmloutput
end

I call this file using:
local port = dofile("xmlparser.lc")("netconfig.xml", "port")

And this same thing happens over there too, where I try to create a xml file for the scanned WiFi networks.

local t = {}
setmetatable(t, { __mode = 'kv' })
local function listap(t)
    local temp, ssid, v, imgnumber = 0
    file.remove("aplist.xml")
    file.open("aplist.xml","w+")
    local w = file.writeline
    w([[<?xml version="1.0" encoding="UTF-8"?>]])
    w("<aplist>")
    for ssid, v in pairs(t) do
        print("inside for loop")
        print("heap is", node.heap())
        local authmode, rssi, bssid, channel = string.match(v, "(%d),(-?%d+),(%x%x:%x%x:%x%x:%x%x:%x%x:%x%x),(%d+)")
        print(ssid,authmode,rssi,bssid,channel)
        temp = tonumber(rssi)
        if temp < 0 and temp > -50 then
            imgnumber = 4
        elseif temp < -50 and temp > -80 then
            imgnumber = 3
        elseif temp < -80 and temp > -90 then
            imgnumber = 2
        else
            imgnumber = 1
        end
        w("<network>")
        w("<ssid>"..ssid.."</ssid>")
        w("<auth>"..authmode.."</auth>")
        w("<str>"..imgnumber.."</str>")
        w("<bssid>"..bssid.."</bssid>")
        w("<chn>"..channel.."</chn>")
        w("</network>")
    end
    w("</aplist>")
    file.close()
end
wifi.sta.getap(listap)

I call this file using:
dofile("apscan.lc")

I always endup with memory less than I started with (sometimes as high as 5kb) even though I call the garbagecollector. Moreover The heap seems to decrease further if the word I am searching for is near the end of file. I also observed that if the word I am looking for is not present, the heap has the less difference I started with.

[MarsTechHAN]

That's one of the feature of Lua...
The function will as virable still stronge inside RAM
You can use
'''
package.loaded['filename']=nil
'''
to unload

[TerryE]

@MarsTechHAN package.loaded['filename']=nil only works if you've loaded the module via require()
@saitejalakkimsetty, you'd be better off hoisting up the parser one level and processing the (xmlfile, xmlword) as .... so you could:

local port = loadfile("xmlparser.lc")("netconfig.xml", "port")

There are all sorts of mechanisms for leakage. You need to supply an independently repeatable test case.

[saitejal]

@TerryE This is an example written in our wiki page:

   local function listap(t)
      for k,v in pairs(t) do
        print(k.." : "..v)
      end
    end
    wifi.sta.getap(listap)

The output was:

> =node.heap()
17288
> dofile('temp1.lua')
> abc : 4,-71,78:da:6e:e6:04:7a,11
def : 0,-92,00:18:0a:51:1f:64,1
ghi : 4,-44,98:fc:11:54:5e:e7,1
jkl : 4,-56,68:7f:74:09:1a:8c,1
mno : 4,-92,06:18:0a:51:1f:64,1
pqr : 4,-70,7a:da:6e:e6:04:7b,11
=node.heap()
16384
> collectgarbage()
> =node.heap()
16384
>

This itself leaks the memory!
I am on our latest dev prebuild.

[TerryE]

@saitejalakkimsetty, I am not sure that this is a bug. There was a known issue with the wifi.sta.getap() routine not correctly cleaning up its Lua registry entry for the callback routine, and that you could free this up by calling it a second time with any firmware based routine which accepts a table parameter e.g. wifi.sta.getap(type) However, what you are seeing is an artefact of Garbage Collection rather a bug. If you did

for i = 1, 20 do dofile('temp1.lua') collectgarbage() print (node.heap()) end

and this showed a steady loss in memory, then this would be a true leak.

Have a look at my Unofficial nodeMCU FAQ on the esp8266.com site for more discussion.

[saitejal]

@TerryE Thanks Terry. I removed the code inside the listap function into a separate file and called it. I was able to save all the memory except 400 bytes.

[MarsTechHAN]

@TerryE Thanks for your correction, I'm not a qualified Lua programer😁

TODO: @MarsTechHAN
Fix the issue wifi.sta.getap() 's mem leak.

[saitejal]

Temporary fiix has arrived, but still consumes around 200 bytes. The fix is similar to moving the code from callback function into a dofile, and then calling the dofile through the function. They both consume exactly same amount of ram.