esp8266 memory problems #5240
Replies: 1 comment
-
Posted at 2015-11-10 by hygy Here is the modified ds18b20 code. It uses a callback when the temperature is ready.
Posted at 2015-11-11 by tve As far as I can tell, the out of memory error comes from JsVars: Posted at 2015-11-11 by Ollie @HyGy I've just confirmed on gitter that using IP address in place of hostname allows me to send data to Sparkfun using https://data.sparkfun.com/streams/6JxNaX7o06S0RxJgvRdD Will bet that is your issue? Maybe try divorce your code from the temperature polling as that would appear to be a separate issue - maybe the probe you have? (I was able read temp using the DS18B20 module and a DS18B20 probe on ESP8266 just fine - also posted about that on gitter) Posted at 2015-11-11 by @gfwilliams @HyGy so you're saying it runs fine, but crashes occasionally? Or it never completes successfully? If it never completes successfully then it's just likely that what you've written uses up too much memory:
If it is crashing only after some hours, try running The disconnect is worrying too. If you're using something like the NodeMCU board, the USB is handled by a separate chip - about the only thing that could cause the connection to drop if if there was some problem with the power supply. @tve I'm afraid there isn't - the usual stack trace stuff runs by storing the stack trace in variables, so it wouldn't work as there's no memory! Usually these things aren't caused by the actual function that fails with out of memory though - it'd be something else eating it up slowly and not releasing it. As I feel I have to say in these circumstances: if you used an 'official' Espruino board this would work without problems - because I can afford to properly support and test those boards Posted at 2015-11-11 by @gfwilliams @ollie thanks - sorry, I missed your post while typing the response here :)
That sounds like a memory leak in the ESP8266 code then. Probably the callback that's being locked but not unlocked in the C code. Posted at 2015-11-11 by Kolban Yup ... we need to do better in handling gethostbyname functions on the ESP8266 ... the following Issue will be used to track: #590 Posted at 2015-11-11 by hygy @gfwilliams it is running some time, whitout problem, then it dies. When this happens, i reset the board, the problem will be the same exactyl, then I try to plug and unplug the board from the usb port but it is wrong. So I send the saved code to esp8266 again with the espruino ide, then save() it, after that it is working. I'll try tu put the given code lines to have more debug messages. But I think there is stg wrong with the http get module, maybe the esp8266 implementation is buggy somewhere. Posted at 2015-11-11 by hygy Wooo I write this in the console: what is that: ----> "ÿ"
Posted at 2015-11-11 by Kolban I believe that Espruino has a concept called "hidden variables" and these are variables that are rooted from a JavaScript object called "\0xFF" ... i.e. an object name composed of a single character that is unprintable and has the ordinal value of 255 decimal. Posted at 2015-11-11 by @gfwilliams Yes, it's a bit hacky at the moment, but Posted at 2015-11-11 by hygy
Posted at 2015-11-11 by @gfwilliams But... do any of them get bigger after you have done an HTTP request or two? Posted at 2015-11-11 by hygy I put this line console.log(process.memory()); in the http response.on('data', function(data) { And now it is crashing immediately, when get into this part. Posted at 2015-11-12 by @gfwilliams Crashing? Or getting Something sounds pretty broken there then. @Kolban I see you had your windows-based debug build that you put in the repositories. Did you ever finish that? If you could reproduce the issue on there (you might want to make sure build_platform_config builds the image with 1023 vars too) then debugging would be a lot easier. Posted at 2015-11-12 by hygy Okay, The mean time I connected a second usbttl to UART1: @Kolban is it help for you?
And here is what I got from the espruino console:
Posted at 2015-11-12 by @gfwilliams Hmm, the error happens when you call ... but it's possible this could be due to the new memory saving changes I have made? Not sure... it's be worth trying to run some non-network-related code on there and see if everything works ok. Posted at 2015-11-12 by Erni If it is of any help, here is the error I get, after running simple http get to thingspeak in approx. 2 hours, whitout ds18b20 http://ernstc.dk/arduino/esp_error.html Posted at 2015-11-12 by Kolban @Erni, ... later ... Posted at 2015-11-12 by Kolban I have found at least two major problems and a mystery :-) First problem is related to issue #595 which states that there is no network error handling at the ESP8266 level. In English this means that if the ESP8266 detects a network error, it silently ignores this. When working with Thing Speak, we need to assume that Thing Speak may ignore our HTTP GET request ... when that happens, the lack of the ability to form a connection is not handled by the ESP8266 and it keeps the socket open. This means that we start to leak sockets and quickly die. The solution is to detect the failure to form the connection and shutdown the socket. The second problem is related to the logic of the sample application. It's logic says "Every period, make a GET request to thing speak" ... however that logic doesn't say "At the end of the PREVIOUS GET request, schedule another request to Thingspeak". What this means is that if a request to connect to Thingspeak takes a while, more and more parallel requests could be made until we run out of resources. The solution is to change the logic to use setTimeout at the completion of the last GET request instead of setInterval which keeps sending new GET requests irrespective of whether previous ones completed. The last note is still a mystery. When we use the Espruino "http.get()" method ... that is an asynchronous call ... it takes a callback function to be invoked when the response from the get request is available ... however ... we have a problem. In ESP8266, all network calls are asynchronous and this includes the formation of the connection request in the first place. What should happen when the "http.get()" request connection fails ... because it will be an asynchronous connection and happen later in the story ... this is going to be tricky. Again ... I believe that http.get() assumes a synchronous call while ESP8266 network connections are asynchronous. We may be plain stuck here for a while. Posted at 2015-11-12 by @gfwilliams It is asyncronous... ... however there is no error handling at the moment. In node.js you'd do:
and we don't do anything like that yet. The code will upload, but the error handler will never get called. ... to be honest I'm not sure it's that high priority, as you can work around it with:
Basically if something stalled, the socket would have timed out after 60 seconds anyway and closed. Posted at 2015-11-12 by @gfwilliams wrt network errors: What I do for the other stuff is if there's an error, I clean up everything I can and then just return Posted at 2015-11-12 by hygy I commented out all the onewire and ds18b20 codes and libs from the code when the test run, as @Kolban asked. :) Posted at 2015-11-12 by Kolban If I'm understanding properly ... the callback handler on an http.get() will not be invoked if the original connection request times out. I think I am also understanding that there is no on("error") handler in the current implementation for the object returned by http.get(). If these concepts are true, then we don't have an "elegant" means of detecting that a network request through http.get() has timed out and hence have no way of "handling" such a situation. The best notion we have right now is firing a timer to go off some time in the future when we could sensibly assume that the connection should have been made ... and in that timer function, check see if the connection did indeed happen (or assume that since the timer fired ... we weren't cancelled) ... and handle there. I suggest that this is a great opportunity to form a group of us to determine if this is the desired final semantics we want ... and, if not ... figure out what the correct semantics should be, start a design and then make progress towards implementation. My opinion is that I really don't like the "timer" solution as a long term solution and believe that I would like the on("error") handler on the "get()" request. Posted at 2015-11-12 by @gfwilliams Isn't that just what I said in my last post? Maybe we were both writing it at the same time :) Given Espruino's networking has been in use for some time now, and nobody has made a great fuss about this before, I'd be tempted to say that The issue @HyGy would seem to have is that the ESP8266 code isn't reporting back errors (which it can easily do by returning Once ESP8266 does that, @HyGy's problem will go away - and we can look at Posted at 2015-11-12 by tve
I do not believe that your statement is correct. If the connection fails you will get a "recon_cb". Posted at 2015-11-12 by Kolban @gfwilliams ... In post #24 ... that was a written recap of what I understood in post #21. I am a big one for verbose but less ambiguous wording when it comes to technology discussions. So yes ... #24 was what you said in #21 ... The absence of anyone having made a fuss in the past is not an indication that we have quality or correctness now. I'm a big fan of shooting for the best we can possibly do. You are the architect of all ... if you don't feel that there is value in this area then it is up to folks who think otherwise to civilly make their cases for change to ensure that you have all the information you need to make the decisions for or against a change. My data point is that on("error"), is desirous ... it isn't vital (yet) ... but would have relative high priority in the networking work items. I changed the error handler in the ESP8266 port to now detect connection failures and in subsequent sends() requests does now indeed return -1 ... However ... now I find that causes a problem because I now get:
On my console ... but worse ... my app ends because of the exception because I am no-longer able to detect when I should trigger a setTimeout() for the next round ... I would have to have an on("error") handler (or the solution you suggested using the timer .... which I respectfully ... simply don't like). Posted at 2015-11-12 by Kolban @tve Since then, code has been added to the espconn_reconn_cb() via issue #595. So ... YES ... handling a lack of connection CAN be caught in the espconn_reconn_cb() and YES ... at the time of writing ... handling a lack of connection was NOT being handled (but is now). Posted at 2015-11-13 by @gfwilliams There is a big difference between 'not liking something' and memory leaks, crashes, unexpected resets, etc :) I've made an issue for it here and it will get sorted at some point in the future. I just don't know when :) Posted at 2015-11-13 by @gfwilliams Just to add, continuously rescheduling the same task using a IMO something like this is a lot more straightforward.
Although as you noted earlier it only really works if the period is greater than the socket timeout. Also, the socket error printed to the console will almost certainly have come from idle processing, and won't affect the execution of any code. It just means your handler won't get called. Posted at 2015-11-13 by Moray In my ideal world we would move to promises rather than callbacks wherever possible and especially in network libraries. I expect that's a huge task though. I am planning though to rewrite for my own purposes the ESP8266 module with promises, probably using promiscuous library although it would be good to hear any opinions on good, small promise libraries. I haven't yet tested any with Espruino. Posted at 2015-11-14 by Kolban @gfwilliams I was thinking about the timer workaround ... imagine that a http.get() request is made to a back-end and fails (silently) because there is no callback to say that the network connect request itself failed ... One suggestion was to have a cancel able JS level timer that if not cancelled, would indicate that we had failed to connect ... Imagine that the timer was in fact called ... in our timer callback code we now know we are here because a connect request has failed ... what should we then do? What remediation is possible at the JavaScript level to clean up the internals of todays Espruino? Experience seems to show that I am looping in calls to transmit (to transmit the request). Posted at 2015-11-15 by hygy I'm not sure, how is this workaround working. The setTimeout runs timeout, but the http.get need to be wayted, while it calls the callback. But if we are there, then something is happend, whay it is called the callback, and if the callback called after 10 minutes, it will wait 10 minutes. Posted at 2015-11-15 by hygy I put this workaround into the code, and its the same...
Posted at 2015-11-16 by @gfwilliams There is no need to free memory or 'clean up the internals of Espruino', because if the socket times out then it's a socket error, which should be reported by sending It only doesn't do that right now because the ESP8266 port is broken. When it is fixed the problem will go away. Look at it this way: Right now, if Espruino gets an error reported on the socket it frees everything automatically. Sure, it doesn't report the error to your code, but everything gets freed. It's broken right now because it's not getting told there's a socket error. So suppose I spend time and add this error reporting code you want? It won't work on ESP8266 right now because ESP8266 isn't reporting the error back to Espruino. There's absolutely no point me doing it until the ESP8266 port is working as it should... but then when it's working nobody will have this problem any more and the urgency will go away. Posted at 2015-11-17 by hygy @Kolban What do you think when can U correct this problem? @gfwilliams if it is help, to handle the error I think there is a good idea (and it will be usable later too), so we can handle or ignore depends of the application. Thanks HyGy Posted at 2015-11-17 by Kolban @HyGy As part of Issue #595, we added a detection of a failure to form a connection and associated cleanup. What that means is that when an HTTP request to a partner fails, ESP8266 will now detect that occurrence AND release the Espruino/ESP8266 resources. Any build north of 2015-11-14 should contain the fix. Posted at 2015-11-17 by @gfwilliams Great, thanks @Kolban! With the HTTPS stuff (see the other thread) I'll hopefully get the async Do you currently handle name resolution as part of the connection? If not, I'd hold off doing that. Hopefully the async changes will make that a lot easier too. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2015-11-10 by hygy
Hi,
I have some memory problems with a js code.
I connect to wifi, resolve an ip addr, connect to onewire, get the ds18s20 sensor data, and every minutes it sends the data to thingspeak.
For a while (what is sometimes 5-10 horus, sometimes less), I got this (it disconnect from espruino ide, and I need to reconnect):
Here is the code:
The smallest value what I get, is the response in the sendTs function in respons.on data part:
{ "free": 17, "usage": 1006, "total": 1023, "history": 1 }
1331
I used a changed ds18b20 library to support ds18s20. I send it in the next post.
thanks,
HyGy
Beta Was this translation helpful? Give feedback.
All reactions