Improved memory usage in Espruino #63
Replies: 30 comments
-
Posted at 2014-06-26 by BogdanG Hi Gordon, Thanks |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-17 by @gfwilliams There's another beta of this up (based on the latest version of Espruino) at: http://www.espruino.com/binaries/git/commits/2fc9d09ff088bba2fd3d31f1b64b659be89283c1 Please can some of you give it a go with your projects and report back if there are any problems? I'd really like to get this merged into the version that everyone has - it means that a lot more code and data can be squeezed into Espruino. It's the same to flash as any of the git builds - click the link, then right-click on |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-22 by @gfwilliams Nobody? :( New build up here: http://www.espruino.com/binaries/git/commits/ba9ce23084b74a18ccba91a0a88c3cf0c62821a9/ This one not only uses variable blocks more efficiently, I've also been able to get the block size from 20 bytes down to 16. That means that on the Espruino board, where you could have got 1800 variables you can now get 2250 (I haven't updated the specs for each board yet though). |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-22 by @gfwilliams This branch also manages to get the block size for low-end boards down to 12 bytes, while still allowing up to 1023 blocks. It means that the VLDiscovery board now stores 500 blocks instead of 250. Not only that but as arrays are packed, you can actually get 4x as many array elements in the boards memory as you could have before! |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-23 by DrAzzy Woah. I just read the bit about the block size going down to 16 bytes. I'll have to give this a try . The weather here has been great every weekend, so I've had a lot less time than usual to play with electronics. That means we could get 3250 jsvars with the bigram builds... |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-24 by DrAzzy Eurgh. What git command would I use to get this branch so I can build it with more jsvars? I'd like to test it with my desk-lamp, but that needs >2k (even after I cut out the 8x12 font, it runs out of memory) jsvars to run. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-24 by @gfwilliams You should be able to do Doing it that way won't let you commit any changes, but otherwise it'll be fine. 3250 vars would be pretty good :) The only bad point is that there's still the same fixed overhead of 4 bytes for long strings, so for big chunks of data it's only 12/16=75% efficient rather than 16/20=80%. The extra vars make up for it though :) I guess even the value packing should help you out too - I'd be interested to see how many variables it uses in the new build relative to the old. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-24 by DrAzzy git pull? git pull what? I'm sorry for being slow on this - the only git command I know is 'git clone https://github.com/espruino/espruino' |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-24 by @gfwilliams Ahh. It's probably worth trying to find a quick git howto - but |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-24 by DrAzzy Fixed 3250 jsvar build: http://drazzy.com/espruino/espruino_1v68_espruino_1r3_smalljsvar_bigram_wiznet.bin In order to make the new FW work at all, I had to pick through the flash memory with the STM flash module erasing debris from previous save()'s with bigram builds. I was never able to get the "magic touch" to work (where you hit btn1 at just the right moment after rst and it starts the Espruino ignoring the saved program). |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-24 by DrAzzy This function breaks with the new build (either using your 1800jsvar build, or my 3250 jsvar build):
This worked in v65 bigram, and the error message makes no sense to me. My BMP180 is returning bogus values - It says the pressure is -28347 Pa? (BMP180 module code that might be relevant is reproduced here for convenience):
Numbers going into that: "ac1":7972,"ac2":-1182,"ac3":-14539,"ac4":33201,"ac5":25505,"ac6":18922,"b1":6515,"b2":47,"mb":-32768,"mc":-11786,"md":2512
As an aside, the loss in efficiency of storing strings - does that apply to function code too? I say this because code is what takes up most of the space (that's been my experience, and iirc, Sascha indicated that was the |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-25 by @gfwilliams Thanks... With the bmp180, does it also fail on 'normal' builds? If so, that is almost certainly 32 bit as you say. For the first error, does that fail on 'normal' 1v67 too? It seems like it might - my guess is that you're parsing invalid JSON and that it now throws an exception. Perhaps it shouldn't? I'll have to look at the docs... |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-25 by @gfwilliams The loss in efficiency does affect strings as well, but I think that given the rise in available variables it probably has a net benefit. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-25 by @gfwilliams About JSON.parse: Looks like it actually is spec compliant now :) https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-25 by DrAzzy I'll start doing those tests tonight or tomorrow. Seeing that 3250 jsvars is such a beautiful sight; so far my code uses around 1800 just sitting there (with a few hundred used when it's doing something) The json the page is serving is valid... I'll need to throw in some console.log() to see if it's somehow not getting the whole thing?
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-25 by @gfwilliams Hmm - yes, I guess it might be getting corrupt data :( Could be some issue with HTTP on the new build I guess, but if you could dump the data you get it might give us some hints |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-27 by DrAzzy Okay, the BMP gives bogus results on the normal firmware too - Looks like it must have been the 32bit int change. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-27 by DrAzzy Neither of these issues turned out to be specific to the improved memory usage builds. Okay, the BMP gives bogus results on the normal firmware too - Looks like it must have been the 32bit int change. The http response is mondo mangled - same way every time, and it comes out correctly when viewed in a normal browser, and worked correctly in v65. So somewhere between v65 and v68, there's been a really nasty regression in http. (note - these are being tested with Wiznet)
Expected result:
Actual result - where did these newlines even come from?
|
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-28 by @gfwilliams eep. thanks - I'll get on this... It's probably because I'm now re-using the stream code that's used for Serial devices... |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-07-28 by @gfwilliams Ok - fixed in Git, or it'll be in 1v68 |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-04 by @gfwilliams I've just merged all this into the main branch, so it'll be in 1v70. Please check out the latest builds though - the sooner I'm happy that nothing has regressed, the sooner I can release 1v70 which has all these features in! |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-05 by JumJum Just tested latest build(http://www.espruino.com/binaries/git/commits/8e7aac987b82d5843face571789fdab1e2c6cc5b/espruino_1v70_espruino_1r3.bin) with this:
And got Uncaught Error: Field or method does not exist..... |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-05 by DrAzzy That bug is new. It does not reproduce under v69 (latest release) but it does under v70 builds (I reproduced it on http://drazzy.com/espruino/bigram/espruino_1v70_08-05_espruino_1r3_bigram_wiznet.bin - those are built from a git clone made at around 1:14 AM on the listed day) I had no idea you could do LED1.write() before, honesly. LED2.write(1); //works |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-06 by @gfwilliams Thanks for letting me know - I'll try and get a fix in for this today. Looks like it's converting the pin number to an integer by accident. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-06 by @gfwilliams Just fixed. In an hour or so the latest should be available at: www.espruino.com/binaries/git/commits/551b5c4adc7962e2485224c767816b9534736d15 |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-06 by JumJum Thanks, just tested and works fine. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-09 by @joakim Sorry if this is the wrong thread, but I'm also getting bogus readings from the BMP180 as reported in comment #12. For example Is there something I can do to work around this in the meantime, or should I wait for a fix? |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-10 by DrAzzy I'm gonna try and fix this tonight. I think it can be fixed with judicious use of parenthesis Edit: Okay, yeah, I see the problem. It looks like that was written for an unsigned 32-bit integer (the code was ported from arduino), and still has workarounds for handling the large numbers.
In my tests at ambient pressure, B7 is 1.6 billion or so - so when it left-shifts it, it turns into a negative number. In at least the current ambient conditions, changing that to
seems to work, but I don't think it will work at all pressures - I fear that B7 can get large enough that it could roll over to a negative number on the line before that one (since they test whether it's greater than 0x80000000, which is the point at which it would have already rolled over) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-10 by @joakim Thanks a lot! Works great, for my current ambient pressure at least. It will do the trick until the underlying problem is fixed :) |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-08-11 by @gfwilliams I've just pulled in @drazzy's changes, so hopefully this will magically be fixed the next time you upload your code. |
Beta Was this translation helpful? Give feedback.
-
Posted at 2014-06-26 by @gfwilliams
Hi - I thought some of you might be interested in the packed_array branch I just uploaded. I've made a build of it at http://www.espruino.com/binaries/espruino_1v65_espruino_1r3_packed_array_beta.bin
Basically, if a value is an integer between -32768 and -32767, it packs it into the same variable block as the variable name. That means that normal sparse arrays and objects are now twice as efficient when filled with small integers.
This is a pretty big change - It passes all the tests but there may still some issues when running code - If anyone can come up with a simple code example that shows up problems it'd be greatly appreciated!
A quick example of the improvements:
So:
@bogdang will be happy anyway - finally something that improves memory usage on the VLdiscovery!
Beta Was this translation helpful? Give feedback.
All reactions