Node decoding + Threading + Bulk upload #33
Conversation
…ut for me 2014-05-02 18:37:38,343 INFO Serial RX: 10 49 0 47 0 203 0 183 12 was giving: 2014-05-02 18:37:38,345 DEBUG Data string: [[0,10,0,0,0,37662610412320084583014736539245998496614132666784207011840]]
Conflicts: src/emonhub_listener.py
…aking out at the point this is detected + addition of byte and correct handling of signed long
…decoding if file does not exist
… body of post request rather than url
…ss dispatchers properly, tested uploading to two servers, turning off internet connection to remote, independent buffering, all seems to work
|
Ok, Im quite happy with this now, from brief testing its a working solution for:
Im quite new to Python so interested to hear your more experienced thoughts on it. |
- Allow for posting to different emoncms interfaces ie: input/bulk or node/multiple - These minimal changes give full compatibility with remote based emoncms node interface if application needs this approach without bulking out emonhub with code duplication
|
Latest commit which I've bundled in here:
|
|
This is tricky for me to review. It would be helpful if we could break it down into separate pull requests:
(I guess the last one could come in any order, whatever is most convenient. I'm slightly nervous about multithreading it "just like that" because I have plenty of experience of multithreading agony, maybe I'm being overly pessimistic!) |
|
In fact, is there even a fourth thing in here? Just forwarding on the bytes? I'll pick this up in #22 too |
|
I've separated out the node decoding and bulk upload leaving the changes for threading only https://github.com/emonhub/emonhub/pull/34/files |
…osted to a server at once, designed to limit strain on remote server
|
Added the ability to set the max number of items that can be posted in one bulk upload, so that under conditions where the internet has been down for an extended time it doesnt try and send too much at once, if say emoncms.org is down for a day and hundreds of connected rpi's try and upload at once it might be too much, so this makes it possible to stage the upload. |
|
Glyn and I are now testing this in our own homes on test systems to test its long term reliability. Will also try some tests with unplugging the ethernet for a few hours to see how well it posts the buffered data. |
…rval being the request time + a fixed 5s sleep time
|
(just for ref) http://openenergymonitor.org/emon/node/5239 |
|
Closing as node decoding and bulk upload is now done very nicely by @pb66's emonhub developments and I've opened another pull request with threading additions only. |
A start on adding threading to emonhub. This works with one dispatcher at the moment as I think it needs either a queue for each dispatcher or a pub/sub queue that handles this for us. At the moment I think the queue will share the data between dispatchers rather than duplicate the data which is the behaviour we want.
This pull request contains the node_decoder pull request too.