-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Try to connect to Lego Boost #4
Comments
I have these issues already fixed on my branch, but I did not succeed with maintaining a usable connection yet. Feel free to experiment with it. |
Hi @jfevers , thank you for the feedback. I took a look in @mdevel1 's LEGO_Boost_fixes branch. It looks addressing your issues. Regarding the adtype, micro:bit looks returning only 0x3 "Complete List of 16-bit Service Class UUID". As @mdevel1 notes at commit 9d5333, LEGO boost returns adtype 0x7 "Complete List of 128-bit Service Class UUIDs". I think we can check both adtypes in the match() function. I guess this 0x7 adtype support is probably good for LEGO WeDo2 and Go Direct also because extension source code for those two define 128 bit service IDs. @mdevel1 , it is fun to see your work is making progress :) Pull requests will be welcomed. |
@jfevers @mdevel1 |
Yes, I will try out, but not before 2nd or 3rd of February. -- Dr.-Ing. Jan-Friso Evers-Senne Josef-Sorg-Str. 4179822 Titisee-Neustadt 07651-9727660 0179-1499217 friso@evers-senne.de
-------- Ursprüngliche Nachricht --------Von: kawasaki <notifications@github.com> Datum: 25.01.20 11:04 (GMT+01:00) An: kawasaki/bluepy-scratch-link <bluepy-scratch-link@noreply.github.com> Cc: jfevers <friso@evers-senne.de>, Mention <mention@noreply.github.com> Betreff: Re: [kawasaki/bluepy-scratch-link] Try to connect to Lego Boost (#4) @jfevers @mdevel1
I have created dev branch: https://github.com/kawasaki/bluepy-scratch-link/tree/dev
I cherry-picked two commits from @mdevel1 's LEGO_Boost_fixes branch, and I added a commit to handle both of adtype 0x7 and 0x3. The branch works ok for micro:bit. I expect the branch can detect and connect to LEGO Boost. Would you mind to try the branch?
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.
|
I rebased my branch on your dev branch, and fixed another exception. The connection stays established now, and I can use the editor, but when I run a scratch program, it doesn't have any effect, though the transmitted commands (writes to the characteristics) look valid. I will debug it further when I have free time. |
@jfevers |
It looks that more users want to link Lego Boost with Scratch via Linux. To make bluepy-scratch-link to support it, I cherry picked works by @mdevel1 in LEGO_Boost_fixes branch of mdevel1/blupy-scratch-link repo. I took the liberty to do minor changes in those commits to make code simpler, keep letters per line within 80 and improve commit messages more readable. Anyway, all of those changes are good and make the bluepy-scratch-link more stable for other devices than LEGO Boost. Thank you so much @mdevel1 . I hope that LEGO Boost users to try out bluepy-scratch-link and provide feedbacks. Does it control motors well? How about sensors? |
Dear @kawasaki,
|
@laurentchar Thank you for the trial and the report! Now I have started to look into those logs. |
Hi @laurentchar, thank you again for the logs. It is a fun to analyze them. Still I have not yet fully understood why Lego Boost does not work via bluepy-scratch-link. Just I share what I found. The discovery and connection steps look ok. After that Lego Boost reported available devices to Scratch and bluepy-scratch-link. In the blupy-scratch-link debug log, I found following logs that Lego Boost sent out. It looks all of the available devices are listed including LED. Port numbers look valid, starting from 0, and up to 0x46.
Bluepy-scratch-link sent out these information to Scratch as notifications, then Scratch should be aware of LED. After that, Scratch sent out data to set LED color.
However, the sent out data specifies port number 0xFF. This looks weird. It was already reported that LED has port 0x32. Then, this implies that the bluepy-scratch-link failed to notify the device list to Scratch. I will need some more investigation to understand the cause. For further investigation, I can think of two actions. First action is to add debug prints on Scratch side: (scratch-vm:src/extensions/scratch3_boost/index.js handles the Lego Boost extension). And modify bluepy-scratch-link to emulate Lego Boost based on the log provided. This will help me to do some trial with my local environment. I will need some more days (or weeks?) to try this out. Second action is to try out other notification methods by bluepy-scratch-link. In the @mdevel1 's LEGO_Boost_fixes branch, some notification send out methods are tried. I prepared "boost" branch in my bluepy-scratch-link repo which try out another notification method. @laurentchar , if you have time to afford, could you try my boost branch? Another interesting part of the log was that Lego Boost reported firmware version. As far as I analyze the log, the firmware version of the Lego Boost Move Hub was 0.0.00.1706.
|
Firmware version is reported as: b'\t\x00\x01\x03\x06\x17\x00\x00\0x20 ': |
@laurentchar You are right, I overlooked the last 0x20 since it was printed as white space in the log file. The firmware is pretty new and looks good. |
This weekend, I spent some time and implemented dirty Lego Boost simulator on the bluepy-scratch-link. It returns the messages what real Lego Boost returns to Scratch. The debug log by @laurentchar was a great help. Now I can play with Scratch Lego Boost extension even though I do not have Lego Boost :) Using the simulator, I tried to connect Scratch to the Lego Boost and observed what is going on on Scratch side using browser console log. And I found that the recent commit 2c7a10f ("BLESession.notify: Send out notifications at once to avoid interruption") caused a syntax error on the Scratch side.
The commit merges multiple notifications in bluepy-scrach-link and send them to Scratch as one shot. This sends out multiple JSONs to Scratch. However, Scratch side expects single JSON. Then this resulted in the error. I referred the websocket send() API document. Each message string should be sent out in each "frame", and the message string should be implemented as iterator or asynchronous iterator. I tried to pass multiple notifications to send() using iter(), it resulted in another new error. Maybe I need to pack those multiple notificactions as asynchronous iterator. It looks that it will add some complications. At this moment, I think it is easier to revert the commit 2c7a10f. I reverted it, and confirmed that the Syntax Error on the Scratch side disappears. I tried to trigger "set color light to" block in the Scratch, and confirmed that bluepy-scratch-link received a write request:
This time, port ID is not 0xff. It has correct port ID 0x32. Looks good. @laurentchar , could you try out "boost" branch? The branch does not have the commit which caused the Syntax Error. I expect it will work better. At least it can control LED, I expect. |
Dear @kawasaki Nothing changed on Lego Boost (LED stayed blue). I also noticed that scratux-link.py log file was not reporting any activity when the scratux program started and ran. Attached is the full logs of the session (wireshark dumps: use SSL keys provided, scratch-link.log). Thank you. |
The wireshark traces match the scratch-link.py log file until the wireshark line #74 (request: { ... "id":4} with message 'CgBBAQIBAAAAAQ==' i.e. 0a 00 41 01 02 01 00 00 00 01) Starting from wireshark line #76 ("CgBBAAIBAAAAAQ==", i.e. 0a 00 41 00 02 01 00 00 00 01) scratch-link.py log file does not report any messages/transactions. When scratch program starts (setLED color): this is wireshark line #116 ("CgCBMhFRAQD//w==", i.e. 0a 00 81 32 11 51 01 00 ff ff setLight to 0x00FFFF=cyan) and it does not change the MoveHub LED color. |
@laurentchar Thank you very much for trying the boost branch. Now I have started investigating the log. The event recorded in the logs related to the write message CgBBAQIBAAAAAQ==' i.e. 0a 00 41 01 02 01 00 00 00 01. This indicates the request from scratch to Lego Boost to start sending notifications from the motor at port 1. Scratch-link log records that this message was received from Scratch, but it does not record the log after it sent to Lego Boost. Now I guess that deadlock happens within two threads of Scratch link. Will take a close look further. |
According to the debug log by bluepy-scratch-link, it looks that the eternal wait is happening within "self.session.perip.waitForNotifications(1.0)" at line 357 of scratch_link.py, probably. The argument 1.0 specifies to return within 1.0 second. Then I do not expect it blocks eternally, but it looks happening. @laurentchar I have pushed another commit to the 'boost' branch to add some debug prints. Could you try it again and share the debug print by scratch_link.py? It will clarify if the line 357 causes eternal loop or not. If bluepy causes the eternal loop, I will need to think about debug within bluepy, or try other python BLE modules. If bluepy does not cause the eternal loop, still I am missing dead lock cause within bluepy-scratch-link. |
Dear @kawasaki |
I restarted scratch_link.py several times but coudn't change the LED color anymore. When MoveHub connects to scratch_link.py (LED turns blue) the last debug line is:
When I start the scratch program nothing happen anymore (LED does not change, no more log from scratch_link.py). |
@laurentchar Thank you for the log. It always helps. I confirmed that the dead lock happens in bluepy-scratch-link, and I think I understand the reason. It one thought that the dead lock happened in bluepy, but actually it happened in the callback function that bluepy-scratch-link implemented. There are two threads in bluepy-scratch-link: one for websocket, and the other for BLE handling. As an exception, I made the BLE thread to send webocket only for notification sending, but this part looks causing the deadlock. I will try to clean this implementation using queue. |
@laurentchar I have made another commit to the 'boost' branch tip. The BLE thread no longer touches websocket. The notifications are passed from BLE thread to websocket loop thread through queue. I wish this fixes the daedlock issue with Lego Boost. Not 100% sure, but please give it a try. |
@kawasaki Thank you for your commit. The scratch program (change LED color) does not work (MoveHub LED does not change color). |
Hi @laurentchar , thank you for the log. I feel sorry that it didn't go well. I misunderstood the root cause. Still the dead lock happens, and I think the latest log indicates that dead lock might happen in bluepy. Then next action will be to add debug logs in bluepy. This co-work between you and me is a kind of fun for me, but I wonder if you are ok to continue this style of work. I have not yet still understood the root cause, and it will need more catch-balls between you and me (several more, or dozen more). Next action will need to ask you to edit some files on your machine (bluepy related files). If you think this is too much for you, please let me know. I still wish to make Lego Boost connected with Scratch on Linux... If you can continue the effort, I would like to ask you following actions:
Best wish and cheers |
Hi @kawasaki , yes I want to continue to debug the issue :) I am ready for hundred more tries. Here is the log with commit 66c7803 and additions to the btle.py. |
@laurentchar Thank you for the strong help! It is encouraging :) I have took a look in the latest log. It indicates that the self._helper.stdout.readline() call did not return and caused the deadlock. Let me have some time to dig in further and think about the next action. |
This is a note about what I found today: the scratch_link.py stopped at waitForNotifications() in bluepy btle.py. And the last log indicated that the process stopped at self._helper.stdout.readline() in btle.py. This line tries to read standard output of bluepy-helper process until line terminator '\n'. This readline() stops probably because the bluepy-helper has not yet output the line terminator '\n'. In other words, bluepy-helper has not yet completed to output single message for bluepy. But bluepy started reading the bluepy-helper standard out because the line sefl._poller.poll returned the non-zero file descriptor, it means that bluepy-helper started writing something to standard out. In short, bluepy-helper started to write out a message to bluepy, but has not yet completed. I took a look in bluepy-helper.c. It uses Gnome Glib framework for event handling and uses standard in and standard out for communication with bluepy. It defines some helper functions to send out messages to standard out: resp_begin(), send_uint(), send_str(), and resp_end(). The function resp_begin() starts writing to the standard out, and the function resp_end() completes writing a message with '\n'. This resp_begin() and resp_end() are paired at all places in bluepy-helper.c. For some of those pairs, logic between begin and end is very simple and not to cause stop. However, some of them have loops with link list. If the link is broken, the loop can be broken, and then it explains why '\n' is not written. Next step is to understand status of bluepy-helper at the bluepy-scratch hang. For this purpose, I can think of three actions: 1) Linux /proc status check can be a help. 2) Also, debug print in bluepy-helper can be a tool for further understanding. 3) Wireshark dump on BLE protocol can be the other tool for understanding. Before asking actions, I would like to spend some more time to confirm which action is valid. |
@laurentchar May ask you to try bluepy-scratch-link and lego boost again? It is ok to try with same condition as the last time. Before you stop the bluepy-scratch-link, please do following actions. Get PID (process ID) of bluepy-helper with following command. In the example below, 2100 is the PID.
Start gdb, attach to the process and get back trace.
And please share the gdb backtrace output. It will tell the status of the bluepy-helper at the hang. |
Hi @kawasaki Here is the gdb backtrace. |
@laurentchar Thanks for the log. The backtrace is not much what I expected. It tells that the event loop of the blupy-helper is still working, and does not imply it is hanging. I will think about another approach. |
I took a close look at the bluepy log output, and noticed a weird thing.
"readline" log and "Got: *" log should be paired. One "readline" log is not paired with "Got: *", then it indicates that readline() caused the hang. This was known. The weird thing is that, two more "readline" log and "Got: *" log pairs are printed after the single "readline" log. It means that after the hang, two messages were sent from bluepy-helper to bluepy. This indicates that bluepy-helper does not hang even after the bluepy hang. Also it implies two threads may have called readline() almost same timing and might have caused inconsistent file descriptor status. I checked how two bluepy-scratch-link threads call bluepy APIs and how those calls are connected to readline() calls. And I noticed that bluepy-scratch-link's websocket thread calls bluepy getCharaceteriscits() and getServiceByUUID() APIs without lock guard and they may result in readline() call and may conflict readline() call by bluepy notification handler thread. I misunderstood that these APIs do not need the lock guard. To avoid this conflict, I added lock guard to getCharaceteriscits() and getServiceByUUID() calls. @laurentchar Could you try the latest 'boost' branch of bluepy-schratch-link? Its hash is 5dcbbfa. |
Dear @kawasaki Good news commit 5dcbbfa works! Thank you very much. MoveHub is a bit slow to start (a 1 or 2-second latency between the green button press in scratux and the program execution on MoveHub). As an example I noticed that "turn motor_AB" does turn motor A then motor B. The latency between the start motor_A and motor_B causes MoveHub to not start straight. But this is probably an issue in scratux and the Lego Boost extension. I will dig more. Laurent |
@laurentchar At last! Thank you very much for the good news and your strong support :) Hmm, about the latency, I'm not so sure but bluepy-scratch-link could be the cause. There are a couple of timeout constants in bluepy-scratch-link, and one of them is set to 1 second. If the BLE device sends out many notifications frequently, this 1 second wait is not a problem (micro:bit has no problem). But if Lego Boost does not send out the notifications often, this 1 second wait can be the cause of the long latency. I modified the wait time from 1 second to 1 millisecond, and pushed to the 'boost' branch. Could you try the commit 9ba1396. I'm not so sure, but I think it's worth trying. |
The 1 ms wait time seems to be a bit better (but it is hard to tell). |
@laurentchar Thanks for trying. Wish your work on scratux goes well. I also took a look in Lego boost Scratch extension. It looks that it sends out separated messages to Lego Boost for each motor A and B. If this can be handled as one shot, it may solve the problem. I have pushed the fixes for Lego Boost to the master branch. I'm happy that we made this progress so far. Thank you again for your support. Still the two motors control challenge is left. I would like to keep this github issue open. Sharing your progress will be appreciated. |
Thank you @kawasaki and @laurentchar for your work on getting This works well on Ubuntu Studio 20.04 with an Asus USB-BT-400 (4.0 with LE) USB adapter. For young children to use this without depending on adults, it is more convenient to make it work without using
I think this would be good to add to the
It is also necessary for each user to have their own copy of There are some rough edges about connecting to the LEGO Boost brick: each time it takes 2 attempts:
Please let me know if there's something I can do to investigate the connectivity issue. |
Thank you @miguev for the valuable comments. I agree that those points are room to improve. Let me have some time to think about next actions. |
Hi @miguev The comments on sudo and multiple users support are not specific to Lego Boost. I opened other issues to separate discussion. The 2 attempts required at connection looks unique to Lego Boost. Could you take debug trace with -d option of scratcy_link.py? I would like to take a look in it. Also, I wonder if you see "turn motor_AB" results in different motor start timing between motor A and motor B. @laurentchar reported it with scratux in his environment. |
Hi @kawasaki here it is the debug output from the second time I could reproduce the issue in my previous comment today: First time I tried, the connection was successful on the first try, this is the debug output from that time: Also, the first thing I found today is that having multiple Bluetooth adapters may cause trouble if the "first" one does not support LE. When I tried the above today, I had my old 2.0 adapter plugged in, and this is what happened: This might be a problem with old laptops with integrated Bluetooth that don't support LE. |
Forgot to mention: I too see that "turn motor_AB" results in exactly the same behavior @laurentchar reported: motor A starts first, and maybe half a second later motor B starts. |
@miguev Thanks for the debug outputs! Will look at it closer this weekend. I have not though much about multiple Bluetooth adapters. Will need some thought. Thanks for letting me know that you see "delayed motor start" symptom. I will think what I can do for it. I would like to clarify if it is caused by scratch-link, or caused by scratch extension. |
The debug log "connected-on-second-try.txt" tells that BLE device scan happened twice, for each click "Starting Search". The found device list for the first scan does not include the Lego boost, but that of the second scan includes the Lego boost. Scan itself looks working well. I'm not so sure, but I guess the too short scan time can be the cause of the failure. I would like to extend the scan time from 1 second to 10 seconds, and see if it will make the Lego boost scan more robust. Regarding the multiple Bluetooth adapters, it looks easy to support using bluepy. I have implemented a code to support up to 3 adapters. @miguev I have pushed trial code in the "dev" branch of this github repo. Two changes in the branch address "2 attempts required" issue and "multiple Bluetooth adapters" issue. Could you try the branch in your system and check if the issues solved? |
Tried "dev" branch and found I can connect to LEGO Boost on the first try consistently. It also works with multiple Bluetooth adapters, at least the first time around. These are logs from 2 different PCs: scratch_link_1st_try.txt The (possibly new) issue happens when, without interrupting execution of scratch_link_2_tries.txt On a PC with only two Bluetooth adapters, it seems to get stuck always trying to use only one of them: |
@miguev Thank you for the detailed report! It is good that the problems on first connection and multiple adapters looks resolved. Regarding the reconnect after disconnect, I walked through the logs. The logs at reconnect steps imply that rescan did not find the LEGO Boost. The first connection was ok, but after that it fails. So far, I do not have clear understanding about the cause, but my guess is that sudden disconnect by PC might have surprised the LEGO Boost, then rescan may fail to find out the LEGO Boost. Cleaner disconnection or some wait between disconnect ant rescan might help. I will think about trial code. |
@miguev I have pushed some more changes to the "dev" branch to address the reconnect issue. The changes cleans up BLE connection when Scratch disconnect button is pushed. Could you try it in your environment? |
I have merged the changes to the master branch. I guess and wish the left work is the only "turn motor_AB results in motor A starts first, and around half a second later motor B starts". |
@kawasaki A huge thank you for all the work you've put in to making scratch_link! I'm amazed that people with such expertise do these things for free/fun! I've managed to connect my son's Boost using Scratch running in Firefox 79 on Ubuntu 16.04 on an ancient Acer Aspire 1810TZ (had to buy a separate Bluetooth adapter and build Python 3.7 from source, but that's about all). Your instructions are excellent. I've also noticed the "turn motor_AB starts motor A before motor B", it's not bothering me. After about 30min of messing around in Scratch I saw that Python 3.7 was using over 2.5GB of the 3GB in the laptop. The only thing running in Python 3.7 is the scratch_link. I see now that I was running the Thank you once again for the amazing job you guys have done! |
Hi @jacquesdt Thank you for sharing your experience! Good to know that the bluepy-scratch-link works on your PC. The 2.5GB memory usage by bluepy-scratch-link is too large and weird. In my environment, memory usage is a few MB order. I observed that when I repeat micro:bit connection and disconnection, the memory usage increases by +70MB. Such memory increase is not ideal and will need investigation, but I'm not sure if this is the cause of the 2.5GB memory usage. If you observe the large memory consumption with the master branch, information sharing will be appreciated. |
Thanks, will do. Just so I get some idea what kind of information to try collect: I imagine the console log will be needed, but probably you'll need other things as well? What sorts of things should I go looking for? I'll try run scratch_link through valgrind massiv - it's worth a shot, but is likely to be unusably slow on that old laptop (it also requires debugging symbols). I'll probably get further with instrumenting the Python with psutil for for that I'd need some pointers on where in the Python to go put the print statements. |
@jacquesdt That's the good question:) And thank you for sharing the idea to utilize "psutil". It looks handy. One idea I have now is to add memory usage infomration to all debug log messages so that the log can tell when memory usage increases. (Same manner, I want to add time stamp to the debug messages to make it easier to understand performance issue or dead lock.) Now I'm kind of busy. It make take some weeks for me to work on that memory size debug log idea. |
@kawasaki I checked out We spent some time fiddling with the proximity sensor - I'm not sure how it works, but I'm guessing it's based on some kind of "differential" signal. We saw odd behaviour: the sensor does work, but it's as if the polling frequency is very low. If I tell the Boost "rover" to stop when it sees any colour brick, then it ploughs into my brick wall and continues to push it backwards without ever stopping. If instead I pick the rover up and slowly move the brick wall towards it, then it does stop. Does anyone know the basic principles on which the sensor works? On a more general note, my motivation for using Scratch was that the Boost "programming language" seemed extremely limited. Yesterday I spent some time on the internet and discovered that there is in fact a "Creative Canvas" mode with a really large set of functionality. Interestingly, this is larger than what LEGO reports on their own website. It's also larger than what is available via the BOOST puglin for Scratch. I thought that LEGO itself would be supporting the Scratch effort, but I guess that is not the case. In terms of moving LEGO Boost away from the App and into Python land, I see there is PyBricks. I can't quite figure out whether this is an "official" repo supported by LEGO - it seems not to be. I also see that LEGO seems to have released the specification of v3.x of their Wireless protocol which surprises me a little (but in a good way). |
Hi @jacquesdt
The pylgbst library is very good. Laurent |
@jacquesdt @laurentchar Interesting report and discussion :) I myself do not have Boost and difficult to comment. Yes, it is great that LEGO opens the protocol so that everybody can play with it. Some other programming environments look fun! One point I wonder about Scratch-link is overhead of communication between scratch and BT/BLE devices. To control the motor on the device based on input by the sensors on the device, the communication path "from the sensor to scratch" and "back from scratch to the motor" is not efficient. If the program can be downloaded on the BT/BLE device, program can control the sensor and the motor without communication overhead, and it should be much faster and be more smooth/accurate control. As far as I play with micro:bit and Scratch-link, this overhead is not visible, but I'm not sure if other Scratch-link supported devices have the communication overhead issue (it may be what @jacquesdt saw with the sensor). If the devices can have program in it, it will be the better way for such realtime-ish programming. (Scratch and Scratch-link are good for kids who used to Scratch to expand their target area, but not good at realtime control, probably) |
This issue is quite old and most of the issues discussed in this issue were already addressed. Thank you for all of the contributors. I think it's ok to close this issue now. The latest version has BLE latency issue fix, and it may resolve the staggered motor start issue, hopefully. Another issue #20 about Lego boost is now open. Comments by Lego boost users will be appreciated. Anyway, let me close this :) |
Dear Kawasaki,
I found your bluepy-scratch-link project and got it up and running folowing your README. My goal is to control the Lego Boost robot from scratch 3.
I found one very small issue at line 198 (del(self))
close(self) did not work for me, I chaged it to self.close() and it works.
Then, connecting from Chrom+scratch works fine.
Next step was that in function matches(self,dev,filters) the conversion from string s to type UUID dit not work when trying to cast to int() first. so I changed "given_uuid = UUID(int(s))" to just be "given_uuid = UUID(s)". works.
(btw: I can create a pull request later, if required).
Can you try to confirm these bugs and check if you micro:bit works with my fixes as well?
Now the next big thing:
Scratch requests a service with UUID 00001623-1212-efde-1623-785feabcd123 nad if I sniff to the boost device using gatttool, I can see that it advertises exactly this ID. But so far I have no clue how to make use of this.
Last action I can observe is, that the line "service_class_uuid = dev.getValueText(0x3)" results in NONE and the next sanity check bailes out. So either dev is not filled properly, or the access by getValueText() is not robust to find the right string.
Can you help me adapting your script to Lego boost? I can supply you with scans and details, work on the code and test it.
best,
friso
The text was updated successfully, but these errors were encountered: