[Bug] Code fails to run from the edit window

[cuernodegazpacho]

When attempting to run from the edit window, Code freezes as soon as it hits a reference to an object internals.

In this test, I create a hub instance and print it. Works as expected.

If I add anything that accesses internal references, Code freezes (notice the frozen top controls). It has to be re-started from scratch.

From the terminal window, the same commands work as expected.

Since it works from the command terminal window, I believe the issue is not in pybricks itself, or the micro python code, or the BLE communications. Seems to be something in Code itself.

I've been testing communications with pylgbst, and everything works fine with my setup.

This is from pybricks v3.2 on Mac OSX 11.7. I tried both via Chrome browser, and a downloaded, local Code, with same results in both cases.

[dlech]

This is working for me (tested on both Linux and Mac (macos 13.2)). So, I'm afraid there is a problem with your Mac or your City hub.

• Does the light on the City hub change when Pybricks Code locks up (e.g. starts flashing blue because it disconnects)?
• Do the batteries in the hub need to be replaced/recharged?
• If you have any other Bluetooth devices, turn them all off and see if it makes a difference.
• Also see Troubleshooting Bluetooth in Pybricks Code #270 which describes how to look at the various logs. If you can share those, it could be helpful in troubleshooting.

[cuernodegazpacho]

Thanks for your prompt answer! I have 2 CityHub units, both with fresh batteries, and both cause the same error, so I doubt it's the hub. I will try your suggestions and report later.

[laurensvalk]

If I add anything that accesses internal references, Code freezes (notice the frozen top controls). It has to be re-started from scratch. (...) Since it works from the command terminal window, I believe the issue is not in pybricks itself, or the micro python code, or the BLE communications. Seems to be something in Code itself.

It could still be the firmware. If the hub crashes right when the program starts, I suppose Pybricks Code might not have received the status that a program started, hence the locked buttons.

Does this also happen if you make a program with a motor or sensor?

from pybricks.pupdevices import Motor
from pybricks.parameters import Port
from pybricks.tools import wait

# Initialize a motor on port A.
example_motor = Motor(Port.A)

# Make the motor run clockwise at 500 degrees per second.
example_motor.run(500)

# Wait for three seconds.
wait(3000)

# Make the motor run counterclockwise at 500 degrees per second.
example_motor.run(-500)

# Wait for three seconds.
wait(3000)

What is the Bluetooth name that you gave the hub during installation?

What is the output of the following program?

from micropython import mem_info
mem_info(1)

[zus2]

When attempting to run from the edit window, Code freezes as soon as it hits a reference to an object internals.

In this test, I create a hub instance and print it. Works as expected.

If I add anything that accesses internal references, Code freezes (notice the frozen top controls). It has to be re-started from scratch.

From the terminal window, the same commands work as expected.

Since it works from the command terminal window, I believe the issue is not in pybricks itself, or the micro python code, or the BLE communications. Seems to be something in Code itself.

I've been testing communications with pylgbst, and everything works fine with my setup.

This is from pybricks v3.2 on Mac OSX 11.7. I tried both via Chrome browser, and a downloaded, local Code, with same results in both cases.

did you resolve this Cuerno @cuernodegazpacho ?

[laurensvalk]

@zus2 - if this happens to you, can you please describe the exact steps to reproduce it?

[zus2]

Hi yes same as OP - works from terminal window but as soon as I try to upload the command dir(hub.light) subclass/method - for example it hangs - I have to disconnect and re-pair

I just presumed my osx / system was too old - 10.15 Catalina / ASUS bt400 Broadcom dongle in a 2011 MacBook Air - but the OP was running MacOS 11

[zus2]

The motor test program works in the command interface but not from the run window. One line works OK - import Motor - or import Port - but 2 lines hangs.
I will post the memo_info output

[zus2]

Output from mem_info()

from micropython import mem_info
mem_info(1)
stack: 580 out of 5452
GC: total: 19840, used: 2304, free: 17536
No. of 1-blocks: 27, 2-blocks: 8, max blk sz: 32, max free sz: 1087
GC memory layout; from 20001930:
00000000: h=hhhDh======LhLhh=h=hh===h==hh=====h=h=h=======================
00000400: ========h==h=============================h=======ShThh====h==hhh
00000800: BThhhhhB..hhh=h=.......h=.......................................
(16 lines all free)
00004c00: ........................

[zus2]

I have used the default hub name Pybricks Hub. It's the City hub

[BertLindeman]

Ran a test program on code.pybricks:

MicroPython for LEGO® Powered Up smart hubs.

v3.3.0 (Pybricks Code v2.2.0)

Copyright (c) 2020-2023 The Pybricks Authors

City hub at 3.3.0:
Output from edit window misses the output of the dir command:

('cityhub', '3.3.0', 'v1.20.0-23-g6c633a8dd on 2023-11-24')
<CityHub>
before dir
after  dir
('cityhub', '3.3.0', 'v1.20.0-23-g6c633a8dd on 2023-11-24')
<CityHub>
before dir
after  dir

From the IDE (forgot how to call this thinghy):

Pybricks MicroPython v1.20.0-23-g6c633a8dd on 2023-11-24; Powered Up City Hub with STM32F030RC
Type "help()" for more information.
>>> from pybricks.hubs import CityHub
>>> from pybricks import version
>>> print(version)
('cityhub', '3.3.0', 'v1.20.0-23-g6c633a8dd on 2023-11-24')
>>> hub = CityHub()
>>> print(hub)
<CityHub>
>>> print("before dir")
before dir
>>> dir(hub)
['battery', 'ble', 'button', 'buttons', 'light', 'system']
>>> print("after  dir")
after  dir
>>> 

I saw an update ready for code.pybricks.com, but did not yet update.
Will do now..

[EDIT] Update did not change anything in this.

Bert

[zus2]

Is this a malloc thing ?

1. This runs

from pybricks.hubs import CityHub
hub = CityHub()
print(hub)

2. This runs

from pybricks import version
print(version)

3. This locks up

from pybricks.hubs import CityHub
hub = CityHub()
print(hub)
dir(hub)

4. This also locks up

from pybricks.hubs import CityHub
from pybricks import version

(nothing else)

[laurensvalk]

Thank you both! For each of your programs above, what was the file name?

This gets saved as part of the program, so if it's a memory access issue, we'd need the name to reproduce the exact same scenario.

[laurensvalk]

I see for OP it was test7.py, and I still can't seem to reproduce it.

Does this same issue occur if you try to run this code from an Android phone?

[laurensvalk]

Another question for those able to reproduce it. Is it possible to adapt the crashing program so it looks like a variant of the following?

The idea is to figure out if this only happens if the hub is still connected to the computer, or if it also crashes the same way if the program is run from the hub without a connection.

To test that, we need two things:

• A program that delays the crash, so we can disconnect before it happens. (Or stop the program and shut the hub down before it happens.)
• Another indication like a light to indicate it was successful or not, since we're disconnected.

from pybricks.hubs import CityHub
from pybricks.tools import wait
from pybricks.parameters import Color

hub = CityHub()
hub.light.on(Color.RED)
wait(2000)


# is it possible to insert some code here to reproduce the crash?

hub.light.on(Color.GREEN)

wait(2000)

[laurensvalk]

@BertLindeman, I don't think it reproduced in your case. The dir command is not expected to print anything if there isn't a print() command around it.

[zus2]

The dir command also prints at least in the interactive window.

I have been testing this all day - I am no expert on python language internals , objects, Pybricks etc - but it seems we simply have a bug on the run window which crashes after 2 method or property calls - internals - even 2 import commands will not run. I have tested with CityHub and TechnicHub and 2 computers - BUT both are old - OSX 10.11 and OSX 10.15 Catalina running on MacBooks , with broadcom BT dongles , from 2009-2011. Perhaps it will save everyone a lot of time just to assign this bug to non-supported hardware?

[dlech]

If you want to dig into it more, it would be helpful to capture Bluetooth packets for a working and not working case.

Instructions on how to do this can be found at https://bleak.readthedocs.io/en/latest/troubleshooting.html#macos

[zus2]

Don’t get me wrong - I love fixing things ! But I do not want to flog a dead horse wasting people's time - all hardware has to retire eventually ! I will check out the packet logging info Best

…

On 21 Jan 2024, at 01:25, David Lechner ***@***.***> wrote: If you want to dig into it more, it would be helpful to capture Bluetooth packets for a working and not working case. Instructions on how to do this can be found at https://bleak.readthedocs.io/en/latest/troubleshooting.html#macos <https://bleak.readthedocs.io/en/latest/troubleshooting.html#macos> — Reply to this email directly, view it on GitHub <#947 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKHCFE7W2IORMLA57QUQ4ZDYPROAFAVCNFSM6AAAAAAUX7GKESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGQ3DCMJUHA>. You are receiving this because you were mentioned.

[zus2]

Hi - there are hundreds of lines over several seconds - the time it takes to start logging , switch apps to run the code ( sucessfully or not ) , and stop logging - is that going to be meaningful to you? Paul

…

On 21 Jan 2024, at 01:25, David Lechner ***@***.***> wrote: If you want to dig into it more, it would be helpful to capture Bluetooth packets for a working and not working case. Instructions on how to do this can be found at https://bleak.readthedocs.io/en/latest/troubleshooting.html#macos <https://bleak.readthedocs.io/en/latest/troubleshooting.html#macos> — Reply to this email directly, view it on GitHub <#947 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKHCFE7W2IORMLA57QUQ4ZDYPROAFAVCNFSM6AAAAAAUX7GKESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGQ3DCMJUHA>. You are receiving this because you were mentioned.

[dlech]

yes, we can filter the packets

[laurensvalk]

Don’t get me wrong - I love fixing things ! But I do not want to flog a dead horse wasting people's time - all hardware has to retire eventually !

No time wasted, thanks a lot for your help!

When a certain issue can't be reproduced by the developers, we always appreciate it when the users help us debug it!

Did you say the same issue occured on your Technic hub?

[laurensvalk]

I have a feeling we were getting something like this @NStrijbosch, when we were debugging other Bluetooth features earlier.

I'm wondering if the hub sending the 500ms status heartbeat has anything to do with it, so we can try disabling it in our next debug session.

@dlech, for context - we were debugging something with Bluetooth peripherals, and when the Technic hub was connected to Pybricks Code on the PC, the hub would freeze while waiting for the authentication response. If it was not connected, or if it was connected to Pybricks Code on Android, it was working fine.

It would probably be useful to log the packets there as well.

[zus2]

TechnicHubCode-raw.txt
I also have this exported in btsnoop and pklg formats but could not see a way upload them.

OK so around 10:11:20 I ran this successfully - the hub remained connected and responsive:

from pybricks.hubs import TechnicHub

Then at 10:11:31 I ran this - which locked up the hub. I then disconnected and reconnected bluetooth:

from pybricks.hubs import TechnicHub
from pybricks.parameters import Port

filename test1.py

As I said the actual code seems irrelevant just the number of commands

eg. this runs:

from pybricks.hubs import TechnicHub
sky = "blue"

this crashes:

from pybricks.hubs import TechnicHub
sky = "blue"
sea = "green"

this runs:

sky = "blue"
sea = "green"
beach = "yellow"

this crashes!:

sky = "blue"
sea = "green"
beach = "yellow"
car = "black"

all the code examples in pybricks run correctly from the interactive command line

let me know if you need anything else

[laurensvalk]

As I said the actual code seems irrelevant just the number of commands

Agreed. The one thing all these programs seem to have in common so far, is that they "complete instantly", in that there is no waiting on time, motor movements, and so on. (By contrast, entering the same commands on the interactive REPL takes some time).

So my hunch is that some internal state like "started running" and "finished running", get intermixed.

When you say "the hub locks up", can you describe in detail what happens? Which of the following sounds the most similar?
a) The hub turns off. (light remains off)
b) The hub disconnects or restarts (the light is blinking blue)
c) The light stays solid blue and the hub can be turned off normally.
d) The light stays solid blue, but turning it off with the button won't work. When you try this, the light just starts blinking fast.
e) Something else entirely - i.e. the hub is fine but the online editor is confused?

My bet is on option (d) 😎

[zus2]

c) so actually the hub is not locked up - code is locked up ? The run button turns to a grey circle immediately and together with the stop button and interactive button >>> all show the no-entry symbol on mouse hover - only the bluetooth button allows Disconnect and I can connect as normal and then everything immediately unblocks: So it as if the code upload is not completing or the hub is not replying to say it is complete.

[laurensvalk]

Thanks! So the hub does not lock up but the app does. That makes sense. It may support this hypothesis:

So my hunch is that some internal state like "started running" and "finished running", get intermixed.

This footnote is in our code in the bit just after the status message gets sent from the hub to the PC.

        // wait for message to be sent - note: it is possible to miss status changes
        // if the status changes and then changes back to old_status_flags while we
        // are waiting.
        PT_WAIT_WHILE(pt, msg.is_queued);

So maybe something gets missed here. Although the regular status message should bring it back within a second but maybe that isn't happening.

David may be able to decipher the details from the log you shared.

[zus2]

OK I may have something. It is related to number of bytes in the program.

This works:
sky = "1234askjhalskjdhaslkdjhaslkdjhsalaldkjhaslkdjhsa"
this does not
sky = "12345askjhalskjdhaslkdjhaslkdjhsalaldkjhaslkdjhsa"

file name test1.py

[zus2]

It’s as if an EOL or EOF is being chopped off when the connection is established and the program is uploaded at the same time. In interactive mode (very!) long lines of code work fine. Did I see somewhere that pybricks used to have 2 options - one to run the code once and another to upload and store it on the hub?

[laurensvalk]

Did I see somewhere that pybricks used to have 2 options - one to run the code once and another to upload and store it on the hub?

The program gets saved on the hub either way.

Since you've mentioned that disconnecting and reconnecting works, here's something you can try.

Let's take one of the program that locked up, but did print something. You've posted e.g. this one:

from pybricks.hubs import CityHub
hub = CityHub()
print(hub)
dir(hub)

Now, run this, observe Pybricks Code lockup. Disconnect, reconnect. Now start the program by pressing the green button on the hub instead. Do you see any output now, or does it lock up again?

[zus2]

I did a slight variation on that. This runs and outputs abcd and also by pressing the green button. It pauses and the blue light pulses before outputting the string.

from pybricks.tools import wait
wait(1000)
print('abcd')

However this hangs Code as before , there is no output , no pulsing.
So when I reset bluetooth and press the green button it does nothing - it does not output abcd from the previous program, or abcde from this one and it does not hang Code either.

from pybricks.tools import wait
wait(1000)
print('abcde')

[dlech]

I also have this exported in btsnoop and pklg formats but could not see a way upload them.

To upload them to GitHub, you have to put them in a .zip file. I wasn't able to open the .txt file with wireshark.

[zus2]

Archive.zip

[dlech]

Thanks! This had exactly the information we need!

The programs are getting corrupted when being sent because we are sending more than can fit in a single packet. The Bluetooth driver is supposed to automatically split this type of message up into multiple packets (using long write instead of regular write), but it appears it isn't doing that.

This is probably getting the Bluetooth chip on the hub in a bad state causing it to stop sending status messages which causes the buttons in Pybricks Code to stay disabled. Disconnecting from the hub causes the Bluetooth chip on the hub to reset, so it gets back into a good state and things work again.

We can see here, the max MTU of the Broadcom chip on the computer (104) is less than the max MTU on the hub (158).

There is a long-standing request to add an API to Web Bluetooth to get the negotiated MTU. WebBluetoothCG/web-bluetooth#383. But since that isn't available, we made a workaround by sending the info via a custom characteristic.

But we didn't implement the case where the host computer may have a smaller MTU than the hub on the Technic and City hubs (Move, Prime, Essential and RI are implemented correctly).

https://github.com/pybricks/pybricks-micropython/blob/264dfae94afc56aca0bd482f56b4eba3eaa8e0e7/lib/pbio/drv/bluetooth/bluetooth_stm32_cc2640.c#L1280

So we should be able to fix this in the firmware.

[dlech]

I made a pull request with a potential fix. Firmware for testing is available at pybricks/pybricks-micropython#229 (comment).

[dlech]

Bluetooth driver is supposed to automatically split this type of message up into multiple packets (using long write instead of regular write), but it appears it isn't doing that.

Digging a bit further in the logs, it looks like this driver is actually using a prepared write instead of long write. EDIT: I forgot, these are the same thing.

And the hub never responds to this. It should respond with an error since it doesn't support that type of write.

[zus2]

Spectacular David. I have tested the firmware on a CityHub and a TechnicHub with the ASUS BT400 dongle on 2 different Macs and everything seems to be uploading correctly from Code.

[dlech]

Fantastic! Thanks again for gathering the logs and testing. We couldn't find and fix issues like this without people like you that are willing to invest some time doing those things!

[zus2]

Great project happy to be of help
thanks @laurensvalk and @BertLindeman / and the OP @cuernodegazpacho which led me here