Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stuck in bootloader after initial flashing (P8) #222

Open
arvchristos opened this issue Jun 23, 2021 · 42 comments
Open

Stuck in bootloader after initial flashing (P8) #222

arvchristos opened this issue Jun 23, 2021 · 42 comments

Comments

@arvchristos
Copy link

arvchristos commented Jun 23, 2021

Hi,
I followed the documentation and flashed my P8b (MOY-TON5). After flashing micropython.py, instead of being greeted with the wasp-os watchface, I am stuck in the bootloader (Start with bluetooth icon). After I press the button, I see the following pattern:

Wasp-os logo for 2 seconds, black screen of 5 seconds,Wasp-os logo for 2 seconds (again), <- Start and the BT logo appear

So far, I have tried the following:

  • removing boot.py from the manifest
  • building and flashing both the latest release as well as the master branch tip.
  • Reflashing the bootloader using reload.zip
  • Commenting out StepCounter from register_defaults
  • Wait for the battery to discharge

All of the above did not change the aforementioned pattern. Any ideas?

@Pittconnect
Copy link

Pittconnect commented Jun 24, 2021

Flashing display or stuck in a cycle has happened to me before and I call it softbricking the device. where the watch being very slow in processing and responding to the program command request When I say slow its compared my phone. I use to often press the button prematurely thinking I may be able to break out of the loop of flickering display or switching between two screens. Solution that worked for me was wait for the battery to drain completely and upload the new firmware using dfu.py and not OTA daflasher Android App https://wiki.pine64.org/wiki/Switching_your_PineTime_between_InfiniTime_and_Wasp-os But you may also try recovery bootloader https://github.com/Peetz0r/wasp-reloader/releases/tag/infinitime-0.14.1-recovery

@evanqhuang
Copy link

Flash display or stuck in a cycle has happened before and it called softbricking the device. where the swatch being very slow in processing and responding to the program command request compared with your phone people often press the button prematurely thick can softbrick the device and set it into a loop of flickering display or switching between two screens. Solution is wait for the battery to drain completely and upload the new firmware here are different methods https://wiki.pine64.org/wiki/Switching_your_PineTime_between_InfiniTime_and_Wasp-os But you may also try recovery bootloader https://github.com/Peetz0r/wasp-reloader/releases/tag/infinitime-0.14.1-recovery

I'm also having this problem and flashed the MCUBoot bootloader with SWD and it does not boot, completely black screen. What do you think I should do?

@daniel-thompson
Copy link
Collaborator

Thanks for the comments although I think there might have been a bit of a misunderstanding here. The original reporter is describing a problem using the P8b rather than the PineTime. AFAIK neither Infinitime nor mcuboot have been ported to the P8b.

Regarding the original reports, 5 seconds is the watchdog timeout so it sounds like watch is stuck somewhere during boot and it unable to feed the dog. Of course the big question is why and I'm afraid that is not clear since I've never seen anything like this before.

If I understood the report correclty the bootloader stays on the boot graphic (the wasp-os logo) indefinitely until you don't press the button? And that pressing the button makes the screen go black and then five seconds later there is a reset?

Did the same thing happen with the reloader.zip or did that run properly (when the reloader runs properly it puts up a small blue logo (a pinecone I think) which slowly fills with white as a progress meter. When the pinecone is entirely white the device should reset.

@evanqhuang
Copy link

evanqhuang commented Jun 27, 2021

If I understood the report correclty the bootloader stays on the boot graphic (the wasp-os logo) indefinitely until you don't press the button? And that pressing the button makes the screen go black and then five seconds later there is a reset?

Did the same thing happen with the reloader.zip or did that run properly (when the reloader runs properly it puts up a small blue logo (a pinecone I think) which slowly fills with white as a progress meter. When the pinecone is entirely white the device should reset.

What you just described is 90% accurate. The only thing thats different is that the watch does not stay on the bootloader graphic indefinitely, it resets by itself after almost exactly 5 min. Just tested the reloader and the same problem still persists after the pinecone graphic.

@frazei
Copy link

frazei commented Jun 27, 2021

Hi, I have a P8b MOY-TON5 in the same status but I've followed different steps:

  • I was on ATCWatch (working except for step counter)
  • bootloader (using DaFlasher) => ATCdfu => bootloader-daflasher.zip => stuck on wasp logo with no bluetooth
  • via SWD reset run => wasp logo with "start" and bluetooth
  • P8DFU (again with DaFlasher) => micropython.zip

After that the screen is black, if I do nrf52.cpu curstatevia SWD says running.
If I do reset halt and then reset run I see the wasp logo for about a sencond and then black screen.
If I long-press the button during the wasp logo I'm back to the bootloader (P8DFU).

@daniel-thompson
Copy link
Collaborator

@evanhuang117: Can you confirm you are using a P8b smartwatch? Earlier you said you installed mcuboot which implies you have a different device.

@daniel-thompson
Copy link
Collaborator

So far, I have tried the following:

* removing `boot.py` from the manifest
* building and flashing both the latest release as well as the master branch tip.
* Reflashing the bootloader using `reload.zip`
* Commenting out StepCounter from register_defaults
* Wait for the battery to discharge

All of the above did not change the aforementioned pattern. Any ideas?

I think we should assume that the watchdog it not being fed. There could be two reasons for this:

  1. We crash before we reach a point where we feed it.
  2. The firmware (incorrectly) thinks the button is pressed and this suppresses the feeding of the dog (this is how long press reset is implemented).

Removing boot.py from the manifest is a very good idea since that greatly simplifies what is happening. Basically micropython should initialize itself and then enter a loop waiting for characters:
https://github.com/daniel-thompson/micropython/blob/048bb2da83d0cbe73a829e461bd4f22b86c282bc/ports/nrf/drivers/bluetooth/ble_uart.c#L110

I guess the task is to figure out whether (without boot.py in the manifest) if we reach this bit of code. Perhaps you could try pulsing the motor from this bit of code or something like that (try pulsing it early in main first to make sure the motor pulsing code it OK.

Of course if we have a reproduction on an unsealed system with SWD then we could try to get the system halted during those five seconds before it resets and try to read out a backtrace would give us further clues.

@evanqhuang
Copy link

evanqhuang commented Jun 28, 2021

@evanhuang117: Can you confirm you are using a P8b smartwatch? Earlier you said you installed mcuboot which implies you have a different device.

If I'm correct the P8b's have a magnetic charger? The Wasp bootloader is the closest I've ever had to a working bootloader, all others have not turned on at all.

The builds in this issue for Infinitime are supposed to work for the P8 but all show a black screen. Currently testing removing boot.py, will update later.

Update: I reflashed with GB({'t': 'notify'}) before and after wasp.system.schedule() in main.py in addition to removing boot.py from the manifest and nothing happened, same behavior as before. Not sure if it's my code or a problem with the watch.
BTW, my P8 is connected with SWD with OpenOCD. I tried to halt during the 5 secs and it just accepts the command without any output then resets anyways. If I halt while it's just sitting on the bootloader graphic, it halts then resets 5 seconds later.

> halt
target halted due to debug-request, current mode: Thread 
xPSR: 0xa1000000 pc: 0x0007b91a msp: 0x2000ff98
nrf52.cpu: external reset detected

@arvchristos
Copy link
Author

Thanks for the comments although I think there might have been a bit of a misunderstanding here. The original reporter is describing a problem using the P8b rather than the PineTime. AFAIK neither Infinitime nor mcuboot have been ported to the P8b.

Regarding the original reports, 5 seconds is the watchdog timeout so it sounds like watch is stuck somewhere during boot and it unable to feed the dog. Of course the big question is why and I'm afraid that is not clear since I've never seen anything like this before.

If I understood the report correclty the bootloader stays on the boot graphic (the wasp-os logo) indefinitely until you don't press the button? And that pressing the button makes the screen go black and then five seconds later there is a reset?

Did the same thing happen with the reloader.zip or did that run properly (when the reloader runs properly it puts up a small blue logo (a pinecone I think) which slowly fills with white as a progress meter. When the pinecone is entirely white the device should reset.

Hello, every now and then, the watch reboots from the bootloader and follows the same pattern without pressing the button.

Pressing the button makes the screen go black, then immediatelly the wasp logo appears for a short time, then the screen goes black for 5 seconds and then the wasp logo appears with the bluetooth icon.

reloader.zip worked properly. The white pinecone icon appeared and then started filling as a progress meter. When it became entirely white, the device did reset and then reentered the bootloader.

@Pittconnect
Copy link

@arvchristos Just a note to remember https://www.colmi.com/pages/contact-us do not make / manufacture the hardware like Pine64 design and make their own PineTime watch and it has been common practice among Chinese they often switch vendors and part base on price and availability So if there is no guarantee on consistent behavior response from the P8 watch. However given price and performance of P8 you may get lucky and it all works.

@arvchristos
Copy link
Author

So far, I have tried the following:

* removing `boot.py` from the manifest
* building and flashing both the latest release as well as the master branch tip.
* Reflashing the bootloader using `reload.zip`
* Commenting out StepCounter from register_defaults
* Wait for the battery to discharge

All of the above did not change the aforementioned pattern. Any ideas?

I think we should assume that the watchdog it not being fed. There could be two reasons for this:

  1. We crash before we reach a point where we feed it.
  2. The firmware (incorrectly) thinks the button is pressed and this suppresses the feeding of the dog (this is how long press reset is implemented).

Removing boot.py from the manifest is a very good idea since that greatly simplifies what is happening. Basically micropython should initialize itself and then enter a loop waiting for characters:
https://github.com/daniel-thompson/micropython/blob/048bb2da83d0cbe73a829e461bd4f22b86c282bc/ports/nrf/drivers/bluetooth/ble_uart.c#L110

I guess the task is to figure out whether (without boot.py in the manifest) if we reach this bit of code. Perhaps you could try pulsing the motor from this bit of code or something like that (try pulsing it early in main first to make sure the motor pulsing code it OK.

Of course if we have a reproduction on an unsealed system with SWD then we could try to get the system halted during those five seconds before it resets and try to read out a backtrace would give us further clues.

I tried triggering a motor pulse in main.py and nothing happened so I guess there should be an issue with the device. I have a sealed device so no way to use SWD at the moment.

@fnordsh
Copy link

fnordsh commented Oct 13, 2021

I'm having the same issue. Has anyone had success in correctly booting WaspOS on a sealed P8b?

@fanoush
Copy link

fanoush commented Oct 14, 2021

One tip, the P8 with TON firmware does not have 32kHz crystal soldered on board, if micropython is built with softdevice initialization that requires crystal, it may do this. Previous P8 versions had this component soldered. bootloader may be built without this => it can still work.

@fanoush
Copy link

fanoush commented Oct 14, 2021

it is this line https://github.com/daniel-thompson/micropython/blob/e0cb9de8a7e788a533bd91fa3710ddb4b44ff1fb/ports/nrf/drivers/bluetooth/ble_drv.c#L147
not sure if BLUETOOTH_LFCLK_RC is defined in current build

@maximj
Copy link

maximj commented Oct 16, 2021

I've tested @fanoush 's recommendation on my P8b TON5 watch, adding "CFLAGS += -DBLUETOOTH_LFCLK_RC" to https://github.com/daniel-thompson/micropython/blob/e0cb9de8a7e788a533bd91fa3710ddb4b44ff1fb/ports/nrf/boards/p8/mpconfigboard.mk and rebuilding from source did the trick for me.

@fnordsh
Copy link

fnordsh commented Oct 16, 2021

Hm, didn't work for me, unfortunately. Did you change anything else?
What's your build environment? I'm using arm-none-eabi-gcc 11.2.0 on Manjaro.

@fnordsh
Copy link

fnordsh commented Oct 16, 2021

I just built it with gcc-8 on Debian, and it works! :-)

@evanqhuang
Copy link

I've tested @fanoush 's recommendation on my P8b TON5 watch, adding "CFLAGS += -DBLUETOOTH_LFCLK_RC" to https://github.com/daniel-thompson/micropython/blob/e0cb9de8a7e788a533bd91fa3710ddb4b44ff1fb/ports/nrf/boards/p8/mpconfigboard.mk and rebuilding from source did the trick for me.

Just tested this and it worked, thank you for figuring this out!

@Elenedeath
Copy link

Elenedeath commented Jan 6, 2022

For people, dont want to compile wasp-os specifically for the P8b MOY-TON5 with the CFLAGS += -DBLUETOOTH_LFCLK_RC

if you want to know what i modify to compile it. all my work are here https://github.com/Elenedeath/wasp-os/tree/P8b

this is my zip file based on source for wasp-os 0.4.1
i try with the master but get issue with .text' will not fit in region FLASH_TEXT'.

i try to flash bootloader via daflasher or OpenOCD and all method work.
bootloader-daflasher.zip
micropython.zip
reloader.zip

this is the bootloader.hex for flashing via OpenOCD but github dont accept hex file directly
bootloader.hex.zip

stelzch added a commit to stelzch/wasp-os that referenced this issue Jan 30, 2022
@sauresahne
Copy link

I got a P8b MOY-TON5, but I' m hesitant to change the bootloader.
Daniel-Thompson has a warning with his youtube video not to flash the P8b MOY-TON5. The link points to this thread.
Unfortunatly this warning is not included in the wasp-os github documentation and install instructions, and the latest binary predates the fix by fanoush. I think either the documentation / release notes should get a warning or the release an update.
Also - is #343 an issue with TON5?

@tomchiverton
Copy link
Contributor

So is tge current situation that P8b needs a different compiler flag? We should add a new board type and update the install docs if so?

@daniel-thompson
Copy link
Collaborator

From the thread it looks more like we could set the option and both P8a and P8b would work. Currently wasp-os is looking for a wider pool of contributors/maintainers so the current status of the issue is "pull requests welcome" (whether that is documentation changes or code changes).

To be honest code review is a little haphazard at the moment too. Volunteers are also welcome there!

@tomchiverton
Copy link
Contributor

tomchiverton commented Jul 9, 2023

Well, I'll have a P8 (b, seems to have a magnetic charger) as soon as EBay deliver me one so will be able to confirm the build flag change, and also if master flashes OK or not (sounds like doesn't at the moment).
I really want to contribute some watch faces...

@tomchiverton
Copy link
Contributor

My P8b is here. Using DaFlasher and the files above, it seems stuck on a screen with a black pine cone and blue Bluetooth icon. Uploading micropython.zip , the last thing it says is please wait, this might take up to five minutes.

Progress bar at the top of DaFlasher does not move as when uploading other files.

@tomchiverton
Copy link
Contributor

The watch intermittently restarts and DaFlasher says "Starting flashing..." etc again

@tomchiverton
Copy link
Contributor

The ota-dfu tool from the v4.0.1 release zip, using the .zip from above, does

$ tools/ota-dfu/dfu.py -z micropython.zip -a F3:4C:32:67:B6:5F --legacy
Sending file firmware.bin to F3:4C:32:67:B6:5F
Binary imge size: 299580
Binary CRC32: 169888689
Connecting to F3:4C:32:67:B6:5F
Waiting for Image Size notification
Waiting for INIT DFU notification
Exception at line 166: a bytes-like object is required, not 'str'

@tomchiverton
Copy link
Contributor

Using the v4.0.1 binary release, all is fine.

$ tools/ota-dfu/dfu.py -z build-p8/micropython.zip -a F3:4C:32:67:B6:5F --legacy and I get a progress bar.

Upload complete in 7 minutes and 32 seconds
Waiting for DFU complete notification
Waiting for Firmware Validation notification
Activate and reset

But this just returns me to the pine cone bootloader :(

@tomchiverton
Copy link
Contributor

But I then tried the micropython.zip from above with dfu.py again, and it's given the progress bar (!)

I think this "str" message is just something that happens and the trick is to try again. Dunno what's up with DaFlasher tho.

Watch has now booted OK, but the display is mirrored/inverted ( #343 ) and i can't build wasp-os ( #437 ) - @Elenedeath don't suppose you could apply that one line patch and recompile ?

@fgaz
Copy link
Member

fgaz commented Jul 13, 2023

I think this "str" message is just something that happens and the trick is to try again.

Yes, depending on the device sometimes you have to retry many times

@tomchiverton
Copy link
Contributor

Worth me putting a PR into the docs to say that ?

@fgaz
Copy link
Member

fgaz commented Jul 19, 2023

Yes please! This is not the first time people ask about this

tomchiverton added a commit to tomchiverton/wasp-os that referenced this issue Jul 29, 2023
Add note per wasp-os#222

signed-off-by: tom+spam@falkensweb.com
@thiswillbeyourgithub
Copy link
Contributor

thiswillbeyourgithub commented Sep 16, 2023

I managed to track down from where the error 'Exception at line 163: a bytes-like object is required, not 'str'' is coming from.

It's from the file nrf_ble_dfu_controller.py :

                if '[   ]' in string:  # <- string is bytes
                    print('Connection lost! ')
                    raise Exception('Connection Lost')

The test fails because '[ ]' is of type 'str' and string is of type bytes. You have to use b'[ ]' to fix the test.

I don't know why the reported line number is off though.

Also, remeber to change the verbose=False at the top of the tools file to troubleshoot.

@thiswillbeyourgithub
Copy link
Contributor

I think I made more progress:
In the same file, you can set a logfile to pexpect.spawn. It has to accept bytes so I did a hackish workaround:

f = sys.stdout
f.write_backup = f.write
f.write = lambda x: f.write_backup(str(x))

then add logfile=f as argument to the two pexpect.spawn call.

This way you see in real time wha's happening and you can see that:

[KNotification handle = 0x0012 value: 10 02 01 \r\n\x1b[0;94m[REDACTED]\x1b[0m[LE]> 'Waiting for INIT DFU notification

The notification handle is actually sent just before the code starts to wait for it.

@thiswillbeyourgithub
Copy link
Contributor

thiswillbeyourgithub commented Sep 16, 2023

Finally fixed it:

My solution was to forcibly add a logfile to the pexpect.spawn. This way if the notification waiter timed out : just check backwards in the log if the notification was received.

Here's the relevant function from nrf_ble_dfu_controller.py:

    def _dfu_wait_for_notify(self):
        while True:
            if verbose: print("dfu_wait_for_notify")

            if not self.ble_conn.isalive():
                print("connection not alive")
                return None

            before = self.ble_conn.after
            try:
                index = self.ble_conn.expect('Notification handle = .*? \r\n', timeout=30)

            except pexpect.TIMEOUT:
                #
                # The gatttool does not report link-lost directly.
                # The only way found to detect it is monitoring the prompt '[CON]'
                # and if it goes to '[   ]' this indicates the connection has
                # been broken.
                # In order to get a updated prompt string, issue an empty
                # sendline('').  If it contains the '[   ]' string, then
                # raise an exception. Otherwise, if not a link-lost condition,
                # continue to wait.
                #
                self.ble_conn.sendline('')
                string = self.ble_conn.before
                if b'[   ]' in string:
                    print('Connection lost! ')
                    raise Exception('Connection Lost')
                else:
                    # the notification might have been sent a bit too
                    # early, reading the log to make sure:
                    with open(verboselogger.name, "rb") as f:
                        content = f.read()
                    if not b"Notification handle = " in content:
                        return None
                    else:
                        while not before.startswith(b"Notification handle ="):
                            before = before[1:]
                        hxstr = before.split()[3:]
                        handle = int(float.fromhex(hxstr[0].decode('UTF-8')))
                        return hxstr[2:]


            if index == 0:
                after = self.ble_conn.after
                hxstr = after.split()[3:]
                handle = int(float.fromhex(hxstr[0].decode('UTF-8')))
                return hxstr[2:]

            else:
                print("unexpeced index: {0}".format(index))
                return None

The spawn lines (there are two of them):

        self.ble_conn = pexpect.spawn("gatttool -b '%s' -t random --interactive" % target_mac, logfile=verboselogger)

And the logger at the top:

import tempfile
verboselogger = tempfile.NamedTemporaryFile("wb")

@thiswillbeyourgithub
Copy link
Contributor

Is there somewhere I could make a PR? Anyway I think this can be closed.

@tomchiverton
Copy link
Contributor

tomchiverton commented Sep 16, 2023 via email

@thiswillbeyourgithub
Copy link
Contributor

Oh I just noticed that the ota-dfu dir is a fork from @daniel-thompson so I'll make a PR this week

@daniel-thompson
Copy link
Collaborator

Nice fix!

PR should go to https://github.com/wasp-os/ota-dfu-python .

@thiswillbeyourgithub
Copy link
Contributor

I made the PR. I have not really tested it because I'm not familiar with exactly how it's loaded by the wasp-os code so be warned.

@BeatLink
Copy link

BeatLink commented Nov 4, 2023

Here's the link to the repo with the working code for those coming from google. Clone it and use as normal

https://github.com/thiswillbeyourgithub/ota-dfu-python/tree/bugfixes

Fantastic work @thiswillbeyourgithub

@tolidano
Copy link

tolidano commented Jul 7, 2024

I already sent a daflasher.zip that was from either the release 0.4.1 or master, I cannot remember which. I have the P8b device mentioned here. How do I send a new bootloader-daflasher.zip? Do I have to? When I try, I get invalid state. I am trying to send the micropython from this issue now which is transferring, will know shortly if this is enough.

@tolidano
Copy link

tolidano commented Jul 7, 2024

Success! Well, almost. Just flashing the micropython from here was enough. But! The screen is backwards! How do I flip it around?

Looks like the fix is here #343 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests