Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build sketches without bootloader #3111

Open
NiklasFauth opened this issue Apr 3, 2017 · 19 comments

Comments

Projects
None yet
7 participants
@NiklasFauth
Copy link

commented Apr 3, 2017

Basic Infos

Is it possible to compile sketches without eboot bootloader?

Description

Hey,

I would like to use the rboot bootloader (https://github.com/raburton/rboot) to have multiple ROMs on my ESP written using Arduino. For this, I need the compiled .elf, but without the eboot bootloader so I can create my own image using esptool2. Is there any flag or anything I can comment out to accomplish this?

Thanks in advance,
Niklas

@davisonja

This comment has been minimized.

Copy link

commented Apr 3, 2017

You need to edit the build recipes in platform.text, probably specifically

recipe.objcopy.hex.pattern

Though I suspect it won't support any of the OTA updatery functionality.
You may also need to update some addresses of things (I don't know how large the rboot binary is, if it's more than a single page (4k) things will get more complicated and you'll need to edit the linker files.

@NiklasFauth

This comment has been minimized.

Copy link
Author

commented Apr 3, 2017

Thanks for the fast answer, the platform.txt was exactly what I was looking for!

First, I made a copy of the nodemcu board config and edited the corresponding linker script by increasing the irom0_0_seg address from 0x40201010 to 0x40202010 (rboot is a bit bigger than eboot).

I changed
## Combine gc-sections, archives, and objects recipe.c.combine.pattern="{compiler.path}{compiler.c.elf.cmd}" {compiler.c.elf.flags} {compiler.c.elf.extra_flags} -o "{build.path}/{build.project_name}.elfraw" [...]

so I get the .elfraw without the bootloader, that seems to work.

Then I use esptool2 to create my own binary:

esptool2 -bin -debug -boot2 -1024 -dio /tmp/arduino_build_236811/sketch.ino.elfraw ~/rboot_test/rom0.bin .irom0.text .text .data .rodata

I upload everything using esptool:
esptool.py --port /dev/ttyUSB0 -b 921600 write_flash --flash_mode dio --flash_size 1MB 0x0 rboot.bin 0x02000 rom0.bin 0x82000 rom1.bin

The rboot loader seems to work fine, however, the actual rom crashes:

Serial dump:
`rBoot v1.4.2 - richardaburton@gmail.com
Flash Size: 8 Mbit
Flash Mode: DIO
Flash Speed: 40 MHz

Everything up to date.
Booting rom 0.

Fatal exception (3):
epc1=0x4000438f, epc2=0x00000000, epc3=0x00000000, excvaddr=0x40202010, depc=0x00000000
`

So, any idea why this happens?

Best regards,
Niklas

@davisonja

This comment has been minimized.

Copy link

commented Apr 3, 2017

What's at 0x4000438f? :)
A map file from the linker is useful for finding things like that.

Did you follow the rboot 'Linking user code' section?
A quick glance through rboot's code suggests there shouldn't be any major barriers to doing what you're trying to.

@NiklasFauth

This comment has been minimized.

Copy link
Author

commented Apr 3, 2017

Thanks for the hint!

I generated the map using xtensa-lx106-elf-nm

40003a14 A Uart_Init
40004a4c A SPIWrite
40004b1c A SPIRead
400043c8 A SPI_read_status
4000443c A SPI_write_enable
40004400 A SPI_write_status

EDIT: I tried other sketches like the basic blink sketch, the esp always crashes at 0x4000438f :/

For now I use the addresses from the provided rboot example.
One question: Why do I need to specify the SPI mode, size and clock for creating the binary? I never had problems using other clock rates or bigger flash sizes. Actually, the board I'm developing with has 8MB of flash, even though it's not supported by esptool I can program it without any problems setting the flash size to 1MB, 2MB or 4MB. Might this be a problem?

@davisonja

This comment has been minimized.

Copy link

commented Apr 4, 2017

It's crashing in the middle of a SPIRead call, which I suspect means it hasn't left rboot, which would mean the sketch is irrelevant.
Do you have a binary from something that's not the Arduino system that works?

@devyte

This comment has been minimized.

Copy link
Collaborator

commented Apr 4, 2017

@NiklasFauth

This comment has been minimized.

Copy link
Author

commented Apr 4, 2017

So, I tried a few more things, for example I'm able to execute my "standalone" roms using the eboot bootloader, having the rom at 0x1000. So the problem must be with rboot.

I had a close look on the actual bootloading thing, both eboot and rboot are using "jx a3" for setting the instruction pointer. I noticed this line in eboot that is missing in rboot:

register uint32_t sp asm("a1") = 0x3ffffff0;

Can someone explain me what it does?

Best regards,
Niklas

@davisonja

This comment has been minimized.

Copy link

commented Apr 4, 2017

I expect (given the surrounding lines) that it's initialising the 'stack pointer' or 'sp' to be the end of RAM

@TheOriginalMrWolf

This comment has been minimized.

Copy link

commented Apr 17, 2017

Are you using the right flash mode (DIO)? Most are QIO.

@devyte

This comment has been minimized.

Copy link
Collaborator

commented Sep 8, 2017

@NiklasFauth did you get this to work? To be honest, I would love to move to rboot, it is better known and has better features vs. eboot (at least at first glance).
In addition, I fully support your effort to have multiple images flashed. The OTA process would certainly be safer, because you would always write the "other image", and then just change the boot address to point to it. Then, if for some reason the boot of the new image failed, you could fall back to the "old image". As I explained in my previous comment, I understand that the Nodemcu Lua project uses rboot successfully.
I believe that the current approach with eboot is different. During OTA, there is a fixed "emtpy" area at which the new image is flashed. Then, during boot, the new image is copied over the old normal image.
The consecuence of that is, if there is a power outage during the copy process, the ESP could get soft-bricked.

@devyte

This comment has been minimized.

Copy link
Collaborator

commented Sep 8, 2017

Reference: #905

@davisonja

This comment has been minimized.

Copy link

commented Sep 8, 2017

That's issue #905 :)

Should we be looking at switching to rboot?
It's been a while since I've looked at it, but does it some how detect that 'the boot of the new image failed'? Aside from checks that the image is correct (such as a checksum) I'm not sure how the bootloader would know...

@davisonja

This comment has been minimized.

Copy link

commented Sep 8, 2017

For reference, the solution for #905 is to only mark the update as complete when the copy process has completed successfully; until it has, each startup results in the copy being tried again.

@devyte

This comment has been minimized.

Copy link
Collaborator

commented Sep 8, 2017

@davisonja I don't know the specifics, maybe it's possible to do something similar with eboot as well.
Even so, I like rboot better. It's open source and well known.
The way I understand the Nodemcu logic is that you don't mark the update as complete until very late in the boot process. I think in our case it would be just before calling setup.
The point of the logic is to not just make sure that the update finished successfully, but to also make sure that the new image actually boots, at least up until a certain point. If the new image fails to boot, then you mark the update as failed, and roll back to the previous image.
Example: say that the new image was built wrongly (wrong layout, a code change in main, whatever). You update the ESP, the process succeeds. But then it won't boot anymore => the ESP is soft-bricked.

The Nodemcu logic is something along these lines:
The flash has 2 image slots, call them 0 and 1.
Boot from current image (say slot 0). Update slot 1 with broken image, process succeeds. Boot to image in slot1. Boot process fails (i.e.: doesn't reach the point where the image is marked "good"), fall back to reboot from image in slot 0.
=> no soft brick.

I think there are other details as well, such as if there is an immediate reboot with boot reason crash, then the new image is also discarded and the previous image is booted. I'm not sure of the specifics, it's been a long while since that discussion.

@davisonja

This comment has been minimized.

Copy link

commented Sep 8, 2017

@devyte Okay, that makes sense. As long as the new image knows to mark itself as being okay, that would work happily.

I'm happy to look at rboot as an alternative. eboot is very simple and, I think, fits nicely with the relatively simple approach of arduino (not overly complex, but functional), but it might be nice to have the option of using rboot for those who would like more complex behaviour.

@igrr

This comment has been minimized.

Copy link
Member

commented Sep 8, 2017

Integrating rboot might be easier if you work within the context of makeEspArduino or platform.io. I can imagine that shoehorning another link step into Arduino IDE build process can be done, but would be quite hacky (you somehow need to tell Arduino to compile bootloader files, but not link them into the sketch). Alternative would be to put rboot elf file into the repository, as it is done now with eboot, but one would probably loose a fair bit of rboot's configurability that way.
I would prefer to keep the option of using eboot in Arduino, but i'm not religious about this. If rboot can be integrated nicely, I would not mind.

@davisonja

This comment has been minimized.

Copy link

commented Sep 10, 2017

@milkpirate

This comment has been minimized.

Copy link

commented Jan 10, 2019

That's issue #905 :)

Should we be looking at switching to rboot?
It's been a while since I've looked at it, but does it some how detect that 'the boot of the new image failed'? Aside from checks that the image is correct (such as a checksum) I'm not sure how the bootloader would know...

Hey,

this thread is old but I would like to share my thought about rboot.

At work we are dealing with RAUC. Which is for Linux the equivalent of rboot for whatever you want to call the flash content of an ESP.

Anyway I quite like RAUC and it has nice features like failovers, bundle signitures, compatability strings and so on. It also has a mark-good command which tags the running image ...well: as good. But defining conditions to trigger this command is up to the architects/users.

Whats does a successful boot mean? That LEDs are blinking? All drivers were loaded successfully? That there's a wifi connection? That there's a internet connection? That the correct credentails for whatever are present?

This is not easy to define or check, especially not for the bootloader. rboot does its best to verify you have a working slot (to use RAUC terminology) - which is more than eboot does - and boots it, what then happens might be: hang, watchdog, reboot, backup slot - which, to my understanding, is not possible with eboot. If a slot fails, its up to programmer of that slot, not the bootloader. RAUC (and rboot - if configured correctly) try to maintain the device reachable as well as possible and this is a huge advantage over eboot.

With rboot it is possible to realize nearly all what RAUC can do - especially fallback on broken updates (experiences it quite often that an OTA bricked my ESPs and I had to reprogram them by serial).

So I would relly like to see rboot beeing available to Arduino.

@5chufti

This comment has been minimized.

Copy link

commented Jan 11, 2019

I would say - for this usecase - a boot was successfull if setup() was started.
I opt for a precompiled rboot replacing eboot so as to not change the general behaviour.
So pros can "precompile" their bells-and-whistles rboot while for the general arduino user additional "nice-to-have" rboot features could gradually be implemented ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.