Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

urclock support for optiboot_x and optiboot_dx #1200

Closed
mcuee opened this issue Nov 27, 2022 · 20 comments · Fixed by #1208
Closed

urclock support for optiboot_x and optiboot_dx #1200

mcuee opened this issue Nov 27, 2022 · 20 comments · Fixed by #1208
Labels
enhancement New feature or request

Comments

@mcuee
Copy link
Collaborator

mcuee commented Nov 27, 2022

As or now urboot does not support AVR Mega0, Tiny01/2 and AVR DA/DB/DD yet.

What about the support of optiboot_x and optiboot_dx in -c urclock?

One of the issue is that the bootloader is on top of the flash memory. But it seems to work fine based on my quick testing.

@stefanrueger
Can I assumen it works out of the box? Or we should be cautious and not to try it as of now.

And I am wondering why the meta data are located at 0xbdc0?

$  ./avrdude -c urclock -P COM13 -p m4808 -xbootsize=512 -xshowall

avrdude: AVR device initialized and ready to accept instructions
0 0000-00-00 00.00  application 0 store 0 meta 0 boot 512 x0.0 ......... vector 0 (RESET) ATmega4808

$  ./avrdude -c urclock -P COM13 -p m4808 -D -U Blink.ino_atmega4808_16000000L.hex -xbootsize=512

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9650 (probably m4808)
avrdude: reading input file Blink.ino_atmega4808_16000000L.hex for flash
         with 758 bytes in 1 section within [0x200, 0x4f5]
         using 6 pages and 10 pad bytes
avrdude: preparing flash input for device bootloader
avrdude: writing 758 bytes flash ...

Writing | ################################################## | 100% 0.07 s

avrdude: 758 bytes of flash written
avrdude: verifying flash memory against Blink.ino_atmega4808_16000000L.hex

Reading | ################################################## | 100% 0.06 s

avrdude: 758 bytes of flash verified

avrdude done.  Thank you.

$  ./avrdude -c urclock -P COM13 -p m4808 -xbootsize=512 -xshowall

avrdude: AVR device initialized and ready to accept instructions
0 2022-11-27 19.30 Blink.ino_atmega4808_16000000L.hex 1270 store 47324 meta 46 boot 512 
x0.0 ......... vector 0 (RESET) ATmega4808

$  ./avrdude -c urclock -P COM13 -p m4808 -xbootsize=512 -t

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9650 (probably m4808)
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10

Reading | ################################################## | 100% 0.05 s

0000  01 c0 d8 c0 11 24 80 91  40 00 88 23 69 f0 28 2f  |.....$..@..#i.(/|

avrdude> dump flash 0x200 0x10
>>> dump flash 0x200 0x10

Reading | ################################################## | 100% 0.05 s

0200  47 c0 00 00 55 c0 00 00  53 c0 00 00 51 c0 00 00  |G...U...S...Q...|

avrdude> dump flash 0xbdc0 0x40
>>> dump flash 0xbdc0 0x40

Reading | ################################################## | 100% 0.18 s

bdc0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
bdd0  ff ff 42 6c 69 6e 6b 2e  69 6e 6f 5f 61 74 6d 65  |..Blink.ino_atme|
bde0  67 61 34 38 30 38 5f 31  36 30 30 30 30 30 30 4c  |ga4808_16000000L|
bdf0  2e 68 65 78 00 e6 07 0b  1b 13 1e f6 04 dc b8 23  |.hex... .......#|

avrdude> quit
>>> quit
avrdude>
avrdude done.  Thank you.
@mcuee mcuee added enhancement New feature or request question Further information is requested and removed enhancement New feature or request labels Nov 27, 2022
@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

And it sems to work with ATtiny817 XPlainedPro optiboot_x as well.

Again the question is why the meta data are location at 0x1dc0.

$  ./avrdude -c urclock -P COM23 -p t817 -xbootsize=512 -U ./Blink
Blink.ino_atmega4808_16000000L.hex           Blink_Pro817.ino.t817opti.20c0.mD0.v264.hex

$  ./avrdude -c urclock -P COM23 -p t817 -xbootsize=512 -U ./Blink_Pro817.ino.t817opti.20c0.mD0.v264.hex

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9320 (probably t817)
avrdude: Note: flash memory has been specified, an erase cycle will be performed.
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: reading input file ./Blink_Pro817.ino.t817opti.20c0.mD0.v264.hex for flash
         with 468 bytes in 1 section within [0x200, 0x3d3]
         using 8 pages and 44 pad bytes
avrdude: preparing flash input for device bootloader
avrdude: writing 468 bytes flash ...

Writing | ################################################## | 100% 0.04 s

avrdude: 468 bytes of flash written
avrdude: verifying flash memory against ./Blink_Pro817.ino.t817opti.20c0.mD0.v264.hex

Reading | ################################################## | 100% 0.03 s

avrdude: 468 bytes of flash verified

avrdude done.  Thank you.

$  ./avrdude -c urclock -P COM23 -p t817 -xbootsize=512 -xshowall

avrdude: AVR device initialized and ready to accept instructions
0 2022-11-27 21.05 ./Blink_Pro817.ino.t817opti.20c0.mD0.v264.hex 980 store 6643 meta 57 boot 512
 x0.0 ......... vector 0 (RESET) ATtiny817

$  ./avrdude -c urclock -P COM23 -p t817 -xbootsize=512 -qqt
avrdude> dump flash 0 0x40
>>> dump flash 0 0x40
0000  01 c0 d8 c0 11 24 80 91  40 00 88 23 69 f0 28 2f  |.....$..@..#i.(/|
0010  30 e0 83 fd 03 c0 2d 7f  23 2b 31 f4 80 93 40 00  |0.....-.#+1...@.|
0020  8c bb 80 e0 af d0 ec c0  a8 95 22 9a 2a 9a 10 92  |..........".*...|
0030  01 02 80 91 82 12 83 70  81 30 61 f5 8c e5 90 e0  |.......p.0a.....|

avrdude> dump flash 0x200 0x40
>>> dump flash 0x200 0x40
0200  19 c0 34 c0 33 c0 32 c0  31 c0 30 c0 2f c0 2e c0  |..4.3.2.1.0./...|
0210  2d c0 2c c0 2b c0 2a c0  29 c0 28 c0 63 c0 26 c0  |-.,.+.*.).(.c.&.|
0220  25 c0 24 c0 23 c0 22 c0  21 c0 20 c0 1f c0 1e c0  |%.$.#.".!. .....|
0230  1d c0 1c c0 11 24 1f be  cf ef cd bf df e3 de bf  |.....$..........|

avrdude> dump flash 0x1dc0 0x40
>>> dump flash 0x1dc0 0x40
1dc0  ff ff ff ff ff ff ff 2e  2f 42 6c 69 6e 6b 5f 50  |......../Blink_P|
1dd0  72 6f 38 31 37 2e 69 6e  6f 2e 74 38 31 37 6f 70  |ro817.ino.t817op|
1de0  74 69 2e 32 30 63 30 2e  6d 44 30 2e 76 32 36 34  |ti.20c0.mD0.v264|
1df0  2e 68 65 78 00 e6 07 0b  1b 15 05 d4 03 f3 19 2e  |.hex... ........|

avrdude> quit
>>> quit
avrdude>

@mcuee mcuee added enhancement New feature or request and removed question Further information is requested labels Nov 27, 2022
@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

@stefanrueger
Found one issue with optiboot_dx. Hopefully there is an esay fix. Thanks.

$  ./avrdude -c urclock -P COM6 -p avr128db48 -xbootsize=512 -qqt
avrdude error: connected AVR128DB48's number 61 of interrupts differs from -p AVR128DB48's (0); 
use correct -p ... or override with -F
avrdude error: unable to read signature data for part AVR128DB48, rc=-1
avrdude error: unable to read signature data, rc=-1

$  ./avrdude -c arduino -P COM6 -p avr128db48  -qqt
avrdude>

@stefanrueger
Copy link
Collaborator

Right now, -curclock works for urboot (and therefore parts for which urboot bootloaders exist). Currently these are 183 classic parts.

AVR Mega0, Tiny01/2 and AVR DA/DB/DD

I have yet to study the data sheets of these parts. How would AVRDUDE know the difference? Is there something in avrdude.conf that would tell it that a part is one or the other (other than parsing the name). They all have PM_UPDI set in prog_modes.

-c urclock makes the assumption that the bootloader sits in top flash. This is wrong for Dx parts as @WestfW points out:
stefanrueger/urboot#2 (comment) So, I would assume -curclock not to work for these parts.

Metadata location

-c urclock assumes

  • The vector table sits at bottom flash (address 0) for vector bootloaders (urboot never uses own vector tables)
  • The bootloader sits in top flash
  • Applications grow from bottom to top in flash
  • Flash between the application end and the bootloader is "free"

Metadata (unless switched off by -xnometadata) are put just below the bootloader and grow from top to bottom. All the space between application end and metadata start is called store and can be utilised by the pgm_write_page(sram, flash) function that urboot bootloaders export. The metadata also say where the application ends. So in both your examples, the metadata sit where they should sit.

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

@stefanrueger

Thanks for the explanations. I will label this as enhancement now.

Since -c arduino works with optiboot_x and optiboot_dx now, I was assuming that -c urclock would work out of the box other than the metadata thingy, or I need to add -xstrict and/or -xdelay=. Am I I wrong?

I think the AVR DA/DB/DD problem may be with avrdude.conf and not -c urclock.

@stefanrueger
Copy link
Collaborator

stefanrueger commented Nov 27, 2022

To cut a long short: -c urclock currently supports classic parts, and is likely to work for other parts, but unlikely to work for Dx (as for these the bootloader does not sit in top flash).

I would suggest the community try -c urboot and/or urboot bootloaders first and see how well they work and how well they are adopted. All being well, if this idea catches on, there might be more AVR-literate people than me (such as @SpenceKonde or @MCUdude) who realise "Hang on a second, if we adopt this urprotocol idea we have all of a sudden freed up 100 bytes in the bootloader that I now can play with, I'll give it a shot".

So my personal strategy for solving this avrdude issue (as well as Issue 1, Issue 2 and Issue 4 of the urboot project) is to do nothing about this right now, and let more talented people in the community play with this idea first.

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

@stefanrueger

Fair enough. I totally agree with your thoughts.

Our priority now is to get avrdude 7.1 release done by end of this year. I really hope that happens. I have long holiday in December so I can help on testing side.

I will carry out more tests with regard to this issue (also learn more about UPDI AVR parts) but do not feel pressured to do anything on new parts.

@stefanrueger
Copy link
Collaborator

problem may be with avrdude.conf and not -c urclock.

Yes, I have pushed an update to avrdude.conf.in in PR #1198 (but -c urclock still assumes bootloaders are in top flash, so Dx parts are not currently working)

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

problem may be with avrdude.conf and not -c urclock.

Yes, I have pushed an update to avrdude.conf.in in PR #1198 (but -c urclock still assumes bootloaders are in top flash, so Dx parts are not currently working)

I will test it out later.

Actually it is the same situation for Mega0 and Tiny0/1/2 (optiboot_x).

Please refer to the following run log posted before, the chip is ATtiny817 using optiboot_x from megaTinyCore. You can see that -c urclock works just fine.

0x00 - 0x200: Bootloader
0x200 onwards: application

Not exactly sure why the metadata goes to 0x1dc0 though, still since there is nothing below 0x00 so it has to go to the top of the flash. So it kind of makes sense as well.

Edit: now I understand why it goes to 0x1dc0.
Bootloader size is 0x200.
0 - 0x200 => 0x2000 - 0x200 = 0x1e00.
So meta data goes to below 0x1e00, so 0x1dc0 makes perfect sense to me now.

$  ./avrdude -c urclock -P COM23 -p t817 -xbootsize=512 -xshowall

avrdude: AVR device initialized and ready to accept instructions
0 2022-11-27 21.05 ./Blink_Pro817.ino.t817opti.20c0.mD0.v264.hex 980 store 6643 meta 57 boot 512
 x0.0 ......... vector 0 (RESET) ATtiny817

$  ./avrdude -c urclock -P COM23 -p t817 -xbootsize=512 -qqt
avrdude> dump flash 0 0x40
>>> dump flash 0 0x40
0000  01 c0 d8 c0 11 24 80 91  40 00 88 23 69 f0 28 2f  |.....$..@..#i.(/|
0010  30 e0 83 fd 03 c0 2d 7f  23 2b 31 f4 80 93 40 00  |0.....-.#+1...@.|
0020  8c bb 80 e0 af d0 ec c0  a8 95 22 9a 2a 9a 10 92  |..........".*...|
0030  01 02 80 91 82 12 83 70  81 30 61 f5 8c e5 90 e0  |.......p.0a.....|

avrdude> dump flash 0x200 0x40
>>> dump flash 0x200 0x40
0200  19 c0 34 c0 33 c0 32 c0  31 c0 30 c0 2f c0 2e c0  |..4.3.2.1.0./...|
0210  2d c0 2c c0 2b c0 2a c0  29 c0 28 c0 63 c0 26 c0  |-.,.+.*.).(.c.&.|
0220  25 c0 24 c0 23 c0 22 c0  21 c0 20 c0 1f c0 1e c0  |%.$.#.".!. .....|
0230  1d c0 1c c0 11 24 1f be  cf ef cd bf df e3 de bf  |.....$..........|

avrdude> dump flash 0x1dc0 0x40
>>> dump flash 0x1dc0 0x40
1dc0  ff ff ff ff ff ff ff 2e  2f 42 6c 69 6e 6b 5f 50  |......../Blink_P|
1dd0  72 6f 38 31 37 2e 69 6e  6f 2e 74 38 31 37 6f 70  |ro817.ino.t817op|
1de0  74 69 2e 32 30 63 30 2e  6d 44 30 2e 76 32 36 34  |ti.20c0.mD0.v264|
1df0  2e 68 65 78 00 e6 07 0b  1b 15 05 d4 03 f3 19 2e  |.hex... ........|

avrdude> quit
>>> quit
avrdude>

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

problem may be with avrdude.conf and not -c urclock.

Yes, I have pushed an update to avrdude.conf.in in PR #1198 (but -c urclock still assumes bootloaders are in top flash, so Dx parts are not currently working)

@stefanrueger

Guess what, it works now. 👍

Again the meta data location makes perfect sense to me now.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM6 -p avr128db48
 -xshowall -xbootsize=512

avrdude: AVR device initialized and ready to accept instructions
0 0000-00-00 00.00  application 0 store 0 meta 0 boot 512 x0.0 ......... vector 0 (RESET) AVR128DB48

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM6 -p avr128db48 -xbootsize=512
 -U .\Blink.ino.avr128db48opti.16c1.mB2.wO.v1410.hex
avrdude warning: attempt 1 of 10: not in sync
avrdude error: programmer is not responding; try -xstrict and/or vary -xdelay=100
avrdude warning: attempt 2 of 10: not in sync
avrdude error: programmer is not responding; try -xstrict and/or vary -xdelay=100
avrdude warning: attempt 3 of 10: not in sync
avrdude error: programmer is not responding; try -xstrict and/or vary -xdelay=100
avrdude warning: attempt 4 of 10: not in sync
avrdude error: programmer is not responding; try -xstrict and/or vary -xdelay=100
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM6 -p avr128db48 -xbootsize=512
 -U .\Blink.ino.avr128db48opti.16c1.mB2.wO.v1410.hex

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e970c (probably avr128db48)
avrdude: Note: flash memory has been specified, an erase cycle will be performed.
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: reading input file .\Blink.ino.avr128db48opti.16c1.mB2.wO.v1410.hex for flash
         with 1040 bytes in 1 section within [0x200, 0x60f]
         using 3 pages and 496 pad bytes
avrdude: preparing flash input for device bootloader
avrdude: writing 1040 bytes flash ...

Writing | ################################################## | 100% 0.25 s

avrdude: 1040 bytes of flash written
avrdude: verifying flash memory against .\Blink.ino.avr128db48opti.16c1.mB2.wO.v1410.hex

Reading | ################################################## | 100% 0.19 s

avrdude: 1040 bytes of flash verified

avrdude done.  Thank you.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM6 -p avr128db48
 -xshowall -xbootsize=512

avrdude: AVR device initialized and ready to accept instructions
0 2022-10-11 21.15 Blink.ino.avr128db48opti.16c1.mB2.wO.v1410.hex 1552 store 128946 meta 62 boot 512 
x0.0 ......... vector 0 (RESET) AVR128DB48

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM6 -p avr128db48 -xbootsize=512 -qqt
avrdude> dump 0 0x40
>>> dump flash 0 0x40
0000  11 24 80 91 40 00 81 11  05 c0 98 ed 21 e0 94 bf  |.$..@.......!...|
0010  20 93 41 00 83 fd 02 c0  82 30 21 f4 80 93 40 00  | .A......0!...@.|
0020  8c bb ee c0 8b e0 bf d0  20 9a 28 9a 10 92 e2 05  |........ .(.....|
0030  8a e8 80 93 68 08 83 e0  80 93 67 08 80 ec 80 93  |....h.....g.....|

avrdude> dump flash 0x200 0x40
>>> dump flash 0x200 0x40
0200  79 c0 00 00 87 c0 00 00  85 c0 00 00 2c c1 00 00  |y...........,...|
0210  81 c0 00 00 7f c0 00 00  7d c0 00 00 7b c0 00 00  |........}...{...|
0220  79 c0 00 00 77 c0 00 00  75 c0 00 00 73 c0 00 00  |y...w...u...s...|
0230  71 c0 00 00 6f c0 00 00  6d c0 00 00 6b c0 00 00  |q...o...m...k...|

>>> dump flash 0x1fd00
1fd00  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd10  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd20  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd30  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd40  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd50  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd60  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd70  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd80  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fd90  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fda0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fdb0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
1fdc0  ff ff 42 6c 69 6e 6b 2e  69 6e 6f 2e 61 76 72 31  |..Blink.ino.avr1|
1fdd0  32 38 64 62 34 38 6f 70  74 69 2e 31 36 63 31 2e  |28db48opti.16c1.|
1fde0  6d 42 32 2e 77 4f 2e 76  31 34 31 30 2e 68 65 78  |mB2.wO.v1410.hex|
1fdf0  00 e6 07 0a 0b 15 0f 10  06 00 00 b2 f7 01 00 2f  |...  ........../|

@stefanrueger
Copy link
Collaborator

stefanrueger commented Nov 27, 2022

OK, understood. -c urclock will currently not work with any part that has the bootloader at the bottom of flash. It assumes the bootloader is in top flash. And it assumes the vector table is at the bottom of flash at address 0. So, only classic and xmega parts.

[edit: Use -c urclock -xnometadata or] use -c arduino (which is ignorant about the bootloader and does not protect it from overwriting).

You can see that -c urclock works just fine.

Well it does in a sense, but not correctly so: -c urclock is being told the bootloader is 512 bytes long. So -c urclock fiercly protects the top 512 bytes (where it thinks the bootloader is) and puts metadata just below it (if the size of the application allows it). Clearly that is wrong. It also assumes the reset vector is at address 0 and the IRS vectors thereafter. And it writes to the reset vector and writes to the dedicated jump to the application vector.

I realise there need changes to be made to -c urclock that are not too big but I need to read the data sheets first:

  • Is the bootloader in bottom flash in all PM_UPDI parts?
  • Does the vector table follow directly after the bootloader?

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 27, 2022

@MCUdude

I think you should be able to answer the question from @stefanrueger

  • Is the bootloader in bottom flash in all PM_UPDI parts?

I believe the answer is yes. Correct me if I am wrong.

  • Does the vector table follow directly after the bootloader

I believe the answer is yes. Correct me if I am wrong.

@stefanrueger
Copy link
Collaborator

And

  • How does the compiler/linker know how big the bootloader is? [It needs to place the application thereafter and needs to resolve the symbols to absolute addresses]
  • Does avr-gcc support AVR Dx parts? I thought not.

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 28, 2022

And

  • How does the compiler/linker know how big the bootloader is? [It needs to place the application thereafter and needs to resolve the symbols to absolute addresses]

Not sure if @dl8dtl can help to answer your question or not. Or maybe @SpenceKonde can help here.

  • Does avr-gcc support AVR Dx parts? I thought not.

At least DxCore works fine to build the blink sketch for my testing.

PS C:\Users\xiaof\AppData\Local\Arduino15\packages\DxCore\tools\avr-gcc\7.3.0-atmel3.6.1-azduino4b
\avr\include\avr> ls ioavr*

    Directory: C:\Users\xiaof\AppData\Local\Arduino15\packages\DxCore\tools\avr-gcc\
7.3.0-atmel3.6.1-azduino4b\avr\include\avr

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---            6/4/2021  7:10 pm         308268 ioavr128da28.h
-a---            6/4/2021  7:10 pm         310900 ioavr128da32.h
-a---            6/4/2021  7:10 pm         354404 ioavr128da48.h
-a---            6/4/2021  7:10 pm         366896 ioavr128da64.h
-a---            6/4/2021  7:10 pm         339634 ioavr128db28.h
-a---            6/4/2021  7:10 pm         341708 ioavr128db32.h
-a---            6/4/2021  7:10 pm         392395 ioavr128db48.h
-a---            6/4/2021  7:10 pm         401810 ioavr128db64.h
-a---            6/4/2021  7:10 pm         307794 ioavr32da28.h
-a---            6/4/2021  7:10 pm         310426 ioavr32da32.h
-a---            6/4/2021  7:10 pm         353930 ioavr32da48.h
-a---            6/4/2021  7:10 pm         339428 ioavr32db28.h
-a---            6/4/2021  7:10 pm         341502 ioavr32db32.h
-a---            6/4/2021  7:10 pm         392189 ioavr32db48.h
-a---            6/4/2021  7:10 pm         308062 ioavr64da28.h
-a---            6/4/2021  7:10 pm         310694 ioavr64da32.h
-a---            6/4/2021  7:10 pm         354198 ioavr64da48.h
-a---            6/4/2021  7:10 pm         366690 ioavr64da64.h
-a---            6/4/2021  7:10 pm         339428 ioavr64db28.h
-a---            6/4/2021  7:10 pm         341502 ioavr64db32.h
-a---            6/4/2021  7:10 pm         392189 ioavr64db48.h
-a---            6/4/2021  7:10 pm         401604 ioavr64db64.h
-a---            6/4/2021  7:10 pm         304067 ioavr64dd14.h
-a---            6/4/2021  7:10 pm         304859 ioavr64dd20.h
-a---            6/4/2021  7:10 pm         305960 ioavr64dd28.h
-a---            6/4/2021  7:10 pm         306488 ioavr64dd32.h

@SpenceKonde
Copy link

SpenceKonde commented Nov 28, 2022

The optiboot_dx code is at
https://github.com/SpenceKonde/DxCore/tree/master/megaavr/bootloaders/optiboot_dx

The compiler needs to know about the bootloader's size when it compiles the application on modern AVRs, because it needs to start the app code at 0x0200 instead of 0x0000 (or whatever the right size is). This was not imo the best decision that went into the design of the AVR Dx-series.

As for how optiboot knows where the app code starts and hence where to jump to - it gets told explicitly where:

LDSECTIONS  = -Wl,-section-start=.text=0 \
        -Wl,--section-start=.application=0x200 \
        -Wl,--section-start=.spmtarg=0x1fa \
        -Wl,--section-start=.version=0x1fe

Then - I'm not entirely sure why they use this method, but I haven't had a need to change change it, even though it makes me kind of nauseous.

// Dummy application that will loop back into the bootloader if not overwritten
// This gives the bootloader somewhere to jump, and by referencing otherwise
//  unused variables/functions in the bootloader, it prevents them from being
//  omitted by the linker, with fewer mysterious link options.
void  __attribute__((section( ".application")))
      __attribute__((naked)) app();
void app()
{
// some code here that references variables and functions in the bootloader. 
// I would rather the mysterious linker options, personally... 
}

and the actual test for entry conditions and jumping to app then is simply:

  ch = RSTCTRL.RSTFR;   // get reset cause

  if (ch == 0) {
    // If we're here at the reset vector with no reset, we are in dangerous territory; the peripherals could be
    // in an arbitrary configuration. We assume everything is freshly reset, and could never make the bootloader fit if we didn't. 
    // Only safe thing to do is to reset immediately via software reset. Usually an entry condition, hence achieving the
    // fool's intent (directly entering the bootloader). This also ensures that dirty resets caused unintentionally
    // reset the application, instead of generating broken behavior. See Ref_Reset.md
    _PROTECTED_WRITE(RSTCTRL.SWRR, 1);
  }
  #if defined(ENTRYCOND_REQUIRE)
    if (ch & RSTCTRL_WDRF_bm || ((ch & ENTRYCOND_REQUIRE) == 0) ) {
  #else //No require specified, treat as all.
    // all means any reset source EXCEPT WDRF or BORF with no other flags will jump to the bootloader. 
    if (ch & RSTCTRL_WDRF_bm || ((ch & 0x35) == 0)) { 
  #endif
    // Start the app.
    // Dont bother trying to stuff it in r2, which requires heroic effort to fish out
    // we'll put it in GPIOR0 (aka GPR.GPR0) where it won't get stomped on.
    //__asm__ __volatile__ ("mov r2, %0\n" :: "r" (ch));
    RSTCTRL.RSTFR=ch; //clear the reset causes before jumping to app
    GPR.GPR0 = ch;       // but, stash the reset cause in GPIOR0 for use by app...
    // At this point WDT is ONLY on if it is fused on, in which case we can't turn it off anyway!
    // so don't waste 2-4 instructions trying.
    // watchdogConfig(WDT_PERIOD_OFF_gc);
    __asm__ __volatile__ (
      "jmp app\n"
    );
  } // end of jumping to app

Yeah avr-gcc works with modern AVRs - you just need to build the toolchain with the correct ATpacks (there is nothing magic about the AVR Dx-series as far as the compiler is concerned - the compiler doesn't provide any handling of flash mapping or anything - you have to implement anything like that that yourself. But thats a design decision microchip made, because they want to promote their XC8 compiler, which nobody else wants to come within 10 meters of because it's a closed source (hence inherently untrustworthy) toolchain. They recently added some features to support using the flash in mapped mode automatically there)
I build the toolchains (but not the executable binaries, as I get output that doesn't produce working .hex files - I stole the binaries from Arduino7 version of the toolchain, the last one they released over 2 years ago. I need to update the toolchain typically twice for each new product family they introduce; this is the latest version, same directory structure that the Arduino IDE uses, this is what gets installed when DxCore is installed through board manager.

https://spencekondetoolchains.s3.amazonaws.com/avr-gcc-7.3.0-atmel3.6.1-azduino6-aarch64-pc-linux-gnu.tar.bz2
https://spencekondetoolchains.s3.amazonaws.com/avr-gcc-7.3.0-atmel3.6.1-azduino6-arm-linux-gnueabihf.tar.bz2
https://spencekondetoolchains.s3.amazonaws.com/avr-gcc-7.3.0-atmel3.6.1-azduino6-i686-pc-linux-gnu.tar.bz2
https://spencekondetoolchains.s3.amazonaws.com/avr-gcc-7.3.0-atmel3.6.1-azduino6-x86_64-apple-darwin14.tar.bz2
https://github.com/SpenceKonde/DxCore/raw/gh-pages/avr-gcc-7.3.0-atmel3.6.1-azduino6-x86_64-pc-linux-gnu.tar.bz2 <--- hosted on github so that the traffic involved in the CI testing remains internal)
https://spencekondetoolchains.s3.amazonaws.com/avr-gcc-7.3.0-atmel3.6.1-azduino6-i686-w64-mingw32.tar.bz2

@stefanrueger
Copy link
Collaborator

The compiler needs to know about the bootloader's size when it compiles the application on modern AVRs

That's what I thought. And how does the compiler know? Which option does the user need when compiling blink.c? Linker options like -Wl,-section-start=.text=0x200 that kind of stuff?

modern AVRs

How can avrdude tell from AVRPART *part that it's a modern AVR? Is it those with UPDI i/face, ie, !!(part->prog_modes & PM_UPDI) or is it a subset of these? I suspect reset always goes to 0x0000 (ie, the bootloader if one is there).

This was not imo the best decision that went into the design of the AVR Dx-series.

So no one in the design team has asked the question And how would the compiler know the start address of the application?. I see poor design like this so often that I am convinced that a higher proportion of people in the IT industry would have failed the Sally-Anne test as children than the average population.

actual test for entry conditions and jumping to app then is simply

Simply? I don't know. Optiboot's default conditions easily lead to WDT timeout reset loops, alternating between application/bootloader. It's the reason that arduino.c needs to sleep 250 ms before it resets the board b/c otherwise two successive avrdude calls don't work; it leads to a more complicated getsync routine than necessary etc. But hey, ho each to its own.

@WestfW
Copy link

WestfW commented Nov 28, 2022

The compiler needs to know about the bootloader's size when it compiles the application on modern AVRs

I don't like it either; I believe "the bootloader should be transparent and the application shouldn't even have to know (at either compile or run time) whether there is a bootloder in use." The old AVR scheme that allowed the bootloader to be in high memory was ideal.
Unfortunately, it seems to be a lost battle. Most ARM chips also have their bootloaders at 0x0

I did come up with a scheme that lets the start address be specified in SOURCE CODE rather than linker switches. ( https://github.com/Optiboot/optiboot/blob/master/optiboot/bootloaders/optiboot/optiboot.c#L605 )
But it requires new linker scripts that use a symbol for __TEXT_REGION_START instead of hardwiring it to 0x0 (as is done in all of the current AVR linker scripts. Sigh.)

@stefanrueger
Copy link
Collaborator

a lost battle

That's OK. Who knows what optimisation this buys the hardware developers; they might actually have a reason for this which we fail to see - I simply don't know.

I did come up with a scheme that lets the start address be specified in SOURCE CODE rather than linker switches.

... of the bootloader (not the application), I assume. Mmh, yes, this is a perennial problem for bootloader writers to which the urboot project found a different solution. That project uses a wrapper urboot-gcc that figures out the size of the compiled bootloader and explicitly tells the source with -DSTART=... where it is going to sit. More optimisation potential than resolving a linker symbol. This method can and will cause the size of the bootloader to change as the compiler may use the then compile-time constant for optimisation (eg, when protecting itself from being overwritten, different compile-time constant start & end address lend themselves to different compile-time optimisation), so there might be a couple of rounds of compilation before this converges.

@mcuee
Copy link
Collaborator Author

mcuee commented Nov 28, 2022

Unfortunately, it seems to be a lost battle. Most ARM chips also have their bootloaders at 0x0

It seems to me many ARM Cortex MCUs will have factory installed bootloader (some USB AVRs have factory installed bootloader as well).

STM32: factory installed bootloader, activated by boot pins
https://www.st.com/resource/en/application_note/cd00167594-stm32-microcontroller-system-memory-boot-mode-stmicroelectronics.pdf

NXP: many have factory installed bootloader
https://www.nxp.com/design/software/development-software/mcuxpresso-software-and-tools-/mcu-bootloader-for-nxp-microcontrollers:MCUBOOT

Microchip/Atmel SAM-BA, usually reside in ROM monitor, but some need to be in flash (eg: SAM D21)
https://www.microchip.com/en-us/development-tool/SAM-BA-In-system-Programmer

@mcuee
Copy link
Collaborator Author

mcuee commented Dec 9, 2022

The compiler needs to know about the bootloader's size when it compiles the application on modern AVRs

That's what I thought. And how does the compiler know? Which option does the user need when compiling blink.c? Linker options like -Wl,-section-start=.text=0x200 that kind of stuff?

Yes, I believe that is the way for MegaCoreX, megaTinyCore and DxCore.

Example from megaTinyCore

Linking everything together...
"C:\\Users\\xiaof\\AppData\\Local\\Arduino15\\packages\\DxCore\\tools\\avr-gcc\\7.3.0-atmel3.6.1-azduino6/bin/avr-gcc" 
-Wall -Os -g -flto -fuse-linker-plugin -Wl,--gc-sections -Wl,--section-start=.text=0x200 -mmcu=attiny817 
-o "C:\\Users\\xiaof\\AppData\\Local\\Temp\\arduino_build_803015/Blink_Pro817.ino.elf" 

$ avr-readelf -S Blink_Pro817.ino.elf
There are 17 section headers, starting at offset 0x71d0:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .data             PROGBITS        00803e00 000248 000000 00  WA  0   0  1
  [ 2] .text             PROGBITS        00000200 000074 0001d4 00  AX  0   0  2
  [ 3] .bss              NOBITS          00803e00 000248 00000a 00  WA  0   0  1
  [ 4] .comment          PROGBITS        00000000 000248 000011 01  MS  0   0  1
  [ 5] .note.gnu.av[...] NOTE            00000000 00025c 00003c 00      0   0  4
  [ 6] .debug_aranges    PROGBITS        00000000 000298 000070 00      0   0  8
  [ 7] .debug_info       PROGBITS        00000000 000308 002e10 00      0   0  1
  [ 8] .debug_abbrev     PROGBITS        00000000 003118 001bf1 00      0   0  1
  [ 9] .debug_line       PROGBITS        00000000 004d09 000628 00      0   0  1
  [10] .debug_frame      PROGBITS        00000000 005334 000064 00      0   0  4
  [11] .debug_str        PROGBITS        00000000 005398 001214 00      0   0  1
  [12] .debug_loc        PROGBITS        00000000 0065ac 00017f 00      0   0  1
  [13] .debug_ranges     PROGBITS        00000000 00672b 000038 00      0   0  1
  [14] .shstrtab         STRTAB          00000000 00711a 0000b4 00      0   0  1
  [15] .symtab           SYMTAB          00000000 006764 0005b0 10     16  32  4
  [16] .strtab           STRTAB          00000000 006d14 000406 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), p (processor specific)

$ head Blink_Pro817.ino.hex
:1002000019C034C033C032C031C030C02FC02EC07E
:100210002DC02CC02BC02AC029C028C063C026C056
:1002200025C024C023C022C021C020C01FC01EC0C2
:100230001DC01CC011241FBECFEFCDBFDFE3DEBF4A
:100240008091400080934000811105C098ED21E02D
:1002500094BF209341008CBB2EE3A0E0BEE301C01D
:100260001D92AA30B207E1F777D0B2C0C9CF8FB7DD
:10027000F8944091023E5091033E6091043E70918B
:10028000053E8FBF2FB7F8948091023E9091033EB8
:10029000A091043EB091053E2FBF841B950BA60B89

@mcuee
Copy link
Collaborator Author

mcuee commented Dec 9, 2022

modern AVRs

How can avrdude tell from AVRPART *part that it's a modern AVR? Is it those with UPDI i/face, ie, !!(part->prog_modes & PM_UPDI) or is it a subset of these? I suspect reset always goes to 0x0000 (ie, the bootloader if one is there).

I believe that should be fine, basically Mega0/1 (optiboot_x), Tiny0/1/2 (optiboot_x) and AVR DA/DB/DD (optiboot_dx) as of now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants