Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide Urclock programmer #1171

Merged
merged 31 commits into from
Nov 22, 2022
Merged

Provide Urclock programmer #1171

merged 31 commits into from
Nov 22, 2022

Conversation

stefanrueger
Copy link
Collaborator

@stefanrueger stefanrueger commented Nov 7, 2022

This PR implements discussion #940.

  • The -c urclock programmer is compatible with -c arduino in the sense that it can also deal with arduino bootloaders
  • Urclock fully supports vector bootloaders at no extra code cost for the bootloader
  • Its uprotocol allows programming much shorter bootloaders, eg, this one for the ATmega328p Uno board that has a LED on pad PB5. This bootloader is only 350 bytes long and occupies 384 bytes (as opposed to optiboot's 512) yet it
    • Exports a pgm_write_page(sram, progmem) function that one can call at FLASHEND-4+1 to use flash for storage
    • Supports EEPROM reads/writes
    • Protects itself from overwriting
    • Has a Chip Erase function (which is much quicker than uploading 0xff's)
  • Urclock when compiled with libreadline supports bootloaders in terminal mode (preventing them from WDT timeout)

OK, how to test this? Grab a 16 MHz atmega328p board with an LED on Pin PB5 (Uno, ProMini), flush it with one of the following 115200 baud bootloaders and do your usual stuff. Notice the low flicker of the LED when in terminal mode? Tells you, it hasn't timed out...

Size Usage Version Features CE Hex file
256 256 u7.6 w-u-jpr atmega328p_led+b5_fr_ur_vbl.hex
350 384 u7.6 weu-jpr atmega328p_ee_led+b5_fr_ce_ur_vbl.hex
460 512 u7.6 wes-hpr atmega328p_ee_led+b5_fr_ce.hex
474 512 o8.3 --s-h-r optiboot_atmega328.hex
490 512 u7.6 weudhpr atmega328p_ee_led+b1_csb0_fr_ce_ur.hex
  • Size: Bootloader code size including small table at top end
  • Useage: How many bytes of flash are needed, ie, HW boot section or a multiple of the page size
  • Version: For example, u7.6 is an urboot version, o5.2 is an optiboot version
  • Features:
    • w urboot provides pgm_write_page(sram, flash) for the application at FLASHEND-4+1
    • e EEPROM read/write support
    • u uses urprotocol requiring avrdude -c urclock for programming
    • s uses skeleton of STK500v1 protocol; -c urclock and -c arduino both work
    • d dual boot (over-the-air programming from external SPI flash)
    • h hardware boot section: make sure fuses are set for reset to jump to boot section
    • j vector bootloader: uploaded applications need to be patched externally, eg, using avrdude -c urclock
    • p bootloader protects itself from being overwritten
    • r preserves reset flags for the application in the register R2
  • Hex file: typically MCU name, oscillator frequency (16 MHz default) and baud rate (115200 default) followed by
    • ee bootloader supports EEPROM read/write
    • led-b1 toggles an active-low LED on pin B1, + designates an active-high LED
    • csb0 for dual boot uses, in this example, pin B0 as chip select of external SPI flash memory
    • fr bootloader provides non-essential code for smoother error handing
    • ce bootloader provides a chip erase command
    • ur uses urprotocol and requires avrdude -c urclock for programming
    • vbl vector bootloader: set fuses to jump to reset, not the HW boot section

Testing. Please try your worst to break -c urclock or any of the urboot bootloaders :)

Note that the dual-boot bootloader in the last line expects an external SPI flash memory with CS on Pin B0. If it isn't there the bootloader will realise there is no flash connected and gracefully skips booting from external memory. Just be sure not to connect the signal OPEN_GATE of your crocodile enclosure to B0...B3 as the bootloader toggles the SPI lines.

Metadata. Probably a matter of taste, I personally quite like them: Use -c urclock to upload blink.hex on your board; then query

avrdude -qqc urclock -P /dev/ttyUSB0 -pm328p -xshowall
00000bd39209 2022-11-01 19.16 blink.hex 354 store 32002 meta 28 boot 384 u7.6 Oweu-jpr vector 25 (SPM_Ready) ATmega328P

If it's a bootloader that shows u in the features then you can even leave away the -pm328p bit; the urprotocol tells
AVRDUDE for which part the bootloader on the board was compiled.

avrdude -qqc urclock -P /dev/ttyUSB0 -xshowall
00000bd39209 2022-11-01 19.16 blink.hex 354 store 32002 meta 28 boot 384 u7.6 Oweu-jpr vector 25 (SPM_Ready) ATmega328P

If you specify a different part, it will complain as usual

$ avrdude -qqc urclock -P /dev/ttyUSB0 -xshowall -pm1284p
avrdude error: connected part ATmega328P differs in signature from -p ATmega1284P (override with -F or use -p ATmega328P)
avrdude error: unable to read signature data for part ATmega1284P, rc=-1
avrdude error: unable to read signature data, rc=-1

Extended options. Check the documentation, in the first instance $ man avrdude (I will update avrdude.texi later), or execute

$ avrdude -qqc urclock -P /dev/ttyUSB0 -xhelp avrdude -c
urclock extended options:
  -xshowall         Show all info for connected part and exit
  -xshowid           ... unique Urclock ID
  -xshowdate         ... last-modified date of flash application
  -xshowfilename     ... filename of last uploaded application
  -xshowapp          ... application size
  -xshowstore        ... store size
  -xshowmeta         ... metadata size
  -xshowboot         ... bootloader size
  -xshowversion      ... bootloader version and capabilities
  -xshowvbl          ... vector bootloader level, vec # and name
  -xid=<arg>        Location of Urclock ID, eg, F.12345.6
  -xtitle=<arg>     Title stored and shown in lieu of a filename
  -xbootsize=<arg>  Manual override for bootloader size
  -xvectornum=<arg>  ... for vector number
  -xeepromrw         ... for EEPROM read/write capability
  -xemulate_ce       ... for making avrdude emulate chip erase
  -xforcetrim       Upload of unchanged files, trim it if needed
  -xinitstore       Fill store with 0xff on writing to flash
  -xnofilename      Don't store filename on writing to flash
  -xnodate           ... application filename and no date either
  -xnometadata       ... metadata at all (ie, no store support)
  -xdelay=<arg>     Add delay [ms] after reset, can be negative
  -xhelp            Show this help menu and exit

Enjoy.

@mcuee mcuee added the enhancement New feature or request label Nov 7, 2022
@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

First test seems to be good, using atmega328p_16mhz_115200bps_ee_led+b5_fr_ce.hex.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock
 -P COM3 -b 115200 -p m328p -e

avrdude_pr1171: AVR device initialized and ready to accept instructions
avrdude_pr1171: device signature = 0x1e950f (probably m328p)
erasing chip

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock
 -P COM3 -b 115200 -p m328p -qq -t
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> write eeprom 0 0xaa 0x55 0xaa
>>> write eeprom 0 0xaa 0x55 0xaa
avrdude> flush
>>> flush
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  aa 55 aa ff ff ff ff ff  ff ff ff ff ff ff ff ff  |.U..............|

avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> write flash 0 0xaa 0x55 0xaa
>>> write flash 0 0xaa 0x55 0xaa
avrdude> flush
>>> flush
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  aa 55 aa ff ff ff ff ff  ff ff ff ff ff ff ff ff  |.U..............|

avrdude> erase
>>> erase
erasing chip ...
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> quit
>>> quit
avrdude>

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -x showall
0000ffffffff 0000-00-00 00.00  application 0 store 0 meta 0 boot 512 u7.6 owes-hpr vector 0 (RESET) ATmega328P

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -U .\hex\Blink.ino.standard.hex
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -x showall
0000ffffffff 2022-06-10 21.21 .\hex\Blink.ino.standard.hex 924 store 31292 meta 40 boot 512 
u7.6 owes-hpr vector 0 (RESET) ATmega328P

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

Somehow pgerase command is not shown in terminal mode.

.\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qqt
avrdude> help
>>> help
Valid commands:
  dump    : dump <memory> [<addr> <len> | <addr> ... | <addr> | ...]
  read    : alias for dump
  write   : write <memory> <addr> <data>[,] {<data>[,]}
          : write <memory> <addr> <len> <data>[,] {<data>[,]} ...
  flush   : synchronise flash & EEPROM writes with the device
  abort   : abort flash & EEPROM writes (reset the r/w cache)
  erase   : perform a chip erase
  sig     : display device signature bytes
  part    : display the current part information
  send    : send a raw command: send <b1> <b2> <b3> <b4>
  verbose : change verbosity
  quell   : set quell level for progress bars
  help    : show help message
  ?       : same as help
  quit    : quit after writing out cache for flash & EEPROM

Note that not all programmer derivatives support all commands. Flash and
EEPROM type memories are normally read and written using a cache via paged
read and write access; the cache is synchronised on quit or flush commands.
The part command displays valid memory types for use with dump and write.

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

Using atmega328p_16mhz_115200bps_ee_led+b5_fr_ce_ur_vbl.hex and it seems to be good as well. I also tried to read the fuse bits, erase the chip to see if the EEPROM is reserved, everything seems to be good so far.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -x showall
0000ffffffff 0000-00-00 00.00  application 0 store 0 meta 0 boot 384 u7.6 
Oweu-jpr vector 25 (SPM_Ready) ATmega328P
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -U .\hex\Blink.ino.standard.hex
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -x showall
0000ffffffff 2022-06-10 21.21 .\hex\Blink.ino.standard.hex 924 store 31420 meta 40 boot 384 u7.6 
Oweu-jpr vector 25 (SPM_Ready) ATmega328P
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM3
 -b 115200 -p m328p -qq -t
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  0c 94 40 3f 0c 94 6e 00  0c 94 6e 00 0c 94 6e 00  | .@? .n. .n. .n.|

avrdude> dump flash 0x1000 0x10
>>> dump flash 0x1000 0x10
1000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> write flash 0x1000 0xaa 0x55 0xaa
>>> write flash 0x1000 0xaa 0x55 0xaa
avrdude> flush
>>> flush
avrdude> dump flash 0x1000 0x10
>>> dump flash 0x1000 0x10
1000  aa 55 aa ff ff ff ff ff  ff ff ff ff ff ff ff ff  |.U..............|

avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> write eeprom 0 0x55 0xaa 0x55
>>> write eeprom 0 0x55 0xaa 0x55
avrdude> flush
>>> flush
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  55 aa 55 ff ff ff ff ff  ff ff ff ff ff ff ff ff  |U.U.............|

avrdude> quit
>>> quit
avrdude>

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

Somehow I'd like to see more complete sentence in the help file. Just a personal taste.
The functions work well.

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq
-xhelp avrdude -c
avrdude_pr1171 -c urclock extended options:
  -xshowall         Show all info for connected part and exit
  -xshowid           ... unique Urclock ID --> Show unique Urclock ID and exit
  -xshowdate         ... last-modified date of flash application --> same change as above
  -xshowfilename     ... filename of last uploaded application --> same change as above
  -xshowapp          ... application size --> same change as above
  -xshowstore        ... store size --> same change as above
  -xshowmeta         ... metadata size --> same change as above
  -xshowboot         ... bootloader size --> same change as above
  -xshowversion      ... bootloader version and capabilities --> same change as above
  -xshowvbl          ... vector bootloader level, vec # and name --> same change as above
  -xid=<arg>        Location of Urclock ID, eg, F.12345.6
  -xtitle=<arg>     Title stored and shown in lieu of a filename
  -xbootsize=<arg>  Manual override for bootloader size
  -xvectornum=<arg>  ... for vector number
  -xeepromrw         ... for EEPROM read/write capability
  -xemulate_ce       ... for making avrdude emulate chip erase
  -xforcetrim       Upload of unchanged files, trim it if needed
  -xinitstore       Fill store with 0xff on writing to flash
  -xnofilename      Don't store filename on writing to flash
  -xnodate           ... application filename and no date either -->Don't store application filename and date
  -xnometadata       ... metadata at all (ie, no store support) -->Don't store metadata at all (ie, no store support)
  -xdelay=<arg>     Add delay [ms] after reset, can be negative
  -xhelp            Show this help menu and exit

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowid
0000ffffffff

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowdate
2022-07-28 11.53

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowfilename
.\hex\blink_uno_no_bootloader.hex

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowapp
924

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowstore
31415

.\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowmeta
45

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowboot
384

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowversion
u7.6 Oweu-jpr

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qq -xshowvbl
vector 25 (SPM_Ready)

Copy link
Collaborator

@mcuee mcuee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just some minor comments.

src/avrintel.c Outdated
*
* v 1.1
* 30.08.2022
* 04.11.2022
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if this is good to change to 04-Nov-2022 as it is ambiguous and may get interpreted wrongly by US based people

src/avrintel.h Outdated
@@ -9,7 +9,7 @@
* meta-author Stefan Rueger <stefan.rueger@urclocks.com>
*
* v 1.1
* 30.08.2022
* 04.11.2022
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if this is good to change to 04-Nov-2022 as it is ambiguous and may get interpreted wrongly by US based people

@@ -549,7 +550,7 @@ static int ser_drain(const union filedescriptor *fd, int display) {
unsigned char buf;

timeout.tv_sec = 0;
timeout.tv_usec = 250000;
timeout.tv_usec = serial_drain_timeout*1000L;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is not related to this PR, or is it causing problem for this PR?

@@ -635,7 +636,7 @@ static int net_drain(const union filedescriptor *fd, int display) {
}

timeout.tv_sec = 0;
timeout.tv_usec = 250000;
timeout.tv_usec = serial_drain_timeout*1000L;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is not related to this PR, or is it causing problem for this PR?

# See https://github.com/stefanrueger/urboot
programmer
id = "urclock";
desc = "Urclock programmer for urboot bootloaders (arduino compatible)";
Copy link
Collaborator

@mcuee mcuee Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not always compatible with Arduino, right? -->
Edit: probably it is compatible with existing -c arduino bootloaders.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

@stefanrueger

It seems to still timeout after some time for the terminal mode, using atmega328p_16mhz_115200bps_ee_led+b5_fr_ce_ur_vbl.hex.

 .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qqt
avrdude> avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

@stefanrueger

The first hex file does not support EEPROM read/write but I am not so sure if the fault behavior from urclock can be made better or not.

.\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qqt
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: unable to read eeprom page at addr 0x0000
avrdude_pr1171 error: (dump) error reading eeprom address 0x00000 of part ATmega328P
                      read operation not supported on memory type eeprom
avrdude> avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
qavrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding

The optiboot hex file seems to behave better.

.\avrdude_pr1171 -C .\avrdude_pr1171.conf -c arduino -P COM5 -b 115200 -p m328p -qqt
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> write eeprom 0 0xaa 0x55 0xaa
>>> write eeprom 0 0xaa 0x55 0xaa
avrdude> flush
>>> flush
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: eeprom access error at addr 0x0000
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  aa 55 aa ff ff ff ff ff  ff ff ff ff ff ff ff ff  |.U..............|

avrdude> quit
>>> quit
avrdude> avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: programmer is not responding
avrdude_pr1171 error: eeprom access error at addr 0x0000

.\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200 -p m328p -qqt
avrdude_pr1171 warning: guessing it is optiboot 8.3 with size 512 (better use -xbootsize=<num>)
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

avrdude> quit
>>> quit
avrdude>

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

@stefanrueger
BTW, you may want to use an alternative optiboot bigboot hex file from @WestfW here if you are using the big hex file. It supports EEPROM. It does not seem to support chip erase though (erase command does nothing).

I am not so sure how to test it supports pgm_write_page(sram, flash) or not.

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c usbasp -p m328p
 -qq -U .\bigboot_328.hex && echo OK
OK

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5 -b 115200
 -p m328p -qqt
avrdude_pr1171 warning: guessing it is optiboot 8.3 with size 512 (better use -xbootsize=<num>)
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  aa 55 aa ff ff ff ff ff  ff ff ff ff ff ff ff ff  |.U..............|

avrdude> write eeprom 0 0xaa 0x55 0xaa
>>> write eeprom 0 0xaa 0x55 0xaa
avrdude> flush
>>> flush
avrdude> dump eeprom 0 0x10
>>> dump eeprom 0 0x10
0000  aa 55 aa ff ff ff ff ff  ff ff ff ff ff ff ff ff  |.U..............|

avrdude> quit
>>> quit
avrdude>

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5
 -b 115200 -p m328p -qq -e
avrdude_pr1171 warning: guessing it is optiboot 8.3 with size 512 (better use -xbootsize=<num>)

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5
 -b 115200 -p m328p -qqt
avrdude_pr1171 warning: guessing it is optiboot 8.3 with size 512 (better use -xbootsize=<num>)
avrdude>

@mcuee
Copy link
Collaborator

mcuee commented Nov 8, 2022

@stefanrueger
-c urclock has issues with the bigboot hex file whereas -c arduino is okay. It seems to corrupt the bootloader as well.

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c usbasp -p m328p
 -qq -U .\bigboot_328.hex && echo OK
OK

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c arduino -P COM5
 -b 115200 -p m328p -qq -D -U .\Blink_Uno.ino.hex && echo OK
OK

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c arduino -P COM5
 -b 115200 -p m328p -qq -D -U .\Blink_Uno.ino.hex && echo OK
OK

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c urclock -P COM5
 -b 115200 -p m328p -qq -D -U .\Blink_Uno.ino.hex
avrdude_pr1171 warning: guessing it is optiboot 8.3 with size 512 (better use -xbootsize=<num>)
avrdude_pr1171 error: protocol expects sync byte 0x14 but got 0x10 in urclock_load_baddr()
avrdude_pr1171 error: protocol expects sync byte 0x14 but got 0x10 in urclock_load_baddr()
avrdude_pr1171 error: unable to read flash page at addr 0x0000
avrdude_pr1171 error: unable to read byte at address 0x0000
avrdude_pr1171 error: read operation not supported for memory flash
avrdude_pr1171 error: unable to read all of flash memory, rc=-2
avrdude_pr1171 error: protocol expects sync byte 0x14 but got 0x10

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c arduino -P COM5
 -b 115200 -p m328p -qq -D -U .\Blink_Uno.ino.hex && echo OK
avrdude_pr1171 warning: attempt 1 of 10: not in sync: resp=0x10
avrdude_pr1171 warning: attempt 2 of 10: not in sync: resp=0x10
avrdude_pr1171 warning: attempt 3 of 10: not in sync: resp=0x10
avrdude_pr1171 warning: attempt 4 of 10: not in sync: resp=0x10

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c usbasp -p m328p
 -qq -U flash:v:.\bigboot_328.hex:i && echo OK
avrdude_pr1171 error: verification mismatch, first encountered at addr 0x7de1
               device 0x2e != input 0xcf
avrdude_pr1171 error: verification mismatch

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c usbasp -p m328p
 -qq -U .\bigboot_328.hex && echo OK
OK
C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c arduino -P COM5
 -b 115200 -p m328p -qq -D -U .\Blink_Uno.ino.hex && echo OK
OK
C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171 -C .\avrdude_pr1171.conf -c usbasp -p m328p
 -qq -U flash:v:.\bigboot_328.hex:i && echo OK
OK

bigboot_328.zip

@MCUdude
Copy link
Collaborator

MCUdude commented Nov 8, 2022

Very, very cool! I'm deeply impressed with your work, (even though my limited English vocabulary prevents me from using all the superlatives I would use if writing Norwegian). If this turns out to be as reliable as Optiboot, I might end up bundling Urboot instead of Optiboot with my Arduino cores to help mass-test Urboot. However, this can only happen once Avrdude 7.1 has been released and adopted by the Arduino devs. I'd also have to wait for the PlatformIO devs to update their Avrdude binaries.

I'll give it a try on various hardware later this evening.

I haven't read through all documentation, but it is possible to build urboot to use a hardware serial port instead of software serial?

It would also be amazing if the bootloader would support automatic baud rate selection. There exist optiboot forks out there that implement this, and it's done by measuring the 0x55 sync bytes the stk500 protocol sends before attempting to connect to the target. Even if this increased the compiled size to > 512 bytes, it would still be very convenient on chips with >=64kiB flash.

@MCUdude
Copy link
Collaborator

MCUdude commented Nov 8, 2022

I've tested urboot on an ATmega1284P, and everything works great so far! The features baked into the -x flags are really cool. I'm sure @WestfW could find this bootloader interesting.

A few questions:

  • When passing the -xshowversion flag, the bootloader returns u7.6 owes-hpr. What do the 'o' means?
  • When building urboot, what does lednop do, and what can I do with "template" bootloader contains mov rx,rx nops that can be replaced to toggle LEDs? Can the toggle LEDs be changed at runtime?
  • According to the documentation, fr means smoother error handling. What does this really mean, and what can we expect from a bootloader that's not built to support fr?

@MCUdude
Copy link
Collaborator

MCUdude commented Nov 8, 2022

Terminal mode seems to be broken in this PR. I'm getting a segmentation fault with urclock and with a pickit4:

PICkit4 terminal output
$ ./avrdude -cpickit4_isp -patmega1284p -t

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9705 (probably m1284p)
avrdude> read flash
>>> read flash 

Reading | -------------------------------------------------- | 0% 0.00 s 

avrdude error: flash cache address 0xd08c919f out of range [0, 0x6573552e]
avrdude error: (dump) error reading flash address 0x00000 of part ATmega1284P
               read operation not supported on memory type flash

avrdude> 

avrdude> 
read eeprom 0 0x100
avrdude> read eeprom 0 0x100
>>> read eeprom 0 0x100 

Reading |                                                    | 0% 0.00 s Segmentation fault: 11
Urclock terminal output
Hanss-MacBook-Pro-2:src hans$ ./avrdude -curclock -patmega1284p -t

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9705 (probably m1284p)
avrdude> read flash
>>> read flash 

Reading | -------------------------------------------------- | 0% 0.00 s 

avrdude error: flash cache address 0xd08c919f out of range [0, 0x6573552e]
avrdude error: (dump) error reading flash address 0x00000 of part ATmega1284P
               read operation not supported on memory type flash

avrdude> 

avrdude> 
read eeprom 0 0x100
avrdude> read eeprom 0 0x100
>>> read eeprom 0 0x100 

Reading |                                                    | 0% 0.00 s Segmentation fault: 11

@stefanrueger
Copy link
Collaborator Author

getting a segmentation fault

Whenever I have had a seg fault in avrdude, it was because of make not realising some files were changed (it goes by date, not checksum). Of course, it could also be some real problem but AVRDUDE rarely has these.

@MCUdude
Copy link
Collaborator

MCUdude commented Nov 8, 2022

Whenever I have had a seg fault in avrdude, it was because of make not realising some files were changed (it goes by date, not checksum). Of course, it could also be some real problem but AVRDUDE rarely has these.

You're right. I was sure I ran make clean before pulling this PR. Sorry about the noise. Terminal mode works just fine, by bad

@stefanrueger
Copy link
Collaborator Author

Terminal mode works just fine [for Macs]

Phew! Glad this is the case. @mcuee, @mariusgreuel and I spent a great deal of effort to make this work in all platforms (I think there are wrinkles for Windows, but nothing I canhelp with). See Issue #1028 started back in July...

I have shifted from make to cmake (well I have a makefile that uses cmake :)

all:
	cmake --build build_linux

install:
	cmake --build build_linux; sudo cmake --build build_linux --target install

new:
	sudo rm -rf build_linux
	cmake -D BUILD_DOC=1 -D CMAKE_BUILD_TYPE=RelWithDebInfo -D HAVE_LINUXGPIO=1 -D HAVE_LINUXSPI=1 -D HAVE_PARPORT=1 -B build_linux
	cmake --build build_linux

I still type make and make install, but this uses cmake and is much faster!

@stefanrueger
Copy link
Collaborator Author

@mcuee Thanks for testing! I am glad you found a few things that can be improved.

Somehow pgerase command is not shown in terminal mode.

True, currently the urclock/urboot combo only deals with classic parts, which traditionally don't have page erase capabilities (only chip erase). So, I haven't implemented page erase in urclock (though it could be emulated). Given the neat ellipses write in the terminal write flash 0x2400 128 0xff ... it's actually not really needed. I did implement page erase in the terminal so we are able to test this function on programmers that rely on it (PDI, UPDI).

timeout.tv_usec = serial_drain_timeout*1000L

I parameterised the previously constant drain timeout (250 ms) to be changable in the same way as serial_recv_timeout has been changable individually by each programmer. The reason is that urclock does not need that long a drain time. Using a much shorter drain time of 100 ms makes the response of --showid etc much zippier.

This change does not affect other programmers. They still use 250 ms drain timeout.

@mcuee: Great find with the terminal write eeprom problem when the bootloader does not support it!

I have just fixed this. I also noted that you tried to read fuses in urclock and got something back, but that is misleading: the 0xff returned is a pretend number (optiboot used to do this, too). I changed the behaviour to point out fuses cannot be read in the urboot bootloaders.

I will have to take a closer look at the big boot bootloader. Thanks for pointing out that there might be a different behaviour to -c arduino for this one!

Keep the fantastic testing up, @mcuee. The more problems and issues we find the merrier!

@stefanrueger
Copy link
Collaborator Author

stefanrueger commented Nov 8, 2022

Can the toggle LEDs be changed at runtime?

Template bootloaders can change the LEDs, CS and RX, TX lines at burn time (actually also the WDT timeout). I have written a sketch urloader for a ProMini that I use to burn bootloaders. The ProMini has a number of template bootloaders for various parts on it (t84, t85, m328p, m1284p, m2560, t167, ...) and I use that $2 device to burn bootloaders on my devices. Before I burn them I can set which pin should be the LED, which polarity it has, what the WDT timeout should be etc. The sketch knows about fuses and stuff, so would not forget (as I invariably do) to set the right fuses for hw-supported bootloaders and vector bootloaders. The urloader sketch is written in my own IDE, not in the Arduino IDE, so I need to port it somehow for plain avr-gcc consumption before I publish it.

@stefanrueger
Copy link
Collaborator Author

stefanrueger commented Nov 8, 2022

fr

Well, technically this means FRILLS=6 (fr) or FRILLS=7 (fr_ce) when using make

Here the gory details from the source:

 * RETSWVERS=<0|1> (return correct software version)
 *     
 * Option RETSWVERS=1 creates code to tell the avrdude or otherwise programmer truthfully the
 * current software version. I am not sure the space used for this code is worthwhile. Without it,
 * urboot pretends to be on software version X.X and hardware version X, where X is the major
 * software version (currently 7). RETSWVERS=0 saves 28 bytes. Actually, when URPROTOCOL=1,
 * this option has no effect, as the bootloader does not respond to STK_SW_MAJOR/MINOR requests.
 * 
 * EXITFE=<0|1|2> (exit on frame errors)
 *  
 * Option EXITFE=2 creates code that exits on frame errors in the serial communication. The
 * rationale is that FE's indicate a wrong bit-rate, which might be owing to an external device
 * wanting to communicate with the application. Setting EXITFE to 1 cares about frame errors in the
 * sense that the watchdog timer is only reset on bytes that have no frame error. Not resetting the
 * watchdog timer hastens the bootloader's reset to the application. Setting this option to 0
 * ignores frame errors, which ultimately saves a few bytes of code and delays the inevitable exit
 * from the bootloader that will be caused anyway through protocol errors. 
 *  
 * QEXITERR=<0|1> (quick exit on error)
 *  
 * Option QEXITERR=1 creates code so that when there is a protocol error during the bootloader
 * communication, the application is started within 16 ms as opposed to the default WDT timeout. If
 * QEXITERR is set, then other exit on error situations also make use of this code for quick exit.
 *
 * QEXITEND=<0|1> (quick exit on end)
 * 
 * Option QEXITEND=1 creates code so that when finished programming the application is started
 * within 16 ms as opposed to the default WDT timeout.

@stefanrueger
Copy link
Collaborator Author

it is possible to build urboot to use a hardware serial port instead of software serial?

Defo. Using the USART is actually the default in urboot.c if the part has an USART that is. Otherwise, it's software I/O. Now, the thing is that a 8 MHz MCU on 115200 baud has too big a baud rate error for me to be happy. Therefore I have compiled with SWIO on for this combination of 8 MHz and 115200 baud when building the 350,000 pre-compiled bootloaders.

@stefanrueger
Copy link
Collaborator Author

What do the 'o' means?

If only I knew! I must have used this flag to indicate something or another when developing. Good find. I will remove this (or figure out if it is something important).

@MCUdude
Copy link
Collaborator

MCUdude commented Nov 8, 2022

Defo. Using the USART is actually the default in urboot.c if the part has an USART that is. Otherwise, it's software I/O. Now, the thing is that a 8 MHz MCU on 115200 baud has too big a baud rate error for me to be happy. Therefore I have compiled with SWIO on for this combination of 8 MHz and 115200 baud when building the 350,000 pre-compiled bootloaders.

Cool! I looked around in the makefile, and later found UARTNUM, which is nice to know. I know 350000 pre-compiled bootloaders is alot (my Optiboot fork has only ~8500), but it's also nice to have pre-compiled binaries for other HW UARTS as well, even though this will significantly bump the number of pre-compiled bootloaders.

Another neat thing Optiboot has is a simple API for interfacing with the bootloader's SPM functionality from within the user application. I've created a C++ wrapper library around this API to make it easier for the user to store various data to flash. I'm sure Urboot could provide a similar API the library can interface with instead.

@stefanrueger
Copy link
Collaborator Author

stefanrueger commented Nov 8, 2022

interfacing with the bootloader's SPM functionality

Yes, there is a generic SPM subroutine in urboot executing the various SPM functionality depending on the argument; this to keep the code size small. All that would be needed is a further two bytes in the table at FLASHEND to rjmp to that subroutine. I decided against spending these two bytes as, frankly, I did not see a use case for this. All I could imagine the user to need is the pgm_writepage(), which also exists and which is really useful. So, I made this function explicitly available via an rjmp at FLASHEND+4-1 FLASHEND-4+1, and not the generic SPM sub.

350000 pre-compiled bootloaders is alot

The bash script that compiles them all (mkallbootloaders) makes use of all cores and it takes only some 55 minutes to compile them. What I was really blown away by is how efficient github uploaded them all git push; git realised it was "only" some 110,000 different files (b/c bootloaders with the same F_CPU/baudrate ratio have the same code, eg, 16 MHz @ 115200 is the same as 8 MHz @ 57600) and had uploaded all before I could get out of my chair to put the kettle on. I have yet to understand just how git does this falling short of a mixture of magic and telepathy.

nice to have pre-compiled binaries for other HW UARTS as well, even though this will significantly bump the number

Feel free to poke around in mkurboots, the script that generates the set of bootloaders for one particular part and suggest a PR. This script could also do with considering dual-boot bootloaders. The 1,232 pre-computed ones were generated outside mkurboots with a throw-away one-off script. It is a shame that Atmel never had a fully consistent naming scheme for their parts. It's good, but not perfect. There are exceptions to the numbering scheme and for some parts it is just hard to code all the exceptions of what is USART0, USART1 etc.

@stefanrueger
Copy link
Collaborator Author

stefanrueger commented Nov 8, 2022

If anyone of you has an ATtiny2313, I'd really be interested to know whether the bootloader prominently featuring the top spot in the table at the urboot project actually works. I only tested 5 parts (and don't think I have other parts at home).

This was referenced Nov 21, 2022
@mcuee
Copy link
Collaborator

mcuee commented Nov 21, 2022

I'll merge this PR within the next 24 hours unless there are reports of problems.

I'd like to have the following changes as well.

$ git diff
diff --git a/src/urclock.c b/src/urclock.c
index c66c616..ba081d4 100644
--- a/src/urclock.c
+++ b/src/urclock.c
@@ -2043,9 +2043,7 @@ static int urclock_open(PROGRAMMER *pgm, const char *port) {
     usleep((80+ur.delay)*1000); // Wait until board comes out of reset

   // Drain any extraneous input
-#ifndef WIN32
   serial_drain_timeout = 80;    // ms
-#endif
   serial_drain(&pgm->fd, 0);

   if(urclock_getsync(pgm) < 0)

@mcuee
Copy link
Collaborator

mcuee commented Nov 21, 2022

Whenever a vector bootloader is being compiled and there are enough bytes left, this protection is automatically built in. It is visible as a capital P in the features. So j/P is the same good protection as p/h. You also need to update avrdude to the newest version in PR #1171.

Yes I can see jP now with the latest urboot FW.

C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171v8mod_80ms_pr1185 -C .\avrdude_pr1171v8.conf
 -c urclock -P COM4 -b 7800 -p m328p -U .\Blink_Uno.ino.hex

avrdude_pr1171v8mod_80ms_pr1185: AVR device initialized and ready to accept instructions
avrdude_pr1171v8mod_80ms_pr1185: device signature = 0x1e950f (probably m328p)
avrdude_pr1171v8mod_80ms_pr1185: Note: flash memory has been specified, an erase cycle will be performed.
                                 To disable this feature, specify the -D option.
avrdude_pr1171v8mod_80ms_pr1185: erasing chip
avrdude_pr1171v8mod_80ms_pr1185: reading input file .\Blink_Uno.ino.hex for flash
                                 with 924 bytes in 1 section within [0, 0x39b]
                                 using 8 pages and 100 pad bytes
avrdude_pr1171v8mod_80ms_pr1185: preparing flash input for device bootloader
avrdude_pr1171v8mod_80ms_pr1185: writing 924 bytes flash ...

Writing | ################################################## | 100% 1.50 s

avrdude_pr1171v8mod_80ms_pr1185: 924 bytes of flash written
avrdude_pr1171v8mod_80ms_pr1185: verifying flash memory against .\Blink_Uno.ino.hex

Reading | ################################################## | 100% 1.44 s

avrdude_pr1171v8mod_80ms_pr1185: 924 bytes of flash verified

avrdude_pr1171v8mod_80ms_pr1185 done.  Thank you.

C:\work\avr\avrdude_test\avrdude_bin> echo "dump flash 0 0x10" | .\avrdude_pr1171v8mod_80ms_pr1185
 -C .\avrdude_pr1171v8.conf -c urclock -P COM4 -b 7800 -p m328p -qqt
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  3f cf 5c 00 0c 94 6e 00  0c 94 6e 00 0c 94 6e 00  |?.\. .n. .n. .n.|

avrdude>
avrdude>
C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171v8mod_80ms_pr1185 -C .\avrdude_pr1171v8.conf
 -c urclock -P COM4 -b 7800 -p m328p -qq -xshowall
0000ffffffff 2022-10-17 09.23 Blink_Uno.ino.hex 924 store 31431 meta 29 boot 384 u7.7 
weu-jPrac vector 25 (SPM_Ready) ATmega328P

@mcuee
Copy link
Collaborator

mcuee commented Nov 21, 2022

@stefanrueger
Just want to double check if the chip erase feature is correct or not. Thanks.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1171v8 -C .\avrdude_pr1171v8.conf -c urclock
 -P COM3 -p m328p -U .\hex\Blink.ino.standard.hex -qq && echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_bin> echo "dump flash 0 0x10" | .\avrdude_pr1171v8 -C .\avrdude_pr1171v8.conf
 -c urclock -P COM3 -p m328p -qqt
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  3f cf 5c 00 0c 94 6e 00  0c 94 6e 00 0c 94 6e 00  |?.\. .n. .n. .n.|

avrdude>
avrdude>

PS C:\work\avr\avrdude_test\avrdude_bin> echo "erase" | .\avrdude_pr1171v8 -C .\avrdude_pr1171v8.conf
 -c urclock -P COM3 -p m328p -qqt
avrdude> erase
>>> erase
erasing chip ...
avrdude>
avrdude>

PS C:\work\avr\avrdude_test\avrdude_bin> echo "dump flash 0 0x10" | .\avrdude_pr1171v8 -C .\avrdude_pr1171v8.conf
 -c urclock -P COM3 -p m328p -qqt
avrdude> dump flash 0 0x10
>>> dump flash 0 0x10
0000  3f cf ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |?...............|

avrdude>
avrdude>

@stefanrueger
Copy link
Collaborator Author

Just want to double check if the chip erase feature is correct or not.
avrdude> dump flash 0 0x10
0000 3f cf ff ff ff ff ff ff ff ff ff ff ff ff ff ff |?...............|

@MCUdude and @mcuee: Yes, this is correct behaviour for vector bootloaders. You may think of a vector bootloader as the code in top flash plus the rest vector at address 0. In order for vector bootloaders to work, the reset vector must point to the bootloader at virtually all times otherwise the bootloader is at risk of no longer being reached. 0xcf3f at address 0 is rjmp .-386 a jump backwards from zero to the start of the bootloader. The only exception is when a device is erased, as 0xffff works as sbrs r31,7 (skip one instruction if bit 7 in R31 is set). A reset to address 0 on an otherwise erased flash will therefore eventually run into the start of the bootloader. However, -c urclock has a number of safeguards to ensure the reset vector is maintained for vector bootloaders at all times, even after an erase:

  • When reading Page 0, -c urclock checks whether there is an r/jump to the bootloader, if not it writes one
  • When writing Page 0, -c urclock checks whether there is an r/jump to the bootloader, if not it writes one
  • When uploading anywhere to flash (eg, a data block somewhere), -c urclock ensures the reset vector jumps to the bootloader
  • After erasing the memory, -c urclock puts an rjmp to the bootloader at the reset vector

So, if you use -c urlock avrdude provides external protection for the reset vector of the vector bootloader. Hence, it is a good idea to use the shorter u-type urboot bootloaders that -c arduino cannot deal with (as opposed to the s type ones that -c arduino can deal with in principle). This way, -c arduino, which does not provide protection, cannot mess with the reset vector.

As urboot bootloaders export a pgm_write_page(sram, flash) function for the user, the reset vector is at risk of being overwritten by the user who might decide to write to Page 0. Now, the P-type urboot bootloaders have code in the bootloader pgm_write_page(sram, flash) function that always writes the rjmp opcode with a jump to the bootloader when it is asked to write Page 0. So, this is internal protection. This internal protection also works when the user overwrites the bootloader size with -xbootsize=... with a wrong value. -c urclock follows the instructions of the user but a P protected bootloader knows better and simply puts the right jump to the vector bootloader in.

TLDR; P vector bootloaders are pretty safe and robust. They ensure internally what proper -c urclock use does externally: correct pointing to the bootloader at the reset vector at all times.

@stefanrueger stefanrueger merged commit 02e02be into avrdudes:main Nov 22, 2022
@stefanrueger stefanrueger deleted the urclock branch November 22, 2022 01:14
@mcuee
Copy link
Collaborator

mcuee commented Nov 22, 2022

@stefanrueger

With regard to this commit 2e39891, I am not so sure some of the hex files are compatible with STK500v1.

    { 4096, 0, 0xc52edd05, 0xa3371f94 }, // Caterina-LilyPadUSB.hex
    { 4096, 0, 0x663b8f7e, 0x7efdda2b }, // Caterina-Robot-Control.hex
    { 4096, 0, 0x3c6387e7, 0x7e96eea2 }, // Caterina-Robot-Motor.hex
   <del> { 2048, 0, 0x1cef0d75, 0x6cfbac49 }, // LilyPadBOOT_168.hex</del>

Source codes:
https://github.com/arduino/ArduinoCore-avr/tree/master/bootloaders/caterina-LilyPadUSB
https://github.com/arduino/ArduinoCore-avr/tree/master/bootloaders/caterina-Arduino_Robot
https://github.com/arduino/ArduinoCore-avr/tree/master/bootloaders/lilypad

I believe they share the same Caterina bootloader as the following, and we actually use -c avr109 for the Arduino Leonardo.
https://github.com/arduino/ArduinoCore-avr/tree/master/bootloaders/caterina

AVR109:
https://ww1.microchip.com/downloads/en/Appnotes/doc1644.pdf
AVR910:
http://ww1.microchip.com/downloads/en/appnotes/atmel-0943-in-system-programming_applicationnote_avr910.pdf

STK500v1:
https://www.microchip.com/content/dam/mchp/documents/OTH/ApplicationNotes/ApplicationNotes/doc2525.pdf

Or maybe the bootloader is compatible with both (subset of both)?

I think they are only for AVR109/AVR910 and not compatible with STK500v1, if you look at the codes.
https://github.com/arduino/ArduinoCore-avr/blob/master/bootloaders/caterina-LilyPadUSB/Caterina.c#L531

/** Task to read in AVR910 commands from the CDC data OUT endpoint, process them, perform the required actions
 *  and send the appropriate response back to the host.
 */

Please also refer to my previous comment here.

@stefanrueger
Copy link
Collaborator Author

Thanks, @mcuee. See PR #1190

(I think the fourth bootloader is actually STK500v1, but the first three are not)

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

Thanks, @mcuee. See PR #1190

(I think the fourth bootloader is actually STK500v1, but the first three are not)

You are right. The relevant change in PR #1190 is correct.

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

@stefanrueger

Sorry I find another issue with Optiboot FW with the Nano clone. 80ms is too short for the board if using -c urclock. It works with -c arduino (250ms).

So the urboot autobaud FW works with 80ms but not 250ms. Optiboot FW works with 250ms but not 80ms. It seems to me 150ms may be a good compromise for Windows.

Sorry for the late findings.
Therefore the following commit is not totally correct.
f9aea24

We need something like:

#ifndef WIN32
  serial_drain_timeout = 80;    // ms
#else 
  serial_drain_timeout = 150;    // ms
#endif

The other possibility is not to change, as we already ask the user to add -xdelay=200 which works well in this case. Still I think it is good to change as the Nano Clones seem to be popular (with either ATmega328P or ATmega168P).

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

Testing results with the default Arduino optinoot_atmega328p.hex on a typical Arduino Clone (using ATmega16U2 as USB to Serial chip) and I can see the current default timeout value of 80ms is not good for Windows.

I need the following patch to get it working. This is the same for the Nano clone (ATmega328P, 16MHz).

PS C:\work\avr\avrdude_test\avrdude_main> git diff
diff --git a/src/urclock.c b/src/urclock.c
index 7bfb2d7..f58a2d7 100644
--- a/src/urclock.c
+++ b/src/urclock.c
@@ -2154,7 +2154,11 @@ static int urclock_open(PROGRAMMER *pgm, const char *port) {
     usleep((80+ur.delay)*1000); // Wait until board comes out of reset

   // Drain any extraneous input
+#ifndef WIN32
   serial_drain_timeout = 80;    // ms
+#else
+  serial_drain_timeout = 150;    // ms
+#endif
   serial_drain(&pgm->fd, 0);

   if(urclock_getsync(pgm) < 0)

Testing results:

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c usbasp -p m328p -U .\optiboot_atmega328.hex -qq
 && echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM3 -p m328p -qq -t
avrdude warning: attempt 1 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 2 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 3 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 4 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 5 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 6 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 7 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 8 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 9 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 10 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 11 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 12 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 13 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 14 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 15 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 16 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 17 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 18 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 19 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 20 of 20: not in sync
avrdude error: unable to open programmer urclock on port COM3

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM3 -p m328p -qq -t -xdelay=100
avrdude> quit
>>> quit
avrdude>
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c arduino -P COM3 -p m328p -qq -t
avrdude> quit
>>> quit
avrdude> avrdude error: programmer is not responding (Note: this is normal for `-c ardujno` because of time out)

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM3 -p m328p -xshowall

avrdude_150ms: AVR device initialized and ready to accept instructions
0 0000-00-00 00.00  application 0 store 0 meta 0 boot 512 o4.4 --s-h-r-- vector 0 (RESET) ATmega328P

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

Nano ATmega328P (5V, 16MHz).

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c usbasp -p m328p -U .\optiboot_atmega328.hex
 -qq && echo OK
OK
PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM10 -p m328p -xshowall
avrdude warning: attempt 1 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 2 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 3 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 4 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 5 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 6 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 7 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 8 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 9 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 10 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 11 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 12 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 13 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 14 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 15 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 16 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 17 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 18 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 19 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 20 of 20: not in sync
avrdude error: unable to open programmer urclock on port COM10

avrdude done.  Thank you.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c urclock -P COM10 -p m328p -xshowall -xdelay=100

avrdude: AVR device initialized and ready to accept instructions
0 0000-00-00 00.00  application 0 store 0 meta 0 boot 512 o4.4 --s-h-r-- vector 0 (RESET) ATmega328P

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM10 -p m328p -xshowall

avrdude_150ms: AVR device initialized and ready to accept instructions
0 0000-00-00 00.00  application 0 store 0 meta 0 boot 512 o4.4 --s-h-r-- vector 0 (RESET) ATmega328P

150ms is okay with urboot autobaud FW as well.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c usbasp -p m328p 
-U ..\..\urboot\bootloaders\atmega328p\autobaud\atmega328p_autobaud_ee_led-b1_fr_ce_ur_vbl.hex -qq 
&& echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM10 -p m328p -xshowall

avrdude_150ms: AVR device initialized and ready to accept instructions
0000ffffffff 0000-00-00 00.00  application 0 store 0 meta 0 boot 384 u7.7 weu-jPrac vector 25
 (SPM_Ready) ATmega328P

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM10 -p m328p
 -U .\hex\Blink.ino.standard.hex

avrdude_150ms: AVR device initialized and ready to accept instructions
avrdude_150ms: device signature = 0x1e950f (probably m328p)
avrdude_150ms: Note: flash memory has been specified, an erase cycle will be performed.
               To disable this feature, specify the -D option.
avrdude_150ms: erasing chip
               delaying chip erase until first -U upload to flash
avrdude_150ms: reading input file .\hex\Blink.ino.standard.hex for flash
               with 924 bytes in 1 section within [0, 0x39b]
               using 8 pages and 100 pad bytes
avrdude_150ms: preparing flash input for device bootloader
avrdude_150ms: writing 924 bytes flash ...

Writing | ################################################## | 100% 6.67 s

avrdude_150ms: 924 bytes of flash written
avrdude_150ms: verifying flash memory against .\hex\Blink.ino.standard.hex

Reading | ################################################## | 100% 5.45 s

avrdude_150ms: 924 bytes of flash verified

avrdude_150ms done.  Thank you.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM10 -p m328p -qq -xshowall
0 2022-06-10 21.21 Blink.ino.standard.hex 924 store 31298 meta 34 boot 512 o4.4 --s-h-r-- vector 0
 (RESET) ATmega328P

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

@stefanrueger

There is a strange thing that the following command does not echo back ok, under Windows.

Windows Terminal

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM10 -p m328p
 -U .\hex\Blink.ino.standard.hex -qq && echo OK
PS C:\work\avr\avrdude_test\avrdude_bin>

MSYS2

MINGW64 /c/work/avr/avrdude_test/avrdude_bin
$ ./avrdude_150ms -c urclock -P COM10 -p m328p -U ./hex/Blink.ino.standard.hex -qq && echo OK

MINGW64 /c/work/avr/avrdude_test/avrdude_bin

Windows command prompt:

c:\work\avr\avrdude_test\avrdude_bin>.\avrdude_150ms -c urclock -P COM10 -p m328p
 -U .\hex\Blink.ino.standard.hex -qq && echo OK

c:\work\avr\avrdude_test\avrdude_bin>

This does not happen with -c arduino. It only happens with -c urclock.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude -c usbasp -p m328p -U .\optiboot_atmega328.hex
 -qq && echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c arduino -P COM10 -p m328p
 -U .\hex\Blink.ino.standard.hex -qq && echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_150ms -c urclock -P COM10 -p m328p
 -U .\hex\Blink.ino.standard.hex -qq && echo OK
PS C:\work\avr\avrdude_test\avrdude_bin>

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

@stefanrueger

Even under Linux, 80ms does not seem to be enough for the optiboot_atmega328.hex file. Even adding -xdelay=200 does not help.

This happens with both the Uno ATmega16U2 clone and the Nano CH340 clone.

Unfortunately changing to 150ms timeout does not help either. Strange.

mcuee@UbuntuSwift3:~/build/avr/avrdude_bin$ avrdude -c urclock -P /dev/ttyUSB0 -p m328p
avrdude warning: attempt 1 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 2 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 3 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 4 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 5 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 6 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 7 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 8 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 9 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 10 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 11 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 12 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 13 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 14 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 15 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 16 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 17 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 18 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 19 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 20 of 20: not in sync
avrdude error: unable to open programmer urclock on port /dev/ttyUSB0

avrdude done.  Thank you.

mcuee@UbuntuSwift3:~/build/avr/avrdude_bin$ avrdude -c urclock -P /dev/ttyUSB0 -p m328p -xdelay=200

avrdude: AVR device initialized and ready to accept instructions
avrdude error: protocol expects OK byte 0x10 but got 0x95 in urclock_read_sig_bytes()
avrdude error: unable to read signature data for part ATmega328P, rc=-1
avrdude error: unable to read signature data, rc=-1
avrdude error: protocol expects sync byte 0x14 but got 0x0f

avrdude done.  Thank you.

mcuee@UbuntuSwift3:~/build/avr/avrdude_bin$ avrdude -c arduino -P /dev/ttyUSB0 -p m328p

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e950f (probably m328p)

avrdude done.  Thank you.

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

Even with urboot stk500v1 FW it does not work. This is strange.

mcuee@UbuntuSwift3:~/build/avr/avrdude_bin$ ./avrdude -c usbasp -p m328p ../urboot/bootloaders/atmega328p/autobaud/atmega328p_autobaud_ee_led+b1_fr_ce.hex -qq && echo OK
OK

mcuee@UbuntuSwift3:~/build/avr/avrdude_bin$ ./avrdude -c urclock -P /dev/ttyACM0 -p m328p -xshowall
avrdude warning: attempt 1 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 2 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 3 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 4 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 5 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 6 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 7 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 8 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 9 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 10 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 11 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 12 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 13 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 14 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 15 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 16 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 17 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 18 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 19 of 20: not in sync
avrdude warning: programmer is not responding; try, eg, -xdelay=200
avrdude warning: attempt 20 of 20: not in sync
avrdude error: unable to open programmer urclock on port /dev/ttyACM0

avrdude done.  Thank you.

mcuee@UbuntuSwift3:~/build/avr/avrdude_bin$ ./avrdude -c arduino -P /dev/ttyACM0 -p m328p

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e950f (probably m328p)

avrdude done.  Thank you.

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

@stefanrueger

Something is not so right under Linux. I will check again tomorrow to see if I have somethings not done correctly like the fuse settings or other things.

@mcuee
Copy link
Collaborator

mcuee commented Nov 23, 2022

I will create seperate ssues for the echo thingy as I can reproduce under Linux as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants