Adding Chip erase and multithreading for make

[MX682X]

Ok so I hope this works.
Added mulithreading by creating extra receipies so that make will handle the parallelism itself.
Furthermore, I changed omake.bat to check for the Arduino IDE supplied make, basically allowing to build the bootloaders without installing anything else if you have the Arduino IDE installed, which I can kinda expect if you want to work with the DxCore.
This means, that you can just write (on windows) omake all_dx -j8 to build all bootloaders for all dx parts (8 is the amount of parallel jobs, can be different)

Chip erase is enabled (however I'd say it does not make anything faster) so I wonder if it should be even kept, as this is taking alot of space (at least 20 words). And we don't have any instruction to spare. So to support lower baudrates (BAUDH), we would have to replace jmp 0x200 with an rjmp .+(200-24) -_-. Or avoid erase at all. Side note: in order to check if the Flash is erased, i need to set FLMAP to 0 and can't restore it inside the bootloader, but i don't think this should matter.

Hopefully this will build ^^'

P.S.: As the Chip erase is ignored in the old bootloader, The core should be backwards compatible.

[MX682X]

Oh, one more idea: How about auto-baud support? Of course, this would require some changes in avrdude to send the correct packets (maybe even a config file option?) but this would make it easier to increase programming speeds.

P.S.: i have no idea how to add auto-baud support

[WestfW]

Um. Arduino IDE hasn't included Make in quite a long time... (not since they stopped using WinAVR as the toolchain, I believe.)

Does the parallelism work? IIRC, a lot of optiboot targets compile to the same .elf file, and then rename it depending on the actual target (not sure about optiboot_dx) relatively late in the game... Wouldn't be good to parallel make different versions of optiboot_dx.elf...

Autobaud is not always desirable. For instance, the "Arduino as ISP" programmer sketch uses 19200bps, at least partially to prevent the bootloader from looking like the programmer.

[MX682X]

Um. Arduino IDE hasn't included Make in quite a long time... (not since they stopped using WinAVR as the toolchain, I believe.)

Well, it was there in my Directory. I don't know why. (I used the indexing program Everything to look for a make.exe)

Does the parallelism work? IIRC, a lot of optiboot targets compile to the same .elf file, and then rename it depending on the actual target (not sure about optiboot_dx) relatively late in the game... Wouldn't be good to parallel make different versions of optiboot_dx.elf...

optiboot dx was calling make for each target through build_all_dx.bat so it always had to wait for the previous call to complete. However, make has support for multiple parallel jobs.
The problem with Windows is that it has a built-in anti-malware protection which slows down I/O significantly. It scans every friggin binary.
With -j4 I got to about 80% load on a 4C8T CPU.

Also, there are a lot of different configs for UART pins and Bootloader entry conditions plus different code for 128k and 64k, so we will need a couple of .elf

Autobaud is not always desirable. For instance, the "Arduino as ISP" programmer sketch uses 19200bps, at least partially to prevent the bootloader from looking like the programmer.

I haven't explained my though yet. As we need backwards compability, a starting baudrate is neccessary, there is no questioning of that. However, we could use the universal command to negotiate a baudrate (e.g. a lookup table with the usual baudrates, 10 = 115200, 15 = 230400, 17 = 250000 baud) and the bootloader sends the maximal possible value (e.g. 17 for dx), for optiboot_dx it would be 250000.
And then have a second Univerial command to request a baud-change. The new baud is set (only max is needed, so we can use a compile-time constant to load into the baud-register and the response (STK_INSYNC) is sent in the new baud rate (maybe with a delay of 1-5ms? so that avr-dude has time to change it's own baudrate). This way it should be possible to recognize failures, as in: wrong packet? timeout? continue with old baudrate.

ok, so while writing this, I realized that using auto-baud won't work, but I belive this negotiation is the best way to improve programming speeds of big devices

EDIT: Maybe even have a parity when bumping up the speed

[xxxajk]

Arduino can still use Make, via the CLI tools.
I use it all the time instead of the IDE, which I only use to update stuff, when it is important.
The build system that I put together, runs on Linux and mac, but not sure about windows because I don't use it and don't have access to anything that does. (I avoid it like the plague it is)

See here : https://github.com/xxxajk/Arduino_Make

The advantages of it is that you can do parallel builds no matter what, since each build gets a unique directory.
Also it keeps everything in one place and doesn't puke all over /tmp. This allows a full inspection of the entire build process, and since it keeps everything unless you explicitly clean it out, the builds can be faster.

Note too that the CLI tools DO accept a job thread number.

[SpenceKonde]

Yeah I'd been thinking about this while I've been running around the past couple of weeks and haven't had time to attend to this - the chip erase is lovely in theory, but it hits rocks over the backwards compatibility.

Thanks, I'll see if this builds for me!