urclock communication issues #1245
Comments
|
Ah, I do not have good testing facility to carry out test for urboot for ATtiny chips. My tests were more on ATmega328P and ATmega2560. @MCUdude has done quite some tests of urboot on ATtiny chips but he is carrying out tests on macOS. @stefanrueger |
|
I will try ATTiny88 (MH Tiny) first. Step 1 is to get optiboot working first.
Click for the run log |
|
Step 2 is to get Urboot working on the board. I have tried the pre-compiled bootloader and they do not seem to work for my board (I suspect the reason is the short WDT timeout of 1s). In the end I build one hex file which seems to work fine under Linux. Note: LED is PD0 and not PB0. |
|
And then it seems to work under Windows as well. So I can not reproduce the isse on my side. I am using a CH340 USB-TTL converter. |
|
@mariusgreuel Have you tried to set and vary Does the same board board work under Linux/MacOS? Yes, it's a pity that the windows timeout behaves in a different way to the Linux/Posix timeout. AVRDUDE use the same number of milliseconds for Windows timeout and Posix timeout but there is a different behaviour when reading bytes. I once tried to understand Unfortunately, I cannot test what that Windows timeout does to
When the AVR sends two bytes as a reply to the GET_SYNC command they will be sent within microseconds of each other. The timeout should not matter. If one byte can be read, then the other should follow. Catering for autobaud requires a bit of fiddling in the synchronisation process, which makes the synchronisation more complex. Urboot bootloaders can gobble up the first sync request from AVRDUDE in order to determine and set the baud rate. However, sometimes the first sync request is gobbled up by the board needing longer than expected to come out of reset (this is what @mcuee In this case I find the way you present the tests confusing and hard to process. This comment says "But it does not work very well" without giving an indication what does not work, followed by many lines of -v` output that seem to indicate all works well?
I suspect you picked a pre-compiled bootloader with the wrong TX and RX pins of the board? You need to compile this one. Please edit comments when they give the wrong conclusions. As the comments stay for a long time, people who come here from a search engine may be mislead... |
|
Sorry for the confusion. As mentioned, I do not have good testing facility for the ATtiny chips to run optiboot/urboot. This board is not very good for Optiboot/urboot -- it is difficult to go into the bootloader after flashing the blinking LED sketch. So the easies way is to flash the bootloader everytime for the testing. It is more meant to run Micronucleus bootloader. I have deleted two messages and edited the remaining messgaes. Please check if there are still confusions or not. Thanks. More info about the botad -- MH-Tiny ATtiny88 board (Nano formfactor). |
The Attiny88 has no "designated" communication pins as it has no USART; in that case the pre-compiled urboot bootloader assumes RX=PB0 and TX=PB1 and you'd either have to hook up comms with the host at PB0/PB1 or, if the board has an USB/serial converter, compile the bootloader to use the correct pins of the particular board. |
|
Thanks for clarifying the tests and how you got there, @mcuee. Off topic: I am also pleased to learn that urboot has been tested to work well with ATtiny88. Also good to know the MH Tiny board has |
It does not have USB Serial Converter on-board. And I did use PB0/PB1 as well. I tend to think the issue is with 1s WDT timeout. |
That will be good. I will suggest you create two versions, one with WDT=8s (for people who do not have a proper USB-TTL converter with DTR pin and not able to hook up proper reset circuitry) and the other with WDT=1s (for people who can hook-up proper reset circuitry with the USB-to-TTL converter DTR pin and 0.1uF autoreset capacitor). The good thing of using |
Ahh - OK. Understood. I looked at the differences of what is working (TX=AtmelPD6 RX=AtmelPD7) and what was not (TX=AtmelPB1 RX=AtmelPB0), so concluded that was the issue. So, the only board-specific thing is the position of the LED (PD0, not PB0 as you compiled the urboot bootloader for). What's the LED polarity? Is it low-active or high-active? | two versions, one with WDT=8s [and] 1s About time I thought about how to use the template versions of urboot bootloaders. Their binary are done in a way, so that the LED pin, LED polarity, WDTO, TX, RX can be recognised and exchanged in the binary. I have an urloader sketch for a ProMini that acts as bootloader programmer, which creates the right bootloader and correct fuses. Need to publish that. Maybe I can transfer that idea to AVRDUDE that then can patch bootloaders on the fly for these things. One of the ideas for v8.0 ... |
|
Yes you are rgth that LED pin is PD0 and it is high active. |
|
I now can reproduce the issue in Linux with the FTDI USB/serial converter. Turns out the read timeout of 10 ms is cutting it too fine for the FTDI. See PR #1246 |
|
@stefanrueger and @mariusgreuel Did you change the default latency timer of your FTDI adapter from 16ms to 1ms? That is the first thing I do after reading the above advice from Dr Azzy. @mariusgreuel |
|
@mcuee Great shout! I had not known FTDI has a default latency of 16 ms. Actually, I had not imagined that such a parameter existed, let alone is set to 16 ms per default! One never ceases to learn.... This must be the reason why the FTDI didn't work: I can confirm that setting a receive timeout of 16 ms just about works (15 ms does not), so I set it to 25 ms in PR #1246 to have a margin. I still think |
|
OK, there might still be issues with Windows around the fact that timeouts behave differently so I leave the subject of this issue and await @mariusgreuel's response to whether PR #1246 significantly changes the experienced issues. |
@stefanrueger
I agree with you here as well. Let's wait for the investigation from @mariusgreuel here. I will see if I can reproduce the issue with ATTiny167 under Windows. |
|
@mariusgreuel would like to work on #1186 as well. @stefanrueger, @MCUdude and @dl8dtl |
@stefanrueger But you can see the performance difference between 16ms latency timer and 1ms latency timer from the below testing results. Using 1ms latency timer saves 50% of the flash reading time in this test. PR #1246 is still good to go as it does not impact the performance, no regression either. |
Even though I can not reproduce the issue under WIndows, I can reproduce the issue under Linux. And PR #1246 is good to fixx the issue. But in this case, @mariusgreuel's issue may not be related to FTDI adpater 16ms latency timer. And if I change the latency timer to 1ms, then the issue goes away for git main as well. PR #1246 is still good. We can also see that Linux has a bit of better performance compared to Windows. Take note this is on the same Acer laptop (dual boot Windows 11 and Ununtu Linux 20.04) |
|
@stefanrueger
Hopefully PR #1246 may help. If not, please help to see if you can try the following things,
|
|
mariusgreuel
changed the title
|
I switched from Windows to a raspi/Ubuntu, and with the latest timing changes, I still have communication problems depending on the serial adapter I am using. For instance, here is the output of a genuine FT232H adapter: pi@raspi:~/repos/avrdude/build_linux/src $ ./avrdude -c urclock -p t167 -P /dev/ttyUSB0 -vvvv
avrdude: Version 7.0-20221229 (0f956e9)
Copyright the AVRDUDE authors;
see https://github.com/avrdudes/avrdude/blob/main/AUTHORS
System wide configuration file is /home/pi/repos/avrdude/build_linux/src/avrdude.conf
User configuration file is /home/pi/.avrduderc
User configuration file does not exist or is not a regular file, skipping
avrdude: libreadline is used; avrdude -t -c urclock should work interactively
Using Port : /dev/ttyUSB0
Using Programmer : urclock
avrdude: 245 ms: enter urclock_getsync()
avrdude: send: 0 [30] [20]
avrdude: ser_recv(): programmer is not responding
avrdude: 270 ms: sleeping for 32 ms
avrdude: send: [20] [20]
avrdude: ser_recv(): programmer is not responding
avrdude: 327 ms: sleeping for 64 ms
avrdude: send: [20] [20]
avrdude: recv: . [00] . [ea]
avrdude: send: [20] [20]
avrdude: recv: . [fc] [20]
avrdude: send: [20] [20]
avrdude: recv: . [1d] [20]
avrdude: send: [20] [20]
avrdude: recv: . [1d] [20]
avrdude: send: [20]
avrdude: recv: . [1d]
avrdude set_uP() [urclock.c:1054] warning: uP_table does not know mcuid 1306
avrdude: 459 ms: all good, ready to rock
AVR Part : ATtiny167
Chip Erase delay : 15000 us
RESET disposition : possible i/o
RETRY pulse : SCK
Serial program mode : yes
Parallel program mode : yes
Timeout : 200
StabDelay : 100
CmdexeDelay : 25
SyncLoops : 32
PollIndex : 3
PollValue : 0x53
Memory Detail :
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
eeprom 65 10 4 0 no 512 4 0 4000 4500 0xff 0xff
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
flash 65 10 64 0 yes 16384 128 128 4500 4500 0xff 0xff
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
lfuse 0 0 0 0 no 1 1 0 9000 9000 0x00 0x00
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
hfuse 0 0 0 0 no 1 1 0 9000 9000 0x00 0x00
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
efuse 0 0 0 0 no 1 1 0 9000 9000 0x00 0x00
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
lock 0 0 0 0 no 1 1 0 9000 9000 0x00 0x00
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
signature 0 0 0 0 no 3 1 0 0 0 0x00 0x00
Block Poll Page Polled
Memory Type Alias Mode Delay Size Indx Paged Size Size #Pages MinW MaxW ReadBack
----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
calibration 0 0 0 0 no 1 1 0 0 0 0x00 0x00
Programmer Type : Urclock
Description : Urclock programmer for urboot bootloaders using urprotocol
Urboot protocol for ATtiny167
avrdude: send: P [50] [20]
avrdude: ser_recv(): programmer is not responding
avrdude urclock_recv() [urclock.c:1923] error: programmer is not responding; try -xstrict and/or vary -xdelay=100
avrdude main() [main.c:1246] error: initialization failed, rc=-1
- double check the connections and try again
- use -B to set lower ISP clock frequency, e.g. -B 125kHz
- use -F to override this check
avrdude: send: Q [51] [20]
avrdude: ser_recv(): programmer is not responding
avrdude urclock_recv() [urclock.c:1923] error: programmer is not responding; try -xstrict and/or vary -xdelay=100
avrdude done. Thank you.I produced the output using a manual-reset method. The urclock clock seems to be quite sensitive on when the bootloader starts. When I release the reset button while avrdude is already running, I can create a >50% failure rate. |
I have both tried -xdelay and -xstrict, but I did not notice any differences
I have a Arduino Nano clone with a m168p and a CH340 on it and that worked fine. The t167 board I am currently resetting manually. |
Good catch. I was aware of the 16ms latency, but I did not connect the dots. I tried it on my Windows machine, and it did not really seem to help. As libftdi goes, I do not think the latency timer applies to bit-bang/MPSSE mode. There are some differences in the D2XX drivers compared to libftdi/libusb solution that I had to work around, however, it never occurred to me to change the latency timer. I will try to remember that the next time I play with that. |
As the Digispark boards were mentioned on Stefans urboot page, I just grabbed the hex files from there. Plus, as some board require tweaking the fuses, I did not want to mess up my expensive Arduino boards, and turned to the cheap clones :-).
Yes, 115k is on the edge of what a 16MHz AVR can do reliably. But then, these are the bootloaders linked from the README. BTW, I could not make the Digispark T85 clones work at all. They do not need different fuse settings, do they? Out of the box, they run at 16MHz off the PLL, I think (16.5MHz with the micronucleus bootloader active, I believe). Here, I think the 115k baud-rate combined with a PLL/RC oscillator may push it over the edge, not sure. |
This is kind of in-line with my expectation based on my testing of FT232RL based Nano Atmega328P Clone. The latency timer somehow does not cause the issue under Windows even though it causes issue under Linux before the merge of PR #1246.
I believe that is correct.
That will be great. It is not urgent so take your time to work on avrdude-libftdi. |
|
And indeed it is okay under Windows as well. I can see the LED is blinking. @mariusgreuel |
|
@mcuee Thanks for testing. Looks like the t85 board has a tricky frequency. Any way you can measure it? Like measuring the timing of a blink sketch with an osci (or another board with a halfway reliable quartz). If you use the measured real frequency as |
I have three weeks' of long vacation this December. I will try to measure the frequency next week when I am back to work. I will test 8MHz and 16MHz clock settings as well. |
Edit: hmm, later I got 115200bps working for 16MHz PLL clock; and 57600bps working with 8MHz internal RC oscillator.. |
Thanks for reporting this. @mariusgreuel I'll try to think about what happens when avrdude reads arbitrary data before the board gets into the bootloader. Maybe |
The overall comms error is compounded by sender quantisation error, sender freq error, receiver quantisation error, receiver freq error, then multiplied by 10 bits per byte. That shouldn't go beyond a quarter bit, so a 2% clock error on one side may well ruin the comms (irrespective of UART or SW I/O). So, what's the story of a t85 F_CPU? What kind of error can we expect? Even a terrible 1000 ppm ceramic resonator "only" has a clock error of 0.1% (negligible for serial errors). |
My paticular ATTiny85 board is using the internal RC oscillator. FYI, I could not get Optiboot working either. It seems to me my board may runnling at a frequency closer to 16.5MHz than the norminal 16MHz. Working configuration fuse settings: The ATTiny167 board has 16MHz external crystal. |
@stefanrueger Hmm, now it works with all the following hex files. I think you are right 115200bps can work with the board. For 8MHz internal RC oscillator, I can not get 115200bps to work, but 57600bps works fine. Working hex files are included in the zip file attached |
|
Tested under Windows as well.
|
A simple way of measuring the clock frequency is to enable the CKOUT fuse bit. Then the clock signal will be outputted on pin PB4, which can easily be measured using an oscilloscope or a frequency counter. |
Nice tip. Based on my latest test results, in the end the clock frequency is not the issue. I think my problems are mostly related to the use of manual reset. In the end I can not reproduce the issue reported by @mariusgreuel. I need to have better testing facility for urboot/optiboot. I need to find some components (10k resistors and 100nF capacitors) first and then a few USB-TTL converters with the DTR pin available so that I can have the auto-reset circuitries. As of now I only have two FTDI adapters with the DTR pin exposed. I have quite a few CH340 adapters but they do not have the exposed DTR pins. They usually have only 5V/TX/RX/GND or 5V/3V3/TX/RX/GND pins. The single PL2303HX adapter does not have the DTR pin either. I was trying to reuse two damaged Uno Clones with the ATmega328P SMD chips removed. They have the RESET circuitry and RXD/TXD pins. Somehow they do not work. I have not figured out why they do not work yet. Can you share your test setup for the ATtiny chips? |
|
@stefanrueger and @mariusgreuel I think in the end, I can not reproduce the issue under Linux/Windows in the end, other than the issue of FT232 based Nano Clone under Linux which was already addressed by PR #1246. We will have to wait for more results from @mariusgreuel. |
|
@mariusgreuel Could you try what happens when the serial interface is drained periodically (rather than at the start only): diff --git a/src/urclock.c b/src/urclock.c
index 7f32d88..e461947 100644
--- a/src/urclock.c
+++ b/src/urclock.c
@@ -1985,6 +1985,7 @@ static int urclock_getsync(const PROGRAMMER *pgm) {
if(!ur.gs.seen || iob[0] != ur.gs.stk_insync || iob[1] != ur.gs.stk_ok || iob[0] == iob[1]) {
ur.gs.stk_insync = iob[0];
ur.gs.stk_ok = iob[1];
+ serial_drain(&pgm->fd, 0); // Drain periodically to guard against initial line noise
ur.gs.seen = 1;
} else
break;
@@ -2247,9 +2248,7 @@ static int urclock_open(PROGRAMMER *pgm, const char *port) {
usleep((125+ur.delay)*1000); // Wait until board starts up accommodating effective drain time
#endif
- // Drain any extraneous input
serial_drain_timeout = 20; // ms
- serial_drain(&pgm->fd, 0);
pmsg_debug("%4ld ms: enter urclock_getsync()\n", avr_mstimestamp());
if(urclock_getsync(pgm) < 0)It is these three bytes that seem to jeopardise the connection. They might be some kind of line noise/garbage/FTDI intialisation artifact, but let's see whether above intervention gets rid of the problem.
|
@stefanrueger Im the meantime, I came to the same conclusion. I think what happens during a manual-reset communication failure is that there is one extra byte in the RX buffer that enters after you drain the buffer. So the response is [1d][20] instead of [20][1d], with one remaining RX byte not read. I fixed that by adding this line: diff --git a/src/urclock.c b/src/urclock.c
index 7f32d88e..49bc9809 100644
--- a/src/urclock.c
+++ b/src/urclock.c
@@ -1978,6 +1978,7 @@ static int urclock_getsync(const PROGRAMMER *pgm) {
* that are strict about the protocol, eg, picoboot, the presence of -xstrict implies that
* comms should use Cmnd_STK_GET_SYNC for getting in sync.
*/
+ serial_drain(&pgm->fd, 0); // Drain periodically to guard against initial line noise
iob[0] = attempt == 0? autobaud_sync: ur.strict? Cmnd_STK_GET_SYNC: Sync_CRC_EOP;
iob[1] = Sync_CRC_EOP;
urclock_send(pgm, iob, 2);With that line in place I have not seen this kind of error anymore, using the t167 board and various serial converters. BTW, when I set up the RTS auto-reset mechanism, the error disappeared also. I have two more suggestions:
I am probably going to spend some more time with urboot. I have a project where I hacked some commercial device using a m88p, and there was just not enough space to fit a bootloader. That you managed to squeeze urboot into some 250 bytes is really amazing! |
|
@mariusgreuel Thanks for testing and confirming my hunch. Could you see whether my suggestion also works? I know it sounds counter-intuitive to drop the initial drain and then to periodically drain after receiving (and evaluating!) responses rather than before every request is sent, but
Your other suggestions are great - let's see how I can implement them |
Interesting. i wonder whether the initial drain sees to that. If so it might be that the error also disappears if avrdude is started after manually releasing reset. |
|
@stefanrueger I tried your patch and it appears to work just as well. Tried that on the raspi, too. BTW: When I am varying the reset relative to the start of avrdude , I sometimes get an output like this: In this example, communication was established at attempt 9, however, as it takes two rounds to sync, it fails. This might be confusing to the user since there is 'something happening', yet it still fails. Perhaps we should just make the 'not in sync' message a debug message? |
|
@mariusgreuel Brilliant. So I'll create a PR along these lines. I'd suggest keeping the "not in sync" messages as warning messages but remove the associated "not responding" messages bar the last one to hint at things that might help: Turns out it's difficult to suppress the warning |
|
@mariusgreuel I am pretty chuffed with the changes in PR #1251; my tests show the synchronisation is now much more resilient in manual reset. Fun fact 1: I could even press reset when in terminal mode and it would catch up. So, thanks for raising the issue and for debugging. @mcuee Fun fact 2: I have just found two different versions of the digispark pro board (one of which by digistump) that have the following difference: |
|
Indeed there are different versions of DigiSpark Pro clone boards as I learned from the Internet. Actually there are different versions for the DigiSpark ATtiny85 clone boards as well but the pin assignment seems to be the same. |
@stefanrueger and @mariusgreuel #1186 is of course good, especially when avrdude official Windows binary release will use MSVC build to give the users best out-of-box user experiences with the use of avrdude-libusb and avrdude-libftdi. Hopefully @mariusgreuel will work out the solution soon. On the other hand, can we use the same trick of #1207 to differentiate terminal mode and normal mode? |
|
Just a sanity check on the latest git main (PR #1251 merged). |
|
@stefanrueger Arduino ATtinyCore Optiboot upload will have the right blinking frequency. Potential problem -- 16MHz is out of spec for external clock as per the datasheet. It is only up to 12MHz as per spec.
No change fuse settings. And the hex file is the same. |
Do you mean the blink.hex blinks differently? Well, the bootloader name misses the Do you mean blink frequency during upload? urboot bootloaders don't blink, they indicate upload activity by switching the LED on when entering the read character routine and switching it off when leaving. optiboot blinks before upload for some 300 ms (wasting code and time). |
Yes this is the exactly issue and your solution works perfectly. Somehow I forgot to put attiny88_16mhz_115200bps_ev1f7_swio1_txPD6_rxPD7_ledPD0_wdto8s_ur.zip |
|
Now it works very well under Windows as well. I have a slower hand so I like to use 8 seconds WDT timeout. :-) |
I have just added UART support for ATtiny167 and other LIN/UART parts in the urboot project. These non-standard UARTs are actually quite neat as they have a much lower quantisation error than the standard USARTs of classic parts. So now the default, precompiled urboot bootloaders (eg, for ATtiny167 autobaud ) use its PA0/PA1 lines and the UART. |
I have been tinkering with urboot on Windows, and there seem to be issues with the bootloader communication.
I set up an Digispark Pro with a ATtiny167 and flashed it with
urboot_digispark-pro_16mhz_115200bps_rxb6_txb3_led+b1_ur_vbl.hex.I am running the command
avrdude -c urclock -p t167 -P COM3and get the error
and sometimes
Depending on the USB serial converter I am using, the connection is very unreliable. My trusted FT232H-based serial converter seems to cause the most problems (>50% failure rate), a CP2102-based serial converter seems to work as expected.
The first thing I noticed is that the WIN32 implementation of
ser_recvis broken. Asser_recvdoes not return the number of bytes read, it must return the exact same number of bytes requested or -1 on failure/timeout. When a timeout occurs, the current WIN32ser_recvmay return success, but only a partial amount of bytes requested.urclock_getsyncsets a very short timeout of 10ms, so quite often,ser_recvonly returns one byte instead of two, and everything goes downhill from there.Initially, I thought I can fix this by changing
ser_recvto either receive the full amount of bytes requested or -1 on timeout, but that did not fix it. There seems to be a problem related to the small 10ms timeout, and the FTDI driver sometimes returning zero bytes.Also tried
-xstrict, did not help. I do not think its cabling, signals look good.Using -vvvv and some WIN32 patches, here is a success (notice there is still a
ser_recv(): programmer is not responding)and a failure
Saving further analysis for some other time...
The text was updated successfully, but these errors were encountered: