Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS Catalina: Pro Micro (w/Caterina bootloader) flashing - avrdude: butterfly_recv(): programmer is not responding #6133

Open
sidcarter opened this issue Jun 15, 2019 · 52 comments

Comments

@sidcarter
Copy link
Contributor

@sidcarter sidcarter commented Jun 15, 2019

Trying to flash a contra which I've flashed in the past and I get this error now:

I'm on MacOS Catalina Beta 1 now, which is the main difference now. Merged the latest master into my branch too. I've tried the remove USB, reset and plug USB back in. No change in the error.

I've also uninstalled avrdude and reinstalled it too. No dice.
I've tried the QMK Toolbox and I get the same error there too. QMK Toolbox also has worked in the past. I've downloaded the most recent release and tried too. Didn't work either.

Detecting USB port, reset your controller now........
Device /dev/tty.usbmodem14601 has appeared; assuming it is the controller.
Waiting for /dev/tty.usbmodem14601 to become writable.

Connecting to programmer: .avrdude: butterfly_recv(): programmer is not responding

avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_drain(): read error: Device not configured
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
Found programmer: Id = ""; type =
    Software Version = .; Hardware Version = .
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: buffered memory access not supported. Maybe it isn't
a butterfly/AVR109 but a AVR910 device?
avrdude: initialization failed, rc=-1
         Double check connections and try again, or use -F to override
         this check.

avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: leave prog mode
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: exit bootloader
avrdude: ser_close(): can't reset attributes for device: Device not configured

avrdude done.  Thank you.

make[1]: *** [avrdude] Error 1
make: *** [contra:sirovia:avrdude] Error 1
@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Jun 16, 2019

Could you please post the output of the avrdude command, but with -vv?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 16, 2019

@fauxpark

avrdude -p atmega32u4 -c avr109 -U flash:w:/Users/sidcarter/Labor/qmk_firmware/contra_sirovia.hex:i -P /dev/cu.usbmodem14501  -vv

avrdude: Version 6.3, compiled on Jun 15 2019 at 16:34:39
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "/usr/local/etc/avrdude.conf"
         User configuration file is "/Users/sidcarter/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : /dev/cu.usbmodem14501
         Using Programmer              : avr109
         AVR Part                      : ATmega32U4
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PA0
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  9000  9000 0x00 0x00
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0x00 0x00
           lfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           lock           0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : butterfly
         Description     : Atmel AppNote AVR109 Boot Loader

Connecting to programmer: .avrdude: ser_recv(): programmer is not responding
avrdude: butterfly_recv(): programmer is not responding

avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_drain(): read error: Device not configured
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
Found programmer: Id = "�{{���"; type =
    Software Version = .; Hardware Version = .
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: buffered memory access not supported. Maybe it isn't
a butterfly/AVR109 but a AVR910 device?
avrdude: initialization failed, rc=-1
         Double check connections and try again, or use -F to override
         this check.

avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: leave prog mode
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: exit bootloader
avrdude: ser_close(): can't reset attributes for device: Device not configured

avrdude done.  Thank you.
@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Jun 16, 2019

Do you also see a /dev/tty.usbmodem14501? If so, does that work instead?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 16, 2019

Do you also see a /dev/tty.usbmodem14501? If so, does that work instead?

did try with a tty too - same error

avrdude -p atmega32u4 -c avr109 -U flash:w:/Users/sidcarter/Labor/qmk_firmware/contra_sirovia.hex:i -P /dev/tty.usbmodem14401  -vv

avrdude: Version 6.3, compiled on Jun 15 2019 at 16:34:39
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "/usr/local/etc/avrdude.conf"
         User configuration file is "/Users/sidcarter/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : /dev/tty.usbmodem14401
         Using Programmer              : avr109
         AVR Part                      : ATmega32U4
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PA0
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  9000  9000 0x00 0x00
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0x00 0x00
           lfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           lock           0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : butterfly
         Description     : Atmel AppNote AVR109 Boot Loader

Connecting to programmer: .avrdude: ser_recv(): programmer is not responding
avrdude: butterfly_recv(): programmer is not responding

avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_drain(): read error: Device not configured
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
Found programmer: Id = "W���"; type =
    Software Version = .; Hardware Version = .
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: buffered memory access not supported. Maybe it isn't
a butterfly/AVR109 but a AVR910 device?
avrdude: initialization failed, rc=-1
         Double check connections and try again, or use -F to override
         this check.

avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: leave prog mode
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: exit bootloader
avrdude: ser_close(): can't reset attributes for device: Device not configured

avrdude done.  Thank you.
@drashna

This comment has been minimized.

Copy link
Member

@drashna drashna commented Jun 16, 2019

Can you flash it on a different device?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 16, 2019

Just patched my fc660c

Creating load file for flashing: .build/fc660c_siroleo.hex                                          [OK]
Copying fc660c_siroleo.hex to qmk_firmware folder                                                   [OK]
Checking file size of fc660c_siroleo.hex                                                            [OK]
 * The firmware size is fine - 23200/28672 (5472 bytes free)
dfu-programmer: no device present.
Error: Bootloader not found. Trying again in 5s.
dfu-programmer: no device present.
Error: Bootloader not found. Trying again in 5s.
Bootloader Version: 0x00 (0)
Erasing flash...  Success
Checking memory from 0x0 to 0x6FFF...  Empty.
Checking memory from 0x0 to 0x5AFF...  Empty.
0%                            100%  Programming 0x5B00 bytes...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>]  Success
0%                            100%  Reading 0x7000 bytes...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>]  Success
Validating...  Success
0x5B00 bytes written into 0x7000 bytes memory (81.25%).
@drashna

This comment has been minimized.

Copy link
Member

@drashna drashna commented Jun 17, 2019

Sorry, I mean on a different host computer.

The fc660c is using DFU, not caterina/avrdude, so it's a very different ballpark.

Also, i may be worth running ls /dev/tty.usb* to get a list of devices. Which would check to see which COM ports are active.

Also, it may be that the controller is soft bricked. And if it is, you'll need to ISP Flash the bootloader.
And in that case, you want to take a look at this link: https://docs.qmk.fm/#/isp_flashing_guide
And if you do end up doing that, you may want to look at this link (to replace the bootloader with something a LOT more sane): https://www.reddit.com/r/olkb/comments/8sxgzb/replace_pro_micro_bootloader_with_qmk_dfu/

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 18, 2019

@drashna I think it's just the host computer. let me test it on a local computer and will let you know.

On an unrelated note, just trying to flash a dz60rgb and that crashes the latest QMK Toolbox on macOS Catalina 10.15.

@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Jun 18, 2019

@sidcarter could you please open an issue for that over on https://github.com/qmk/qmk_toolbox, with any stack trace or error it produces (you may have to look in Console.app)?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 18, 2019

Done - #6147 @fauxpark

@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Jun 18, 2019

I meant on the Toolbox repo 😉 but in any case, I can tell from a quick glance at that crash log what the problem is, and I should have picked up on it when you mentioned the DZ60RGB. See qmk/qmk_toolbox#84 (comment)

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 26, 2019

@drashna got to try it on a different Mac. This one had Mac OS 12.6 (Sierra). I was able to flash the contra without any issue.

I recently updated to beta 2 on MacOS Catalina, so I tried to flash the contra again and it's still failing with the same error. Let me know if you need any additional info. thanks

@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Jun 26, 2019

Don't know why I didn't think of this before, but does running avrdude with sudo have any effect?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jun 26, 2019

Don't know why I didn't think of this before, but does running avrdude with sudo have any effect?

tried that too - no difference

@jamztang

This comment has been minimized.

Copy link

@jamztang jamztang commented Jul 3, 2019

@sidcarter were you able to fix for the command line?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Jul 6, 2019

@sidcarter were you able to fix for the command line?

nope. still no luck on the command line either so far.

@tylerwince

This comment has been minimized.

Copy link
Contributor

@tylerwince tylerwince commented Aug 13, 2019

Any update on this? Or a workaround for the new beta?

@sidcarter

This comment has been minimized.

Copy link
Contributor Author

@sidcarter sidcarter commented Aug 15, 2019

@tylerwince sold my contra - so can't test this anymore. now I have no devices that require avrdude. bootloadhid - now that's another ballgame :D

@tylerwince

This comment has been minimized.

Copy link
Contributor

@tylerwince tylerwince commented Aug 15, 2019

Bummer. - I tried to flash an avrdude the other day to no avail. I will try to find a workaround 👍

@drashna

This comment has been minimized.

Copy link
Member

@drashna drashna commented Aug 22, 2019

From what I understand, this may actually be a bug in Catalina, specifically. Not sure there is a workaround. At least at this point.

@sigul

This comment has been minimized.

Copy link
Contributor

@sigul sigul commented Sep 15, 2019

Still need to be fixed.
Here's what I got from the last version of Catalina.

Caterina device connected
*** Attempting to flash, please don't remove device
Found port: /dev/cu.usbmodem14301

avrdude -p atmega32u4 -c avr109 -U flash:w:/Users/silviogulizia/Downloads/contra_default_EA425786.hex:i -P /dev/cu.usbmodem14301 -C avrdude.conf
avrdude: warning at avrdude.conf:14976: part atmega32u4 overwrites previous definition avrdude.conf:11487.

Connecting to programmer: .avrdude: butterfly_recv(): programmer is not responding
*** Caterina device disconnected

@nooges nooges changed the title Contra flashing - avrdude: butterfly_recv(): programmer is not responding Pro Micro (w/Caterina bootloader) flashing - avrdude: butterfly_recv(): programmer is not responding Sep 16, 2019
@pixelbreaker

This comment has been minimized.

Copy link
Contributor

@pixelbreaker pixelbreaker commented Sep 16, 2019

I'm also having this issue on Catalina latest beta, having to flash using another machine for now 😢

@nooges

This comment has been minimized.

Copy link
Member

@nooges nooges commented Sep 16, 2019

Doing some initial investigation into the issue, it looks like the reset of the MCU (using the 1200 baud toggle method) that's done before flashing isn't working properly on Catalina. If trying to do stty -f /dev/cu.usbmodemXXX 1200, this error shows up in dmesg on Catalina beta only: mctl RS232_S_OUTPUTS failed e0005000.

Testing method:

  1. Press hardware reset button (or short RST and GND) on Pro Micro
  2. Run stty -f /dev/cu.usbmodemXXX 1200
  3. Run sudo dmesg and observe mctl RS232_S_OUTPUTS failed error

The error is also observed when avrdude is run, presumably because it also tries doing the 1200 baud toggle to reset the MCU.

Further testing with Arduino IDE and stock Pro Micro also had the same problem on Catalina.

@yanfali

This comment has been minimized.

Copy link
Contributor

@yanfali yanfali commented Sep 17, 2019

Apple doesn't test anything that slow probably. Since this breaks Arduino IDE, I would except their much larger community to kick up a fuss.

@tamanishi

This comment has been minimized.

Copy link
Contributor

@tamanishi tamanishi commented Oct 5, 2019

10.15 Beta (19A582a)
Not fixed yet...

@yanfali

This comment has been minimized.

Copy link
Contributor

@yanfali yanfali commented Oct 6, 2019

I'm hearing that there are significant security challenges in Catalina for all apps, so we're going to have to wait and see what the fallout is on the brew side.

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Oct 8, 2019

I can't flash my Pro micro clone on 10.15 final public stable.

This is my output:

➜  qmk_firmware git:(master) ✗ make ardaplanck:turkishish:avrdude
QMK Firmware 0.7.35
Making ardaplanck with keymap turkishish and target avrdude

avr-gcc (GCC) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Size before:
   text	   data	    bss	    dec	    hex	filename
      0	  24920	      0	  24920	   6158	.build/ardaplanck_turkishish.hex

Copying ardaplanck_turkishish.hex to qmk_firmware folder                                            [OK]
Checking file size of ardaplanck_turkishish.hex                                                     [OK]
 * The firmware size is fine - 24920/28672 (86%, 3752 bytes free)
Detecting USB port, reset your controller now..........................................................
Device /dev/tty.usbmodem14201 has appeared; assuming it is the controller.
Waiting for /dev/tty.usbmodem14201 to become writable.

Connecting to programmer: .avrdude: butterfly_recv(): programmer is not responding

avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_drain(): read error: Device not configured
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
Found programmer: Id = ""; type =
    Software Version = .; Hardware Version = .
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: buffered memory access not supported. Maybe it isn't
a butterfly/AVR109 but a AVR910 device?
avrdude: initialization failed, rc=-1
         Double check connections and try again, or use -F to override
         this check.

avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: leave prog mode
avrdude: ser_send(): write error: Device not configured
avrdude: ser_recv(): read error: Device not configured
avrdude: butterfly_recv(): programmer is not responding
avrdude: error: programmer did not respond to command: exit bootloader
avrdude: ser_close(): can't reset attributes for device: Device not configured

avrdude done.  Thank you.

make[1]: *** [avrdude] Error 1
make: *** [ardaplanck:turkishish:avrdude] Error 1

Can I use vagrant etc. for the time being to circumvent this? Or How do I install with Catalina installed ?

Thanks in advance,

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Oct 10, 2019

For the time being, I use Windows QMK toolbox and an ancient Linux netbook that has been lying around to flash the hex file or compilation. I still cannot flash with make command, with Catalina.

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Oct 11, 2019

I'm not pro about this subject, but I believe it's because macOS Catalina dropped support for 32 bit apps.

In here, I found a 64 bit build platformio/platform-atmelavr#147 (comment)

http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino17-x86_64-apple-darwin12.tar.bz2

Also, this pull request includes avr-gcc along other tools which they claim fixes the moaning about macOS Catalina 10.15.

I'm not sure whether these are merged already, but I'd appreciate an input.

@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Oct 11, 2019

I'm doubtful that avr-gcc plays a part in this. Our scripts already install the 64-bit build from Homebrew, whereas Arduino seems to have been using 32-bit up to now.

Additionally, the avrdude in the Toolbox is also 64-bit (it's taken from the Homebrew bottle):

fauxpark@marissa:Resources$ pwd
/Applications/QMK Toolbox.app/Contents/Resources
fauxpark@marissa:Resources$ file avrdude
avrdude: Mach-O 64-bit executable x86_64

The bottle was just updated a couple of hours ago to add the binary served to Catalina machines, but Mojave and below were not touched, and it still does not seem to work.

@nooges nooges changed the title Pro Micro (w/Caterina bootloader) flashing - avrdude: butterfly_recv(): programmer is not responding macOS Catalina: Pro Micro (w/Caterina bootloader) flashing - avrdude: butterfly_recv(): programmer is not responding Oct 11, 2019
@facchinm

This comment has been minimized.

Copy link

@facchinm facchinm commented Oct 11, 2019

@fauxpark latest AVR core (1.8.1) bundled with Arduino IDE 1.8.10 is fully 64bit on macOS (all tools and complementary stuff).
The issue is really about Apple and should be fixed by Apple IMO (we can't ask hundred of thousand users to buy a programmer / another arduino to update the bootloader)

@nooges

This comment has been minimized.

Copy link
Member

@nooges nooges commented Oct 11, 2019

Yeah we need to file a Radar for this so Apple can fix it.

@facchinm

This comment has been minimized.

Copy link

@facchinm facchinm commented Oct 11, 2019

@nooges please coordinate with @gvarisco that is taking care of the report on our side 😉

@d53dave

This comment has been minimized.

Copy link

@d53dave d53dave commented Oct 11, 2019

Just chiming in to say I'm seeing the same issue.

@gvarisco

This comment has been minimized.

Copy link

@gvarisco gvarisco commented Oct 13, 2019

@nooges hey! I've just filled a case (FB7376052 for reference) with Apple mentioning the fact that macOS Catalina update broke the bootloader recognition for atmega32u4-based Arduino boards and I hope to hear back from them as this is impacting lot of users. I've added this GH issue as well as a reference. Drop me a line at g.varisco AT arduino.cc and I'll make sure to keep you updated on how this goes.

@lattedesu

This comment has been minimized.

Copy link

@lattedesu lattedesu commented Oct 13, 2019

Hey guys, just out of curiosity, when I upgrade to Catalina, will it work with docker or a vm solution such as Virtualbox? This issue is simply preventing me to upgrade.

@fauxpark

This comment has been minimized.

Copy link
Member

@fauxpark fauxpark commented Oct 13, 2019

@lattedesu I don't know about Docker, but you can definitely use Windows or Linux on Virtualbox and pass through the bootloader device in order to flash :)

This only applies to the Caterina bootloader (Pro Micro, Arduino, etc), and possibly Massdrop boards as the ATSAM bootloader also appears as a serial port. dfu-programmer, dfu-util and teensyloader all seem to work fine.

@w-A-L-L-e

This comment has been minimized.

Copy link

@w-A-L-L-e w-A-L-L-e commented Oct 15, 2019

Any updates on this??? Just updated from mojave to catalina yesterday and instantly noticed it broke avrdude flashing with caterina bootloaders. The device reset still works fine and port is changed/available from dev/cu.usbmodem.C1 to /dev/cu.usbmodem14011 after reset. But avrdude indeed can't communicate with the bootloader -> basically ser_receive/ser_send is broken. It's not an avr-gcc issue as compiling works fine and produces a .hex that is usuable when flashing with other methods(external programmer for instance). It's purely a communication issue introduced on MacOs Catalina between avrdude and the caterina cdc bootloader (used on various arduino leonardos and clones). Mojave and any older os x (Sierra, High Sierra etc,) all work fine.

Als serial comms to regular sketch code still works fine. It's only when in bootloader that serial communication is failing here.

@w-A-L-L-e

This comment has been minimized.

Copy link

@w-A-L-L-e w-A-L-L-e commented Oct 15, 2019

More info. Suspecting that vdc assistant on mac os catelina that's interfering with the communication. This background service tries to detect the arduino as a usb camera (and is sending most likely garbage on the serial that crashes the bootloader) and therefore messes up the communication with avrdude. When arduino leonardo enters bootloader in console log there are these errors popping up :
CMIO_DPA_VDC_Server_Assistant.cpp:1383:GetDeviceByGUID No match for specified GUID 0x1420000023418036

DeviceArrived caught an exception from determining the deviceType (0xffffffffffffffff) or removing the device

CMIO_DPA_VDC_Server_USBClient.cpp:334:USBClient [guid:0x1420000023418036] Could not find SC_VIDEOCONTROL interface

DeviceArrived Found a camera (0x1420000023418036) , but was not able to start it up (0x0 -- (os/kern) successful)

Ok : UPDATE this is not the issue, nothing is fixed when stopping VDC Assistant. So bug must be elsewhere.

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.cmio.VDCAssistant.plist

After stopping it same bug is still happening avrdude: butterfly_recv(): programmer is not responding
Found programmer: Id = ""; type =
Software Version = .; Hardware Version = .

I see that the bootloader crashes though as ours does some special led blinking on different steps and that is also disrupted on macos catalina.

@w-A-L-L-e

This comment has been minimized.

Copy link

@w-A-L-L-e w-A-L-L-e commented Oct 16, 2019

Today big +900Mb update on Catalina, installed it. But still no dice, same issue remains with avrdude being broken here. Also noticed other broken things for instance the mini display ports no longer support higher 4k resolutions anymore. So apart from 32bit a whole other range of stuff got broken on Catalina (really regret this upgrade so far...). To others as advice: for now stick with Mojave its more stable / less bugs.

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Oct 17, 2019

I have just sent a feedback to Apple via https://feedbackassistant.apple.com/ , regarding this issue. I wonder if anything would turn up, but I hope someone at Apple addresses this.

@tamanishi

This comment has been minimized.

Copy link
Contributor

@tamanishi tamanishi commented Oct 19, 2019

10.15.1 Beta (19B77a) 😢

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Oct 19, 2019

I can confirm flashing works on VirtualBox which is installed over Catalina. I've tried with Ubuntu 16.04 which was already installed, but I'm sure it'll work on 18.04 or newer non-lts versions. I was tired as hell to compile the hex file on mac and then flash it on Windows. I've tried with a vagrant box which I had installed already for a local environment of another project.

You have to install the VirtualBox extension pack (you can find it on downloads page of Virtualbox) for USB support, and then you have to enable USB forwarding. I've enabled USB 2.0. Shut down the vm, at the settings page of the vm, go to USB settings, quickly set the keyboard at bootloader mode (short the reset pin), and click the green plus arrow at the right section (add a device icon). You'll see the Arduino (or whatever it's shown for your case) on USB devices list. Set it, save it and boot the virtual machine.

To test that USB works, while inside the vm (you can ssh or whatever), set your keyboard at bootloader mode again (short the reset pin), and type lsusb while at the bootloader mode. If you see Arduino device at the USB devices list while the device is at bootloader mode, you're good to go.

Just a thing, my default vagrant user didn't have serial port write access through USB, so I had to run the command as superuser, something like sudo make keyboard:layout:avrdude to circumvent this.

@p1ne

This comment has been minimized.

Copy link

@p1ne p1ne commented Oct 23, 2019

Here I created Vagrantfile which does everything via provisioning. One can use it for sample. My project is based on Spark Fun Pro Micro, that's why VID/PID are set for it.

https://github.com/p1ne/fdim-controller/blob/master/Vagrantfile

@pddg

This comment has been minimized.

Copy link

@pddg pddg commented Nov 3, 2019

In my case,I have successfully flush the firmware by using Vagrant + VirtualBox on macOS Catalina.
My environment is as follows.

  • macOS Catalina 10.15
  • Homebrew 2.1.15
  • avrdude version 6.3
  • Vagrant 2.2.5
    • vagrant-vbguest 0.20.0
  • VirtualBox 6.0.14r133895
    • VirtualBox Extension Pack 6.0.14

VirtualBox extension pack is required to use USB device on guest OS. I use following Vagrantfile.

# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|

  config.vm.box = "hashicorp/bionic64"
  config.vm.provider "virtualbox" do |v|
    v.name = "VM to flash firmware into ProMicro"
    v.customize ["modifyvm", :id, "--usb", "on"]
    v.customize ["modifyvm", :id, "--usbehci", "on"]
    # Please rewrite according to your environment. VID/PID is required.
    v.customize ["usbfilter", "add", "0",
                 "--target", :id,
                 "--name", "Arduino Micro (writable)",
                 "--vendorid", "2341",
                 "--productid", "0037",
                 "--remote", "no"]
    v.customize ["usbfilter", "add", "1",
                 "--target", :id,
                 "--name", "Arduino Micro",
                 "--vendorid", "2341",
                 "--productid", "8037",
                 "--remote", "no"]
  end

  # I used local qmk_firmware repository.
  # Please rewrite according to your environment.
  config.vm.synced_folder "/Users/username/qmk_firmware", "/qmk_firmware"
  config.vbguest.auto_update = true

  # Install build tools
  config.vm.provision "shell", privileged: false, inline: <<-SHELL 
    sudo apt-get update
    sudo apt-get install -y python3-pip avrdude
    /qmk_firmware/util/qmk_install.sh
   SHELL
end

The procedure is as follows.

  1. Install Vagrant and VirtualBox and Extension pack
  2. Install vagrant plugin (vagrant plugin install vagrant-vbguest)
  3. Update Guest additions (vagrant vbguest)
  4. Confirm the VID/PID of your device (VBoxManage list usbhost)
    • In bootloader mode, the VID/PID will change. You should confirm it too.
  5. Write Vagrantfile (I restarted my Mac after this)
  6. Unplug the USB device
  7. Launch VM (vagrant up) and SSH into it (vagrant ssh)
  8. Plug the USB device
  9. Build your firmware (cd /qmk_firmware && make KEYBOARD:KEYMAP:avrdude)
  10. Short the reset pin if the command require to reset the device.

I was able to flush the firmware this way, but the procedure may be inaccurate :(

@RaoulDuke-Esq

This comment has been minimized.

Copy link

@RaoulDuke-Esq RaoulDuke-Esq commented Nov 9, 2019

Had my fingers crossed that this would be fixed in 10.15.1, but alas... Will have to continue hoping for change. In the meanwhile, gonna use a spare RPI for QMK flashing purposes.

@tomwinget

This comment has been minimized.

Copy link

@tomwinget tomwinget commented Nov 12, 2019

In my case,I have successfully flush the firmware by using Vagrant + VirtualBox on macOS Catalina.
My environment is as follows.

  • macOS Catalina 10.15

  • Homebrew 2.1.15

  • avrdude version 6.3

  • Vagrant 2.2.5

    • vagrant-vbguest 0.20.0
  • VirtualBox 6.0.14r133895

    • VirtualBox Extension Pack 6.0.14

VirtualBox extension pack is required to use USB device on guest OS. I use following Vagrantfile.

# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|

  config.vm.box = "hashicorp/bionic64"
  config.vm.provider "virtualbox" do |v|
    v.name = "VM to flash firmware into ProMicro"
    v.customize ["modifyvm", :id, "--usb", "on"]
    v.customize ["modifyvm", :id, "--usbehci", "on"]
    # Please rewrite according to your environment. VID/PID is required.
    v.customize ["usbfilter", "add", "0",
                 "--target", :id,
                 "--name", "Arduino Micro (writable)",
                 "--vendorid", "2341",
                 "--productid", "0037",
                 "--remote", "no"]
    v.customize ["usbfilter", "add", "1",
                 "--target", :id,
                 "--name", "Arduino Micro",
                 "--vendorid", "2341",
                 "--productid", "8037",
                 "--remote", "no"]
  end

  # I used local qmk_firmware repository.
  # Please rewrite according to your environment.
  config.vm.synced_folder "/Users/username/qmk_firmware", "/qmk_firmware"
  config.vbguest.auto_update = true

  # Install build tools
  config.vm.provision "shell", privileged: false, inline: <<-SHELL 
    sudo apt-get update
    sudo apt-get install -y python3-pip avrdude
    /qmk_firmware/util/qmk_install.sh
   SHELL
end

The procedure is as follows.

  1. Install Vagrant and VirtualBox and Extension pack

  2. Install vagrant plugin (vagrant plugin install vagrant-vbguest)

  3. Update Guest additions (vagrant vbguest)

  4. Confirm the VID/PID of your device (VBoxManage list usbhost)

    • In bootloader mode, the VID/PID will change. You should confirm it too.
  5. Write Vagrantfile (I restarted my Mac after this)

  6. Unplug the USB device

  7. Launch VM (vagrant up) and SSH into it (vagrant ssh)

  8. Plug the USB device

  9. Build your firmware (cd /qmk_firmware && make KEYBOARD:KEYMAP:avrdude)

  10. Short the reset pin if the command require to reset the device.

I was able to flush the firmware this way, but the procedure may be inaccurate :(

This worked for me with a few adjustments and clarifications:

  • the "writable" arduino needs the PID/VID from the pro-micro in reset mode and the second is your normally running pro-micro (when attached as a keyboard)
  • run vagrant up before you run vagrant vbguest
  • your keyboard won't work while the vagrant box is up as this filters the USB device through it, so be ready to use another keyboard (at least this was my experience)

Great work and thank you for making this so easy!

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Nov 12, 2019

For vagrant, add the USB device while the keyboard is at the reset mode. Think like your keyboard is made of two devices. This way, in normal mode, it'll work as any keyboard from your main OS, and will be forwarded to virtual machine only when the device is at bootloader mode (reset pin is pressed).

Also, I had a vagrant-puppet-virtualbox combo for another project which I've installed QMK over and flashed firmware various times, never had a need to type the product id or vendor id etc. .

@danielpickett

This comment has been minimized.

Copy link

@danielpickett danielpickett commented Nov 15, 2019

Been struggling with this for two or three evenings now. I'm able to build firmware using the Vagrantfile included with the QMK repo. I've updated the VID and PID of my Pro Micro both in normal mode and again in bootloader/reset mode. I thought I had it. I got the prompt to reset controller, but then got Device /dev/ttyACM0 has appeared; assuming it is the controller after shorting my reset pins. Additional resets do not help. It's just waiting. I'm new to all this. Any help? @pddg @tomwinget

vagrant@debian9:/vagrant$ make dpickett40:default:avrdude
QMK Firmware 0.7.75
Making dpickett40 with keymap default and target avrdude

avr-gcc (GCC) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Size before:
   text	   data	    bss	    dec	    hex	filename
      0	  18916	      0	  18916	   49e4	.build/dpickett40_default.hex

Copying dpickett40_default.hex to qmk_firmware folder                                               [OK]
Checking file size of dpickett40_default.hex                                                        [OK]
 * The firmware size is fine - 18916/28672 (65%, 9756 bytes free)
Detecting USB port, reset your controller now..............................
Device /dev/ttyACM0 has appeared; assuming it is the controller.
Waiting for /dev/ttyACM0 to become writable.....................................................................................................................................................................................................................................................
@tomwinget

This comment has been minimized.

Copy link

@tomwinget tomwinget commented Nov 15, 2019

Been struggling with this for two or three evenings now. I'm able to build firmware using the Vagrantfile included with the QMK repo. I've updated the VID and PID of my Pro Micro both in normal mode and again in bootloader/reset mode. I thought I had it. I got the prompt to reset controller, but then got Device /dev/ttyACM0 has appeared; assuming it is the controller after shorting my reset pins. Additional resets do not help. It's just waiting. I'm new to all this. Any help? @pddg @tomwinget

vagrant@debian9:/vagrant$ make dpickett40:default:avrdude
QMK Firmware 0.7.75
Making dpickett40 with keymap default and target avrdude

avr-gcc (GCC) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Size before:
   text	   data	    bss	    dec	    hex	filename
      0	  18916	      0	  18916	   49e4	.build/dpickett40_default.hex

Copying dpickett40_default.hex to qmk_firmware folder                                               [OK]
Checking file size of dpickett40_default.hex                                                        [OK]
 * The firmware size is fine - 18916/28672 (65%, 9756 bytes free)
Detecting USB port, reset your controller now..............................
Device /dev/ttyACM0 has appeared; assuming it is the controller.
Waiting for /dev/ttyACM0 to become writable.....................................................................................................................................................................................................................................................

Try running the make command with sudo, I had the same problem but that fixed it for me.

@danielpickett

This comment has been minimized.

Copy link

@danielpickett danielpickett commented Nov 15, 2019

Yay!! I just flashed my keyboard on Catalina! @tomwinget, you are the best. I was afraid I was going to have to downgrade macOS.

Apparently I also did not have arvdude installed in the VirtualBox, but was able to solve that with sudo apt-get install avrdude.

@Ardakilic

This comment has been minimized.

Copy link
Contributor

@Ardakilic Ardakilic commented Nov 15, 2019

Running the command as superuser helps, because the vagrant user doesn't have serial port access as mentioned. Also, my ancient netbook (Intel Atom CPU, 2gb RAM, Lubuntu 18.04 installed) had the same issue, because it was so slow it couldn't start uploading in 8 seconds 🤷‍♂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.