Avrdude on macOS is built without libreadline #1173
Comments
|
This is because you use the default libedit and not libreadline. I hit this one when I was testing #1132. I have created PR #1156 to help macOS users. You can use MinGW MSYS2 is still having issues if readline is installed due to broken packaging of MSYS2 libreadline/libtermcap. But I have found a work-around to rebuild libtermcap. github action is okay since it does not use readline installed by default. Click for the issues with MSYS2 mingwI have also created issue #1162 to see if we can adopt arduino-packing project's way of using Containers to cross build static linked avrdude binaries for Linux, macOS and Windows (mingw32). I am able to use simplified arduino-packing scripts here. I can built git main just fine. |
|
If someone can figure out how to make libedit works, then that will be good as well. So I will label this as enhancement. As of now git main will require libreadline for macOS and Linux. We may need to update the Wiki for git main. |
|
@MCUdude |
|
The question is why did we have to break terminal mode for macOS? It worked perfectly fine before.
Yes, but the classic The main reason why I'm still using make is that the commands are much shorter and easier to remember. I might adopt Stefan's makefile: |
The problem is that macOS Homebrew reports that it has libreadline without libreadline being installed, since it treats libedit as the replacement as libreadline. Unfortunately libedit behavior is not the same as libreadline. Maybe there is a simple fix but I do not know. I am not so sure if the problem is there for MacPorts or not. If we need to differentiate libreadline and libedit, probably we need to have some CMake magic from @mariusgreuel; and we need some code change in |
OK, fair enough. But is libreadline statically or dynamically linked? Is this something every user has to install before using an Avrdude binary? It might be possible to report an issue over at the Homebrew repo to fix the issue where Homebrew treats libedit as a replacement. |
By default it is dynamically linked. That was why I created issue #1159. After extensive testing there, finally I came to the conclusion that to me the solution is to adopt arduino-packing way of building the static link binaries using a container and cross-compile. Unfortunately it needs two patches which as is will not be accepted by the avrdude project.
We can try but I am not so positive that they will adopt. They may ask use to fix avrdude as many other projects have no issues with libedit, including avrdude prior to the merge of #1132. |
|
There are many hits about libedit not a real replacement for libreadline. References: |
|
The problem is this line: Line 1233 in 4c92030 If I replace it with But how do I install libreadline using homebrew? When I run |
|
@MCUdude As for |
You will lose the ability to connect to bootloaders in the terminal (they will time out). I feel we should solve this by getting CMake to do the right thing for MacOS before the 7.1 release. |
I think we can detect the presence of Homebrew libreadline by checking the presence of If the above is not present, then it will use system libedit. We can set @mariusgreuel |
There are three issues here.
|
|
Interestingly Homebrew git main build is actually able to use libreadline and not libedit for my system (where I force link by running The formula does not seem to do anything special. |
|
@MCUdude There is a warning about the |
|
@MCUdude Without the force link step, avrdude will use libedit and the issue will be present. But yes it will still be good to be able to automatically detect the lack of libreadline (only libedit is present) and then warn the user to install libreadline before proceeding. |
|
@MCUdude |
|
There is a discussion here for reference -- to me it is good to mandate the use of readline. |
|
Would you please look at the script here for a more comprehensive way of finding readline so that libedit will not be detected as readline? Thanks. The project has similar more complex scripts for libusb and hidapi, but I think those are not necessary. |
|
For auto-tools build Something for reference from Postgres. |
Yet the other quick solution is to follow @MCUdude's suggestion to sort out the immediate issue and then forego the benefits of the terminal time out as of now. In this case, everything will work out of the box under macOS except the terminal mode timeout improvement (no more quick flashing LEDs if you want some visual hints). Take note libedit (or libreadline) is still needed for other parts of |
|
BTW, it is not a big problem to require readline under macOS. You can see common software like Python, sqlite and gtk+ depend on readline. Most likely it is because libedit is not able to do the job. I got smaller x86_64 homebrew installation but you can see readline is still essential. For my ARM64 homebrew installation, it is even more obvious. |
|
So the following is my recommendation for macOS Homebrew users. For macports users, just require readline will do, no need the extra stuff mentioned below. No code changes, just Wiki documentation updates.
for ARM64 or for x86_64. |
|
But if @MCUdude prefers not to require readline, I am also okay with the following one line change. (Line number is for PR #1171 as I expect it to be merged soon). |
|
Future improvement when @mariusgreuel or other CMake experts are available -- to differentiate libedit vs libreadline and set that line accordingly. I am not so sure if @dl8dtl or @mariusgreuel want to fix the Reference:
|
I'm really having a hard time getting the Avrdude build system to pick the correct readline library. When I build using Terminal mode is still broken, just like when I build with I've also tried but I'm getting the exact same result EDIT: |
|
@MCUdude You can just use |
|
Thanks @mcuee! Using build.sh did work. I'm totally OK with Avrdude requiring libreadline in order to work, but I think that the precense/absense of libreadline should be detected and printed on the screen when building with build.sh, cmake, or make. Currently, there's no mention of libreadline anywhere: To me, it's important that building Avrdude "just works", on any system, as long as the user follows the build instructions. The instructions are yet to be updated (for macOS at least). |
Yes, when going as far back as this, I'm able to pipe into Avrdude |
@MCUdude and @stefanrueger If there is no easy workaround, I will agree to use GNU Readline under macOS. I think we should be able to change |
I think Option 3 is no go as it is a regression. What I suggest is a mix of Option 1 and Option 2. For macOS, I hope we can get libedit to work with either As for Windows, it is a different story.
|
|
Another thing, I can provide Linux/macOS/Windows avrdude binaries with GNU Readline here, using the nice infrastructure provided by Arduino project. |
So basically this commit (PR #1132) will not work with libedit in two aspects under macOS.
|
|
BTW, the following is the build command Homebrew is using (macOS Apple Silicon). So it is linked to libedit by default. |
|
Here is the default auto-tools build. Even though it does not claim to find GNU Readline in this case (some modifications will be be needed to make the display that correct), it is actually linked to libedit. Terminal mode call back will work thanks to the workaround by @dl8dtl. But unfortunately pipes will not work. |
|
@MCUdude Again, even though it does not claim to find GNU Readline in this case, it is actually linked to libreadline. Some modifications to |
Well, @MCUdude identified that by using BTW, I think this all has been caused by Stallman overreaching himself. He insisted GNU readline to be GNU copyleft in the hope that everyone who used the library would be forced to open source. Well, Apple and Microsoft certainly did not like the idea and rolled their own line editors for their command lines, which brought us here. |
Indeed this is what happened. |
Indeed there may be no easy workaround. Let's wait for a while to see if @dl8dtl or others who can come out with a potential solution. In any case, there are no real issues as of now to build with GNU Readline with git main. I have updated the macOS build instructions here and I include recommendations to build with GNU Readline to have better user experiences with terminal mode. If there is no easy workaround and we can reinstate PR #1156 later and I can update the Wiki again. |
|
Just wondering what is your preference?
My preference is either Option 1 or Option 3 if someone wants to work on Option 3. I think we can always go with Option 1 for 7.1 Release and later go with Option 3 if there are work-around. BTW, please also review the wiki here. |
|
For what my tuppence are worth, if it was me I'd go for 1, 4 in that order. 2 is no good as pipes are more important than keeping bootloaders alive. I predict 3 to be too much work. |
|
I'd prefer 3, 1, in this order. If piping is impossible with libedit, then option 1 is the second best. If I'm going to incorporate Urboot into my Arduino cores someday, I'd like all features to work on MacOS, Windows and Linux favours. However, ./configure + make should be updated to automatically detect if libreadline is present on macOS (brew or macports), and use this instead of libedit. |
|
@MCUdude @dl8dtl and @stefanrueger I guess @dl8dtl would prefer Option 3 over Option 1 as well. Looks like nobody would like to go with Option 2. Option 1 is the common denominator now. On the other hand, I think we should not prevent 7.1 release because of this, rather we should go with Option 1 for 7.1 release unless Option 3 somehow gets implemented miracally with two to three weeks, We can always keep Option 3 as future enhancement for macOS. Is this an okay compromise? |
|
Yes, I'll see how to find a workaround. |
I am not exactly sure what you mean by Please refer to the comments earlier. There is a error that it does not detect GNU Readline but the result binary is correct. |
If Option 3 can be implemented before Release 7.1, that will be great. I think your idea sounds good. |
When is the 7.1 release planned? I was hoping to get TPI support for JTAG3 compatible programmers in before the release. It's in the works and almost ready. |
Not officially planned but the hope is before Christmas. @dl8dtl agrees that Christmas is a good target date for 7.1 release. |
@dl8dtl That's a very clever idea - and should work! Gets around having to fix
@MCUdude AVRDUDE's ability to keep any bootloader alive does not depend on urboot. It currently also works with optiboot. The trick is
Currently |
It would be vvv neat to have this functionality present when using |
See PR #1207 |
|
I actully figured out a rather hacky way to use CMake to differentiate libedit vs GNU Readline, as macOS detects the symbolic link So I think @dl8dtl's idea to detect at runtime is much better. |
|
Latest git with auto-tools. |
|
auto-tools generated tar-ball under macOS Homebrew. |
|
The above tarball seems to kind of working for MSYS2 as well with proper configure options, except the last linking step (missing Winsock2 library). |
The issue is caused by #1132.
I'm building using make, and after #1132 was merged, I have to hit enter after every command in order for
avrdude>to appear. When I start to type, the text appears on the line below.Shouldn't this "just work"?
The text was updated successfully, but these errors were encountered: