libelf deprecation on macOS

[iMichka]

Hello

I am a maintainer of the Homebrew project and wanted to give a small heads up: we would like to deprecate libelf on macOS.

The upstream libelf website is gone, the remaining source code can only be through archive.org ... (since at least 2019, which was 5 years ago ...). avrdude is one of the last projects that is using libelf in macOS.

Are there any plans for a replacement solution on your side?

[mcuee]

@iMichka

What is the replacement library recommended by Homebrew project?

[mcuee]

As of now, I do not know of any plans to use an alternative library for the avrdude project.

BTW, for Windows MSVC build, we have a local version of libelf.
https://github.com/avrdudes/libelf

[mcuee]

Google says a potential "replacement" of libelf is elfutils. But then Homebrew says it is Linux only.
https://formulae.brew.sh/formula/elfutils
https://github.com/Homebrew/homebrew-core/blob/1cde8401d8be5f28d74a402afe67fd52ac0575ed/Formula/e/elfutils.rb

[mcuee]

Windows MSVC build: local fork version of the legacy libelf.
https://github.com/avrdudes/libelf

Windows: MSYS2 mingw uses legacy libelf.
https://packages.msys2.org/package/mingw-w64-x86_64-libelf

macOS: Macports uses legacy libelf
https://ports.macports.org/port/libelf/details/

macOS: Homebrew still uses legacy libelf now but mark as deprecated.
https://formulae.brew.sh/formula/libelf

Debian Linux -- using elfutils to provide libelf
https://packages.debian.org/search?keywords=libelf

Arch Linux -- using elfutils to provide libelf
https://archlinux.org/packages/core/x86_64/libelf/

FreeBSD/OpenBSD -- using elftoolchain to provide libelf
https://wiki.freebsd.org/LibElf
https://man.openbsd.org/elf
https://sourceforge.net/p/elftoolchain/wiki/Home/

NetBSD seems to be still using legacy libelf.
https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/pkgsrc/devel/libelf/index.html
https://pkgsrc.se/devel/libelf

[mcuee]

Looks like there is no easy replacement of libelf for macOS now. So I think this may not be an easy task and we may not be able to deal with it anytime soon.

[dl8dtl]

Macports have a port of elftoolchain.
FreeBSD has a port of elfutils, so it's not quite clear why it is said to be "Linux only".
Other than this: the usual reason for deprecating something is that there's a better replacement available, rather than something not being developed anymore. There are many things that are no longer being developed since they are fairly complete, and just don't need further development. Take TaylorUUCP for an example: every MacOS these days still comes with it. ;-)

[mcuee]

FreeBSD has a port of elfutils, so it's not quite clear why it is said to be "Linux only".

I think it is from Homebrew project point of view. Homebrew supports macOS (normal Homebrew) and Linux. As of now, it seems Homebrew only supports elfutils under Linux and not macOS.

[mcuee]

Macports have a port of elftoolchain.

We need to check if it works for the purpose of avrdude though. If it works, then the solution is to ask Homebrew to provide elftoolchain.

[dl8dtl]

Macports have a port of elftoolchain.

We need to check if it works for the purpose of avrdude though.

Under FreeBSD, the (OS-provided) elftoolchain does the job. I no longer reference any 3rd-party ELF library in the devel/avrdude port.

Either way, my main point was, don't deprecate something unless there's a replacement which is at least as good as the deprecated item. FreeBSD still also offers the old libelf port (rather than kicking it out), but of course, installing it would conflict with elfutils then. You can only install one of those, because they install files into the same place.

[stefanrueger]

Thanks for the advance warning @iMichka. I agree with @dl8dtl that it seems unfortunate to deprecate elflib without an obvious replacement. AVR elf files won't have changed in the last decade or so, so that a library reading them doesn't need to be "new".

@mcuee @MCUdude Is is possible to drop the reliance of MacOS on libelf before the new AVRDUDE release next week?

[mcuee]

@stefanrueger

I think we should stick to libelf on macOS with avrdude 7.3 release.

Then we can explore the alternative for avrdude 7.4 release.

[mcuee]

Macports have a port of elftoolchain.

We need to check if it works for the purpose of avrdude though. If it works, then the solution is to ask Homebrew to provide elftoolchain.

I just took a look at elftoolchain macports and it is not in a good shape -- failed under Apple Silicon and almost no users (2 downloads in 30 days).
https://ports.macports.org/port/elftoolchain/details/

macports libelf is quite healthy (no issues with Apple Silicon, 641 downloads in 30 days)
https://ports.macports.org/port/libelf/details/

I think we can rule out the use of elftoolchain provided libelf as a candidate to replace libelf.

[mcuee]

As for elfutils, I do not see much efforts to port it to macOS. Google only finds one with the last commit date on 2019-Dec-24.
https://github.com/kdab-hotspot/elfutils

Unfortunately it does not seem to build well under my Mac Mini M1 (Apple macOS 14.3 with fully updated Homebrew). Good thing is that the sub libelf source codes seem to produce a static link library libelf.a.

mcuee@mcuees-Mac-mini build % pwd
/Users/mcuee/build/avr/avrdude_test/elfutils_macos/build

mcuee@mcuees-Mac-mini build %
CC=gcc-13 CFLAGS="-I/opt/homebrew/include -I/opt/homebrew/opt/gettext/include/" LDFLAGS="-L/opt/homebrew/opt/argp-standalone/lib/" PKG_CONFIG_PATH=/opt/homebrew/opt/libarchive/lib/pkgconfig ../configure --enable-maintainer-mode --disable-symbol-versioning

mcuee@mcuees-Mac-mini build % make
...
...
Making all in libcpu
  CC       i386_gendis.o
../../libcpu/i386_gendis.c: In function 'main':
../../libcpu/i386_gendis.c:70:10: error: 'error_message_count' undeclared (first use in this function)
   70 |   return error_message_count != 0;
      |          ^~~~~~~~~~~~~~~~~~~
../../libcpu/i386_gendis.c:70:10: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [i386_gendis.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

mcuee@mcuees-Mac-mini build % make -i
...
...
all kinds of errors
...
...
mcuee@mcuees-Mac-mini build % cd libelf 

mcuee@mcuees-Mac-mini libelf % ls -la libelf.a
-rw-r--r--  1 mcuee  staff  270096 Feb  3 13:05 libelf.a

mcuee@mcuees-Mac-mini libelf % file libelf.a
libelf.a: current ar archive

[iMichka]

What is the replacement library recommended by Homebrew project?

It's a complex situation. I have read all your comments, thanks for investigating.
One solution is maybe binutils that might bring equivalent tooling. It depends on what you were doing with libel.

I think we should stick to libelf on macOS with avrdude 7.3 release.
Then we can explore the alternative for avrdude 7.4 release.

That's fine, don't worry, there is no rush here.

elfutils

elfutils needs massive patching to be able to build it on macOS, so that's probably not the first solution I would consider.