Augeas bottles for Big Sur are broken due to an outdated configure script

[bmw]

brew gist-logs <formula> link OR brew config AND brew doctor output

$ brew config
HOMEBREW_VERSION: 3.2.7-12-gbbd1954
ORIGIN: https://github.com/Homebrew/brew
HEAD: bbd1954523c986dc3e15096c571a2b913a290fac
Last commit: 21 minutes ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 6944bf4e08d9f2fa49a3a938af9f39d04ad84f40
Core tap last commit: 8 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: vim
HOMEBREW_MAKE_JOBS: 8
HOMEBREW_NO_ANALYTICS: set
HOMEBREW_NO_INSECURE_REDIRECT: set
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0.5 build 1205
Git: 2.32.0 => /opt/homebrew/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.5.1-arm64
CLT: 12.5.1.0.1.1623191612
Xcode: N/A
Rosetta 2: false
$ brew doctor
Your system is ready to brew.

• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

I work at EFF on Certbot which is packaged in Homebrew. Due to this problem in augeas which is one of Certbot's dependencies, many of our users are reporting problems using Certbot on Big Sur. This is my attempt to report the root problem and I apologize that these instructions are somewhat convoluted.

What happened (include all command output)?

To reproduce the problem, you can run:

python3 -m venv venv
. ./venv/bin/activate
pip install python-augeas
python break.py

where break.py is:

from augeas import Augeas
a = Augeas(flags=Augeas.NONE | Augeas.NO_MODL_AUTOLOAD)
a.set('/augeas/load/Httpd/lens', 'Httpd.lns')
a.load()
print(a.get('/augeas/load/Httpd/error'))

The output of the last command is:

/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/util.aug:115.2-123.51:Failed to compile comment_multiline
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/util.aug:116.20-.69:exception: Regular expression subtraction 'r1 - r2' failed
    r1: /[^ 	
].*[^ 	
]|[^ 	
]/
    r2: /.*\*/.*/

/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:78.26-.37:Could not load module Util for Util.indent
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:78.26-.37:Undefined variable Util.indent
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:168.4-.10:Undefined variable indent
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:177.13-.19:Undefined variable indent
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:182.21-.27:Undefined variable indent
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:187.18-.25:Undefined variable section
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:187.26-.44:Undefined variable directive
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:190.10-.18:Undefined variable content
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:204.13-.25:Undefined variable Util.stdexcl
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:206.20-.23:Undefined variable lns
/opt/homebrew/Cellar/augeas/1.12.0/share/augeas/lenses/dist/httpd.aug:206.24-.30:Undefined variable filter

What did you expect to happen?

python break.py in the above section should just output None. This is the behavior you should see on versions of macOS other than Big Sur or if you install augeas on Big Sur using brew install augeas --head. I've reproduced this problem both on Intel and M1 Macs running Big Sur. The error you see in my output above is Augeas failing to parse its own lenses.

I believe the root cause here is that the configure script in the latest Augeas release doesn't properly handle the macOS version number change from 10.x to 11. I reported this problem upstream a month ago at hercules-team/augeas#729 and received no response. At least until Augeas does another release, I'd love it if Homebrew would patch its copy of the configure script to properly set the compiler flags to avoid this problem and to fix the issue in the Augeas bottles for Big Sur.

Step-by-step reproduction instructions (by running brew commands)

brew install augeas
brew install python
# Then run the commands in the "What happened" section

[SMillerDev]

You should create a pull request for Augeas and then also add a patch block to the homebrew formula. See https://docs.brew.sh/Updating-Software-in-Homebrew for more information on the latter

[bmw]

Digging into this a bit more, the real root problem here is in GNU's libtool. The files it generates do not properly set compiler flags for macOS 11+. The issues for this in their bug tracker are https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44684 and https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44605 which I just bumped. Homebrew is maintaining a patch for this at https://github.com/Homebrew/formula-patches/blob/50a00afbf4549fbc0ffd3855c884f7d045cf4f93/libtool/dynamic_lookup-11.patch.

With this in mind, would you all accept a similar patch modifying Augeas' libtool.m4 and configure files? This would only need to be done for stable releases as when building from head, Homebrew's patched version of libtool is used avoiding the problem. The issue is that Homebrew's libtool patch hasn't made its way upstream so the configure script included in Augeas releases has a bug.

For what it's worth, I would expect other Homebrew formula using configure scripts from upstream to have similar problems while libtool remains unpatched. I found a few patches for other Homebrew formula doing similar things, but no where near as many as I would expect.

[bmw]

Any thoughts on the above proposal? A patch implementing this would look something like:

diff --git a/Formula/augeas.rb b/Formula/augeas.rb
index 1652af1530..af540583dd 100644
--- a/Formula/augeas.rb
+++ b/Formula/augeas.rb
@@ -23,17 +23,20 @@ class Augeas < Formula
   head do
     url "https://github.com/hercules-team/augeas.git"
 
-    depends_on "autoconf" => :build
-    depends_on "automake" => :build
     depends_on "bison" => :build
     depends_on "libtool" => :build
   end
 
+  depends_on "autoconf" => :build
+  depends_on "automake" => :build
   depends_on "pkg-config" => :build
   depends_on "readline"
 
   uses_from_macos "libxml2"
 
+  # TODO: Patch is only required on stable
+  patch :DATA
+
   def install
     args = %W[--disable-debug --disable-dependency-tracking --prefix=#{prefix}]
 
@@ -57,3 +60,54 @@ class Augeas < Formula
     system bin/"augtool", "print", etc
   end
 end
+__END__
+diff --git a/configure b/configure
+index 4b8fb40..b8b62da 100755
+--- a/configure
++++ b/configure
+@@ -11039,16 +11039,11 @@ $as_echo "$lt_cv_ld_force_load" >&6; }
+       _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
+     darwin1.*)
+       _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
+-    darwin*) # darwin 5.x on
+-      # if running on 10.5 or later, the deployment target defaults
+-      # to the OS version, if on x86, and 10.4, the deployment
+-      # target defaults to 10.4. Don't you love it?
+-      case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
+-	10.0,*86*-darwin8*|10.0,*-darwin[91]*)
+-	  _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
+-	10.[012][,.]*)
++    darwin*)
++      case ${MACOSX_DEPLOYMENT_TARGET},$host in
++        10.[[012]],*|,*powerpc*)
+ 	  _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
+-	10.*)
++        *)
+ 	  _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
+       esac
+     ;;
+diff --git a/gnulib/m4/libtool.m4 b/gnulib/m4/libtool.m4
+index a644432..c8a4d45 100644
+--- a/gnulib/m4/libtool.m4
++++ b/gnulib/m4/libtool.m4
+@@ -1067,16 +1067,11 @@ _LT_EOF
+       _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
+     darwin1.*)
+       _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
+-    darwin*) # darwin 5.x on
+-      # if running on 10.5 or later, the deployment target defaults
+-      # to the OS version, if on x86, and 10.4, the deployment
+-      # target defaults to 10.4. Don't you love it?
+-      case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
+-	10.0,*86*-darwin8*|10.0,*-darwin[[91]]*)
+-	  _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
+-	10.[[012]][[,.]]*)
++    darwin*)
++      case ${MACOSX_DEPLOYMENT_TARGET},$host in
++       10.[[012]],*|,*powerpc*)
+ 	  _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
+-	10.*)
++	*)
+ 	  _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
+       esac
+     ;;

If a patch like this would be accepted, I'd implement the TODO and am happy to put the patch outside of the formula if that is desired.

[SMillerDev]

Please suggest formula changes as a pull request. It's really much easier for us to judge the effects that way.

[bmw]

PR is #84083.