Linuxbrew doesn't build an optimized LLVM (or potentially other CMake packages)

[chandlerc]

brew config output

HOMEBREW_VERSION: 3.1.9-64-geeef3f4
ORIGIN: https://github.com/Homebrew/brew
HEAD: eeef3f4fe7b9a5c1f8355d3867edf4aa0c294b00
Last commit: 8 hours ago
Core tap ORIGIN: https://github.com/Homebrew/linuxbrew-core
Core tap HEAD: 7cdfe2454bc0382eb22884439b04bc0b8ff70ebd
Core tap last commit: 3 hours ago
Core tap branch: fix-libcxx
HOMEBREW_PREFIX: /home/chandlerc/.linuxbrew
HOMEBREW_REPOSITORY: /home/chandlerc/.linuxbrew/Homebrew
HOMEBREW_CELLAR: /home/chandlerc/.linuxbrew/Cellar
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 96
Homebrew Ruby: 2.6.3 => /home/chandlerc/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/bin/ruby
CPU: 96-core 64-bit skylake
Clang: 12.0.0 build (parse error)
Git: 2.32.0 => /bin/git
Curl: 7.74.0 => /usr/bin/curl
Kernel: Linux 5.10.28 x86_64 GNU/Linux
OS: Debian GNU/Linux 11 (bullseye)
Host glibc: 2.31
/usr/bin/gcc: 10.2.1
/usr/bin/ruby: 2.7.3
glibc: N/A
gcc@5: N/A
xorg: N/A

brew doctor output

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and which additional flags to use when
compiling and linking.

Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew-provided
script of the same name. We found the following "config" scripts:
  /home/chandlerc/.cargo/bin/cargo-install-update-config
  /home/chandlerc/.pyenv/shims/python3.7m-config
  /home/chandlerc/.pyenv/shims/python-config
  /home/chandlerc/.pyenv/shims/python3-config
  /home/chandlerc/.pyenv/shims/python3.8-config
  /home/chandlerc/.pyenv/shims/pre-commit-validate-config
  /home/chandlerc/.pyenv/shims/python3.7-config
  /home/chandlerc/.pyenv/shims/python3.6m-config
  /home/chandlerc/.pyenv/shims/python3.9-config
  /home/chandlerc/.pyenv/shims/python3.6-config

Warning: Your Homebrew's prefix is not /home/linuxbrew/.linuxbrew.
Some of Homebrew's bottles (binary packages) can only be used with the default
prefix (/home/linuxbrew/.linuxbrew).
You will encounter build failures with some formulae.
Please create pull requests instead of asking for help on Homebrew's GitHub,
Twitter or any other official channels. You are responsible for resolving
any issues you experience while you are running this
unsupported configuration.


Warning: Some taps are not on the default git origin branch and may not receive
updates. If this is a surprise to you, check out the default branch with:
  git -C $(brew --repo homebrew/core) checkout master

• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

Installing LLVM (including Clang and libc++) for use in building C++ projects.

What happened (include all command output)?

The installation of LLVM, Clang, and all related tools are completely unoptimized.

Timing any significant build with the installed clang ran about 10x slower than expected.

What did you expect to happen?

The installed tools to be optimized.

This appears to be because CMake's default flags for release builds are overridden. Removing that fixes the issue. You can see what this looks like here:

chandlerc@578fe41

I'm happy to submit that as a pull request, but wanted to first check if this is the correct direction to take. I understand that Homebrew wants to override optimization levels in some cases, but it seems like the shims already provide the tools to do that, and removing these flags from CMake seems likely to leave ~all CMake built packages on Linux unoptimized.

Note that I did check that macOS is unaffected. Also it seems macOS is unaffected with the patch above applied.

Step-by-step reproduction instructions (by running brew commands)

brew install llvm
# compile some code with `clang++`, time it, observe it is slow on linux

[carlocab]

Could you try doing brew gist-logs llvm?

[chandlerc]

On Linux? Sure. It'll take a while. I've been staring at the logs for LInux for some time if there is a question I can just answer in the interim.

[carlocab]

Yes. Ideally without your patch in chandlerc@578fe41, but the logs from when you applied that would help too.

[chandlerc]

This is without that patched in: https://gist.github.com/chandlerc/457312cc1d86d78c5600ad9c5fbe1934

i'd like to remove it as soon as your done though, too many details in there for me to be comfortable leaving it up long-term.

I can get one with my patch applied if it helps. But each time I have to remove my complier and wait for 20-30 minutes to rebuild everything, so its somewhat disruptive to get these. I can get one for Mac, but that took >2 hours last time.

[carlocab]

Thanks. I've downloaded a copy of it so you can take it offline. I'll delete it when I'm done having a look.

[carlocab]

Hmm, ok, one thing I can see from here is that superenv should be filtering -O flags here

brew/Library/Homebrew/shims/super/cc

Lines 175 to 179 in eeef3f4

	when /^-g\d?$/, /^-gstabs\d+/, "-gstabs+", /^-ggdb\d?/,
	/^-march=.+/, /^-mtune=.+/, /^-mcpu=.+/,
	/^-O[0-9zs]?$/, "-fast", "-no-cpp-precomp",
	"-pedantic", "-pedantic-errors", "-Wno-long-double",
	"-Wno-unused-but-set-variable"

and then injecting a replacement (O2 on Linux, Os on Mac) in

brew/Library/Homebrew/shims/super/cc

Line 274 in eeef3f4

args << "-#{ENV["HOMEBREW_OPTIMIZATION_LEVEL"]}"

for some reason, this isn't happening on Linux (but should be happening on Mac).

I'll have a closer look to see if there are hints about why this is happening here...

[carlocab]

Yea I think applying chandlerc@578fe41 should be fine.

Ideally, we should fix the issue with the compiler shim on Linux, as I think this also has consequences for non-CMake builds. However, I don't see the issue with relying more heavily on CMake's default Release build flags.

CC @Bo98

[iMichka]

The optimisation level is set here for Linux:

brew/Library/Homebrew/extend/os/linux/extend/ENV/super.rb

Line 18 in 3476ca7

self["HOMEBREW_OPTIMIZATION_LEVEL"] = "O2"

[carlocab]

It might be that the code in extend/os/linux/extent/ENV/super.rb is never run. That might also explain the weird linkage you get from previous attempts to build libc++.

[chandlerc]

Turned my test commit into a PR here: #11467

Let me know what else I should do to help this land.

[Bo98]

The original reason for that flag was to prevent CMake passing -O3 so that we pass our own. In the superenv world this is less of a problem, so the only reason it exists really is because of stdenv.

The change should be no-op in superenv. If it isn't then there's a bit of a bigger bug that needs solving.

[carlocab]

If it isn't then there's a bit of a bigger bug that needs solving.

I think this is the case.

[Bo98]

It's probably because we use cmake --build, which will bypass our make shim. I can look into a cmake shim.

[carlocab]

It's probably because we use cmake --build, which will bypass our make shim. I can look into a cmake shim.

I don't think this is the problem, since it only affects Linux and not macOS. The -Os flag is being injected correctly on macOS, but -O2 is not on Linux.

[chandlerc]

Just to fully confirm, I could see the shims being used on macOS as well in the logs.

[carlocab]

Reopening this, as #11467 is only a partial solution specific to CMake builds. I think the compiler shims still need fixing on Linux.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.