Issue installing 'six' or any packages depending on 'six'

[brandon-leapyear]

brew gist-logs <formula> link OR brew config AND brew doctor output

brew config:
HOMEBREW_VERSION: 3.1.5
ORIGIN: https://github.com/Homebrew/brew
HEAD: 227445ece9414a82aa8929b91ca2e0cca28a561b
Last commit: 4 months ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: c2de1c11c1e9e48d0956ffcf9823a9c84089fcde
Core tap last commit: 4 months ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 4
Homebrew Ruby: 2.6.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/bin/ruby
CPU: quad-core 64-bit ivybridge
Clang: 12.0 build 1200
Git: 2.29.2 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 10.15.5-x86_64
CLT: 12.3.0.0.1.1607026830
Xcode: 12.3 => /Applications/Xcode-12.3.app/Contents/Developer

brew doctor:
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: A newer Command Line Tools release is available.
Update them from Software Update in System Preferences or run:
  softwareupdate --all --install --force

If that doesn't show you any updates, run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/more/.
You should download the Command Line Tools for Xcode 12.4.


Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  six

Warning: Your Xcode (12.3) is outdated.
Please update to Xcode 12.4 (or delete it).
Xcode can be updated from the App Store.

Warning: Broken symlinks were found. Remove them with `brew cleanup`:
  /usr/local/share/zsh/site-functions/_brew_cask

• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

brew install awscli in CI, to install the AWS CLI. The error was also reproduced on a local developer machine, but I'm not sure how reproducible it is locally.

What happened (include all command output)?

I created a brand new Circle CI job running in Mac and ran a single command: brew install six

#!/bin/bash --login -eo pipefail
brew install six
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 4 taps (homebrew/cask-versions, homebrew/core, homebrew/cask and adoptopenjdk/openjdk).
==> New Formulae
alda                       influxdb-cli               poppler-qt5
alerter                    ipinfo-cli                 principalmapper
amfora                     iredis                     procps
apt                        jello                      proj@7
argocd-autopilot           joplin-cli                 psalm
argocd-vault-plugin        jrsonnet                   px
at-spi2-atk                julia                      py-spy
at-spi2-core               kalker                     pydocstyle
atuin                      ki                         pyflow
autoconf@2.69              kubeconform                pyoxidizer
avahi                      lexbor                     pyright
aws-vault                  libaec                     python-launcher
baidupcs-go                libfuse@2                  pythran
bash_unit                  libmobi                    pywhat
bgpq4                      licensefinder              qthreads
bosh-cli                   lima                       range2cidr
buildpulse-test-reporter   linux-headers@4.15         rhit
bupstash                   lsix                       rmw
cadical                    macos-term-size            ronn
caire                      mailcatcher                rover
cidr2range                 mariadb@10.5               rpg-cli
ciphey                     marp-cli                   rsc_2fa
clarinet                   mathlibtools               s4cmd
clazy                      matterbridge               samba
code-minimap               maturin                    scorecard
conmon                     mbedtls@2                  scotch
cruft                      moar                       scry
crun                       mongocli                   search-that-hash
darglint                   mongosh                    simde
ddcctl                     multi-git-status           singularity
demumble                   nbsdgames                  slides
detect-secrets             ncc                        slirp4netns
doc8                       neovim-remote              soapyrtlsdr
docuum                     newrelic-infra-agent       soapysdr
ehco                       nomino                     spectra
eigenpy                    notcurses                  spot
elan-init                  ns-3                       sql-lint
elfutils                   oksh                       sqlbench
erlang@23                  onedrive                   sqlx-cli
fanyi                      openexr@2                  sqsmover
fcp                        openfpgaloader             storj-uplink
firefoxpwa                 openj9                     stylua
fluid-synth@2.1            openliberty-jakartaee8     svgbob
frum                       openliberty-microprofile4  tbb@2020
func-e                     openliberty-webprofile8    terminator
fuse-overlayfs             opensearch                 tmuxp
gcc@10                     opensearch-dashboards      tracker
git-xargs                  osinfo-db                  trojan-go
gitbackup                  osinfo-db-tools            umple
gitwatch                   ots                        universal-ctags
gpg-tui                    pari-elldata               vala-language-server
gradle@6                   pari-galdata               virtualenvwrapper
graphqurl                  pari-galpol                wasmtime
grepip                     pari-seadata               waypoint
gtksourceview5             pari-seadata-big           wildmidi
hcl2json                   parquet-cli                xauth
himalaya                   pgxnclient                 xfig
hubble                     php-cs-fixer@2             xsel
i2c-tools                  pillow                     zellij
imath                      plow                       zet
==> Updated Formulae
Updated 4707 formulae.
==> Renamed Formulae
badtouch -> authoscope                   prestosql -> trino
envoy@1.17 -> envoy@1.18                 weboob -> woob
grakn -> typedb                          wxmac -> wxwidgets
linux-headers -> linux-headers@4.4       wxmac@3.0 -> wxwidgets@3.0
parallelstl -> onedpl
==> Deleted Formulae
aurora-cli                               protobuf-swift
erlang@20                                protobuf@3.7
giter8                                   terraform-provisioner-ansible
libinfinity                              tj
osquery
==> New Casks
8x8-work                                 microsoft-openjdk11
adobe-dng-converter                      midi-router-client
adrive                                   midiview
affinity-designer                        millie
affinity-photo                           mixed-in-key
affinity-publisher                       moebius
android-commandlinetools                 mutesync
arduino-ide-beta                         odbc-manager
arduino-ide-nightly                      openloco
asana                                    p4
assinador-serpro                         physics-101
beeper                                   picoscope-beta
betterdiscord-installer                  pktriot
bit-fiddle                               polkadot-js
blackhole-64ch                           portx
blobsaver                                prisma-studio
burp-suite-professional                  privileges
carbon-copy-cloner5                      pronterface
castr                                    pulse
cinc-workstation                         qudedup-extract-tool
citra-nightly                            radar
classroom-mode-for-minecraft             recut
clay                                     redis-pro
clock-signal                             royal-tsx-beta
crescendo                                safe-exam-browser
cryptonomic-galleon                      safe-multisig
depthmapx                                samsung-dex
devkinsta                                sbarex-qlmarkdown
diagnostics                              shield
disk-expert                              shimonote
dmidiplayer                              shortcutor
dropzone                                 shottr
duplicate-file-finder                    siyuan
ears                                     sleek
enclave                                  sonic-robo-blast-2
final-fantasy-xiv-online                 sonic-robo-blast-2-kart
flomo                                    sparrow
fluent-reader                            specter
fork-dev                                 sqlight
foxglove-studio                          stork
futurerestore-gui                        suuntodm5
geogebra5                                tabby
goneovim                                 tabtopus
gosign                                   telegram-desktop-beta
guilded                                  temurin
hightop                                  temurin11
hush                                     temurin8
hyperkey                                 tint
ilspy                                    tinymediamanager3
instatus-out                             touch-portal
irpf2021                                 touchosc
itraffic                                 transfer
jamkazam                                 trezor-suite
jellyfin-media-player                    ubports-installer
jgrennison-openttd                       unexpectedly
jidusm                                   universal-battle
jiohome                                  usbimager
kdocs                                    usr-sse2-rdm
keyboardholder                           vamiga
kubenav                                  vial
lagrange                                 vitals
landrop                                  vitalsigns
leapp                                    vsd-viewer
librewolf                                vsdx-annotator
lightwright                              vym
maccleaner-pro                           waltr-pro
macfuse-dev                              wezterm
macstroke                                wifi-explorer-pro
maestral                                 wing-personal
mambaforge                               wolfram-player
maxon                                    xampp7
mem                                      yippy
memory-cleaner                           youtube-dl-gui
menu-bar-splitter                        zebra2
microsoft-openjdk
==> Updated Casks
Updated 1789 casks.
==> Deleted Casks
3cxphone                                 mapillary-uploader
adafruit-arduino                         meshcommander
adobe-lens-profile-creator               monitorearth
adventure                                netbeans-php
adware-removal-tool                      nndd
anytrans                                 nure
anytrans-for-android                     opennx
appstudio                                opera-mail
arduino-beta                             pdftotext
audiobookbinder                          phonebrowse
auristor-client                          phoneclean
baiducloud                               phonerescue
beautune                                 phonetrans
beoplay-software-update                  postgres-beta
blue-jeans-browser-plugin                printrun
caramba-switcher                         resxtreme
cellery                                  revisions
cricut-design-space                      rss
crypt                                    shoes
darktable-dev                            spectrum
deadbeef                                 spideroak-share
duckstation                              superbeam
family-tree-builder                      swift-explorer
filedrop                                 telegram-desktop-dev
foreman                                  terminus
gitbox                                   tla-plus-toolbox-nightly
hex                                      todos
hubic                                    touch-bar-pong
imobie-m1-app-checker                    vrep
iograph                                  wakeonlan
itrash                                   wingpersonal
kap-beta                                 wraparound
macclean                                 xamarin

==> Downloading https://ghcr.io/v2/homebrew/core/six/manifests/1.16.0_1
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/six/blobs/sha256:6068e58ff59ea7
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sh
######################################################################## 100.0%
==> Pouring six--1.16.0_1.catalina.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink lib/python3.9/site-packages/six.py
Target /usr/local/lib/python3.9/site-packages/six.py
already exists. You may want to remove it:
  rm '/usr/local/lib/python3.9/site-packages/six.py'

To force the link and overwrite all conflicting files:
  brew link --overwrite six

To list all files that would be deleted:
  brew link --overwrite --dry-run six

Possible conflicting files are:
/usr/local/lib/python3.9/site-packages/six.py
==> Summary
🍺  /usr/local/Cellar/six/1.16.0_1: 20 files, 122.7KB

Exited with code exit status 1

What did you expect to happen?

If six.py already exists, brew install six should not error.

Also, six should not install /usr/local/lib/python3.8 and /usr/local/lib/python3.7. I don't want multiple versions of Python on my machine that I'm not using.

Step-by-step reproduction instructions (by running brew commands)

brew install six

[SMillerDev]

Core tap last commit: 4 months ago

This is very likely your issue. Run brew update, brew upgrade and try again.

[brandon-leapyear]

Nope, still failed

[alebcay]

Also, six should not install /usr/local/lib/python3.8 and /usr/local/lib/python3.7. I don't want multiple versions of Python on my machine that I'm not using.

On this point specifically, six does not install those other versions of python. The six package on Homebrew does however contain the six Python package for three different versions of Python (3.{7,8,9}). If you don't have those older versions of Python installed via Homebrew, the extra Python packages provided here don't do anything.

On the broader issue - is Python already present in the environment? Is six already installed? Homebrew won't overwrite files that already exist in the installation location. This is the behavior for all formulae in Homebrew, not just six.

It just so happens that the Homebrew package for six may conflict with six provided from pip, hence the issue you're seeing.

[brandon-leapyear]

On this point specifically, six does not install those other versions of python. The six package on Homebrew does however contain the six Python package for three different versions of Python (3.{7,8,9}). If you don't have those older versions of Python installed via Homebrew, the extra Python packages provided here don't do anything.

No? If I do rm -rf /usr/local/lib/python3.7 and then brew install six, I get that directory back.

On the broader issue - is Python already present in the environment? Is six already installed? Homebrew won't overwrite files that already exist in the installation location. This is the behavior for all formulae in Homebrew, not just six.

Yes, that makes sense. Is /usr/local/lib/python3.9 the version of Python that comes with Macs by default? Why is the Homebrew six package installing there, then? Shouldn't the Homebrew six package only be installed for the Homebrew python installation?

[alebcay]

No? If I do rm -rf /usr/local/lib/python3.7 and then brew install six, I get that directory back.

There shouldn't be anything in there except stuff pertaining to six. Keep in mind that python@3.7 looks there for packages, but python@3.7 doesn't actually live there.

Is /usr/local/lib/python3.9 the version of Python that comes with Macs by default?

No (at least for me, on Intel Big Sur).

❯ /usr/bin/python3 --version
Python 3.8.2

My guess is that CircleCI has some other stuff loaded into their environment by default, and your local developer machine may have had pip install six in the past at some point.

Why is the Homebrew six package installing there, then?

Because the default install prefix for Homebrew on Intel Macs is /usr/local.

Shouldn't the Homebrew six package only be installed for the Homebrew python installation?

Right now there's no good way to separate Homebrew-managed packages from pip-managed packages (assuming that pip is running with Homebrew Python). We're working on improving this.

[brandon-leapyear]

I see. So I guess in the past, when I installed awscli manually myself with sudo pip3 install awscli, it installed /usr/local/lib/python3.9/site-packages/six.py, and now, six is still installed there and brew complains. So fixing this is just manually removing six.py and having brew manage the installation of six (with any globally pip-installed packages using the brew-managed six now). Is that right?

[alebcay]

That's correct.

[brandon-leapyear]

Great; I think this can be closed then. Thanks for your help!