s3cmd unable to import Python dependencies

[collinhundley]

brew gist-logs <formula> link OR brew config AND brew doctor output

HOMEBREW_VERSION: 4.1.17-6-g64d309c
ORIGIN: https://github.com/Homebrew/brew
HEAD: 64d309c4cf3a199a2c895980abc10b8bd53b9cbc
Last commit: 6 hours ago
Core tap JSON: 24 Oct 17:21 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 10
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 10-core 64-bit arm_firestorm_icestorm
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 8.1.2 => /usr/bin/curl
macOS: 14.0-arm64
CLT: N/A
Xcode: 15.0
Rosetta 2: false

Your system is ready to brew.

Verification

• My "brew doctor output" says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.
• I searched for recent similar issues at https://github.com/Homebrew/homebrew-core/issues?q=is%3Aissue and found no duplicates.

What were you trying to do (and why)?

Attempting to run s3cmd produces an error that it is unable to import dateutil.parser.

What happened (include all command output)?

The following error is logged when trying to run s3cmd:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
ImportError trying to import dateutil.parser.
Please install the python dateutil module:
$ sudo apt-get install python-dateutil
  or
$ sudo yum install python-dateutil
  or
$ pip install python-dateutil
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

What did you expect to happen?

Python resources should be installed by the formula. It does appear that virtualenv is installing these resources here: /opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/

Both dateutil and magic can be seen in the filesystem at this location. However, for some reason s3cmd is unable to import the modules.

Step-by-step reproduction instructions (by running brew commands)

# Install s3cmd:
brew install s3cmd

# Attempt to run:
s3cmd

[branchvincent]

Yea that's strange, can you paste the following info? Here's what I see:

$ head -1 $(which s3cmd)
#!/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/bin/python

$ $(brew --prefix s3cmd)/libexec/bin/python -c 'import dateutil; print(dateutil.__path__)'
['/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/dateutil']

$ $(brew --prefix s3cmd)/libexec/bin/python -m site
sys.path = [
    '/opt/homebrew/Library/Taps/homebrew/homebrew-core',
    '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python312.zip',
    '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12',
    '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload',
    '/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages',
    '/opt/homebrew/opt/six/lib/python3.12/site-packages',
    '/opt/homebrew/lib/python3.12/site-packages',
    '/opt/homebrew/Cellar/pybind11/2.11.1/libexec/lib/python3.12/site-packages',
]
USER_BASE: '/Users/branch/Library/Python/3.12' (doesn't exist)
USER_SITE: '/Users/branch/Library/Python/3.12/lib/python/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

[collinhundley]

@branchvincent here's what I get:

$ head -1 $(which s3cmd)
#!/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/bin/python

$ $(brew --prefix s3cmd)/libexec/bin/python -c 'import dateutil; print(dateutil.__path__)'
['/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/dateutil']

$ $(brew --prefix s3cmd)/libexec/bin/python -m site
sys.path = [
    '/opt/homebrew/Cellar',
    '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python312.zip',
    '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12',
    '/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload',
    '/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages',
    '/opt/homebrew/opt/six/lib/python3.12/site-packages',
    '/opt/homebrew/lib/python3.12/site-packages',
    '/opt/homebrew/opt/six/lib/python3.12/site-packages',
    '/opt/homebrew/opt/six/lib/python3.12/site-packages',
]
USER_BASE: '/Users/collin/Library/Python/3.12' (doesn't exist)
USER_SITE: '/Users/collin/Library/Python/3.12/lib/python/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

The first two outputs are identical to yours, but the last one is a little different.

Just to confirm - are you able to reproduce the issue? I'm seeing the same result on two different machines.

[branchvincent]

No I can't reproduce, and your output looks fine to me. I mainly wanted to see if import dateutil was being clobbered by another package and or if the venv's site-packages /opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages wasn't in sys.path for some reason.

The import error you're getting is specifically from import dateutil.parser here, what happens when you try that?

$ $(brew --prefix s3cmd)/libexec/bin/python -c 'import dateutil.parser as p; print(p.__path__)'
['/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/dateutil/parser']

[collinhundley]

I get this error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/dateutil/parser/__init__.py", line 2, in <module>
    from ._parser import parse, parser, parserinfo, ParserError
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/dateutil/parser/_parser.py", line 43, in <module>
    from six import integer_types, text_type
ImportError: cannot import name 'integer_types' from 'six' (unknown location)

Edit:

I'm not familiar with the six module, but this directory from sys.path does not exist on my machine: /opt/homebrew/opt/six/lib/python3.12/site-packages.

Instead, it's /opt/homebrew/opt/six/1.16.0_3/lib/python3.xx/site-packages - but python3.12 is not present (only 3.9/10/11).

[branchvincent]

Interesting, your six seems out of date then. can you brew reinstall six

[collinhundley]

Reinstalled six and the issue is fixed.

So it looks like if six is already installed, but is out of date, Homebrew won't update it during the installation of s3cmd. Is this a bug?

[chenrui333]

this might be the root cause, Homebrew/brew#16152

also relates to #151657

[cho-m]

Closing as Homebrew/brew#16152 is merged and in release.