Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable linking multiple Python formulae #107517

Closed
wants to merge 10 commits into from

Conversation

carlocab
Copy link
Member

@carlocab carlocab commented Aug 7, 2022

  • python@3.9: use altinstall target
  • python@3.10: link into HOMEBREW_PREFIX
  • python@3.9: cleanup
  • python@3.10: cleanup
  • Update Python aliases
  • versioned_keg_only_allowlist: add Python 3.10

The primary change (see first commit) uses the altinstall target in the Python Makefile to make sure that Python3.9 only installs files that contain Python's major and minor version. That is, python@3.9 will now install only python3.9, python3.9-config, etc., instead of python3 and python3-config.

This will allow users to have both python@3.9 and python@3.10 linked into HOMEBREW_PREFIX simultaneously. Users who wish to call python3.9 as python3 can add libexec/"bin" to their PATH.

Going forward, I believe this will help simplify migrating to newer Python versions, since there will be no need to make the newer version keg-only while most dependent formulae still depend on the older version of Python.

I've made a lot of changes here. For ease of review, I placed substantive changes in the first two commits. The third and fourth commits are refactoring, and the last two fifth and sixth update non-formula files.

I have not tested this on Linux. This may require a few more adjustments there.

See discussion at Homebrew/discussions#2776 and documentation at https://github.com/python/cpython#installing-multiple-versions.

Closes #87075.


  • Have you followed the guidelines for contributing?
  • Have you ensured that your commits follow the commit style guide?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

@BrewTestBot BrewTestBot added the automerge-skip `brew pr-automerge` will skip this pull request label Aug 7, 2022
@carlocab carlocab mentioned this pull request Aug 7, 2022
6 tasks
@carlocab carlocab requested a review from a team August 7, 2022 06:26
@carlocab carlocab force-pushed the link-all-pythons branch 6 times, most recently from a56c57e to 6598029 Compare August 7, 2022 09:22
Formula/python@3.10.rb Outdated Show resolved Hide resolved
@carlocab carlocab force-pushed the link-all-pythons branch 2 times, most recently from b19b708 to 5a5f61a Compare August 7, 2022 09:36
Formula/python@3.10.rb Outdated Show resolved Hide resolved
Formula/python@3.9.rb Outdated Show resolved Hide resolved
Formula/python@3.10.rb Outdated Show resolved Hide resolved
@carlocab

This comment was marked as resolved.

@carlocab carlocab added long build Needs CI-long-timeout CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. CI-linux-self-hosted Build on Linux self-hosted runner labels Aug 7, 2022
Copy link
Member

@Rylan12 Rylan12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this mean that if you run brew install python@3.9 (no alias so no auto upgrade to python@3.10) and then have a script that relies on /opt/homebrew/bin/python3 it will no longer work unless you manually install python@3.10?

@carlocab
Copy link
Member Author

carlocab commented Aug 7, 2022

Doesn't this mean that if you run brew install python@3.9 (no alias so no auto upgrade to python@3.10) and then have a script that relies on /opt/homebrew/bin/python3 it will no longer work unless you manually install python@3.10?

Yes. But the same thing happens whenever we make python@3.9 keg-only and link python@3.10, so the way we've been handling this so far has the same problem.

@carlocab carlocab added do not merge CI-skip-recursive-dependents Pass --skip-recursive-dependents to brew test-bot. CI-skip-dependents Pass --skip-dependents to brew test-bot. and removed CI-linux-self-hosted Build on Linux self-hosted runner long build Needs CI-long-timeout labels Aug 8, 2022
Comment on lines +456 to +465
If you do not need a specific version of Python, and always want Homebrew's `python3` in your PATH:
brew install python3
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Rylan12 this should help a bit with users who really just want python3 (of whatever version) but did brew install python@3.9.

It won't make it go away, since not everyone reads the caveats, but it should help a bit.

Formula/python@3.10.rb Outdated Show resolved Hide resolved
@carlocab carlocab marked this pull request as ready for review August 11, 2022 10:10
@carlocab
Copy link
Member Author

carlocab commented Aug 11, 2022

This is ready for review.

Some dependents will still fail, but all failures should have fixes in progress. See #107517 (comment).

I hope to be able to merge this once CI completes.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. One small question.

link_overwrite "Frameworks/Python.framework/Headers"
link_overwrite "Frameworks/Python.framework/Python"
link_overwrite "Frameworks/Python.framework/Resources"
link_overwrite "Frameworks/Python.framework/Versions/Current"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean whichever version is linked last will be Current?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I contemplated that, but found it too confusing, so decided against it.

In this PR, only the newest Python in Homebrew/core (i.e. python@3.10) will have these symlinks.

I would like to find a way to do it so that the newest Python you have installed will have these symlinks, so that the behaviour is consistent and predictable, but im punting on that for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I contemplated that, but found it too confusing, so decided against it.

Agreed, I think it's better this way.

In this PR, only the newest Python in Homebrew/core (i.e. python@3.10) will have these symlinks.

How do we ensure these get updated/migrated on a new Python version? Audit? Could do something like fail if the alias doesn't point to python.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we ensure these get updated/migrated on a new Python version? Audit? Could do something like fail if the alias doesn't point to python.

Yup, we should add an audit for this. Will look at one soon.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carlocab Sounds great, doesn't need to block this PR 👍🏻

@carlocab carlocab added CI-long-timeout Use longer GitHub Actions CI timeout. and removed do not merge labels Aug 11, 2022
carlocab added a commit to carlocab/homebrew-core that referenced this pull request Aug 11, 2022
This is a backport of Homebrew#107517 to `python@3.8`.
@cho-m cho-m removed the CI-long-timeout Use longer GitHub Actions CI timeout. label Aug 12, 2022
@BrewTestBot
Copy link
Member

:shipit: @carlocab has triggered a merge.

@carlocab carlocab deleted the link-all-pythons branch August 12, 2022 04:26
@carlocab carlocab mentioned this pull request Aug 12, 2022
6 tasks
BrewTestBot pushed a commit that referenced this pull request Aug 12, 2022
This is a backport of #107517 to `python@3.8`.
carlocab added a commit to carlocab/homebrew-core that referenced this pull request Aug 12, 2022
This is a backport of Homebrew#107517 to `python@3.7`.
carlocab added a commit to carlocab/homebrew-core that referenced this pull request Aug 12, 2022
BrewTestBot pushed a commit that referenced this pull request Aug 12, 2022
This is a backport of #107517 to `python@3.7`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge-skip `brew pr-automerge` will skip this pull request CI-linux-self-hosted Build on Linux self-hosted runner CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. long build Needs CI-long-timeout python-3.10-migration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants