How to make extensions available to hg binary

[LucianU]

Output of brew config

HOMEBREW_VERSION: 4.4.22
ORIGIN: https://github.com/Homebrew/brew
HEAD: 89d1d6b8f4adb0b8f6678912f648dca2475b7c7c
Last commit: 25 hours ago
Branch: stable
Core tap JSON: 25 Feb 09:50 UTC
Core cask tap JSON: 25 Feb 09:50 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 3.3.7 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.7/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 16.0.0 build 1600
Git: 2.39.5 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.3.1-arm64
CLT: 16.2.0.0.1.1733547573
Xcode: 16.2
Rosetta 2: false

Output of brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed casks are deprecated or disabled.
You should find replacements for the following casks:
  kindle

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  node@18
  ruff-lsp

Warning: /usr/bin occurs before /opt/homebrew/bin in your PATH.
This means that system-provided programs will be used instead of those
provided by Homebrew. Consider setting your PATH so that
/opt/homebrew/bin occurs before /usr/bin. Here is a one-liner:
  echo 'export PATH="/opt/homebrew/bin:$PATH"' >> /Users/lucian/.bash_profile

The following tools exist at both paths:
  openssl
  pip3
  pp
  python3

Warning: Homebrew's "sbin" was not found in your PATH but you have installed
formulae that put executables in /opt/homebrew/sbin.
Consider setting your PATH for example like so:
  echo 'export PATH="/opt/homebrew/sbin:$PATH"' >> /Users/lucian/.bash_profile

Description of issue

I need to migrate a repo from Mercurial to git and I installed Mercurial with Homebrew.
As far as I can see, the Mercurial binary uses a version of Python provided by Homebrew. How do I make available some Python packages (like the hg-git extension for Mercurial) to this version of Python?

[p-linnane]

We follow PEP 668.:

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.

    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:

    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz

    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with

    brew install pipx

    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.

    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.

    Read more about this behavior here: <https://peps.python.org/pep-0668/>

[LucianU]

It seems there was an issue with my environment or of some other nature.

The project I wanted to convert is a Python project, so I used poetry to install dulwich and hg-git. But when I ran hg (which was installed with brew), it couldn't find these modules, even when the virtual environment was active.

Now, when I tried again, I could import dulwich and hg-git in the Python provided by the environment, but it couldn't find the mercurial Python package. This makes sense since the virtualenv is isolated from global packages. Once I added that to the virtualenv, the issue was fixed.

---

To address specifically your suggestions, how can the Python binary that runs a program installed with brew "see" packages from a virtualenv? For example, my hg binary installed with brew has this on the first line: #!/opt/homebrew/opt/python@3.13/bin/python3.13

In case you're wondering why this is needed, some Python programs use extensions which don't come with the program, so they need to be made accessible somehow.

[gromgit]

my hg binary installed with brew has this on the first line: #!/opt/homebrew/opt/python@3.13/bin/python3.13

That puts it in Homebrew's externally-managed Python environment, so you could just pip3 install <hg_ext_module>, but of course that would "pollute" that environment.

In case you're wondering why this is needed, some Python programs use extensions which don't come with the program, so they need to be made accessible somehow.

I'm neither Mercurial nor a Pythonista, but the fine Mercurial docs say that you can drop an extension pretty much anywhere, and just refer to it in ${HOMEBREW_PREFIX}/etc/mercurial/hg, so it seems like you don't have to "pollute" the Homebrew Python environment.

I don't remember if the situation was discussed as to how to accommodate pip installing extensions to other Homebrew Python apps that were themselves packaged in a virtualenv (I think that's the majority of them). Homebrew maintainers can probably clarify this, but on the whole, I suspect the cleanest way is to avoid using the Homebrew-packaged app and use pipx instead, so something like:

pipx install myapp
pipx inject myapp extmodule

[LucianU]

I think you are right. I think the situation I've encountered is a software-development use case and my impression is that brew is not geared towards this. I don't know if the Homebrew maintainers want to address this explicitly or just leave it as it is. I think it's still useful to install things with brew if you want to use them without customisation, like quickly installing hg to use it in a one-off situation.

[gromgit]

https://docs.brew.sh/Homebrew-and-Python#pep-668-and-virtual-environments already alludes to that:

It is possible to install some Python packages as formulae by using brew install xyz. We do not recommend using these formulae and instead recommend you install them with pip inside a virtualenv. These system-wide Homebrew Python formulae are often Homebrew-specific formulae that are useful as dependencies for other Homebrew formulae. It is not recommended to rely on them.