Mercurial should use Homebrew python so it can support TLS 1.1+

[indygreg]

Bug reports:

The mercurial package currently installs a hg script with a shebang of #!/usr/bin/python.

Apple's Python (even on the latest beta of MacOS) doesn't support TLS 1.1+ and other modern crypto features. Homebrew's Python does support modern crypto because it is built/linked with a modern OpenSSL.

Mercurial 3.9 takes a much more strict approach to connection security and issues warnings when TLS 1.1+ cannot be used. See https://www.mercurial-scm.org/wiki/SecureConnections for more.

What this all means is that Mercurial 3.9 out of the box on Homebrew issues warnings when connecting to https:// URLs.

The issue can be fixed by having Mercurial use Homebrew's Python in its shebang. According to Mercurial's Makefile, this can be resolved by ensuring python resolves to the Homebrew Python 2.7 during installation.

[DomT4]

What has changed from the last time we had this discussion back in Homebrew/legacy-homebrew#42416?

[DomT4]

What has changed from the last time we had this discussion back in Homebrew/legacy-homebrew#42416?

[indygreg]

I forgot about that discussion!

What has changed is that Mercurial 3.9 now issues warnings when it can't use TLS 1.1+.

I also recently learned that /usr/bin/python on OS X doesn't support TLS 1.1+. So basically any [Homebrew] Python process using /usr/bin/python can't practice modern and safe security. This is a glaring security deficiency in Apple's Python. Fortunately Homebrew's Python can use TLS 1.1+. Not only do I think the Mercurial package should use Homebrew's Python, but I also think all Homebrew Python packages should be using Homebrew's Python so they can use modern security technology. Failure to do so leaves Homebrew user's more susceptible to security risks. e.g. TLS 1.0 has known vulnerabilities such as CRIME and BEAST. While there are mitigations available, the best mitigation is to negotiate TLS 1.1 or 1.2 and avoid the TLS 1.0 attack surface altogether. https://wiki.mozilla.org/Security/Server_Side_TLS has a nice overview of the state of TLS.

[indygreg]

I forgot about that discussion!

What has changed is that Mercurial 3.9 now issues warnings when it can't use TLS 1.1+.

I also recently learned that /usr/bin/python on OS X doesn't support TLS 1.1+. So basically any [Homebrew] Python process using /usr/bin/python can't practice modern and safe security. This is a glaring security deficiency in Apple's Python. Fortunately Homebrew's Python can use TLS 1.1+. Not only do I think the Mercurial package should use Homebrew's Python, but I also think all Homebrew Python packages should be using Homebrew's Python so they can use modern security technology. Failure to do so leaves Homebrew user's more susceptible to security risks. e.g. TLS 1.0 has known vulnerabilities such as CRIME and BEAST. While there are mitigations available, the best mitigation is to negotiate TLS 1.1 or 1.2 and avoid the TLS 1.0 attack surface altogether. https://wiki.mozilla.org/Security/Server_Side_TLS has a nice overview of the state of TLS.

[indygreg]

If Homebrew switches away from Apple's Python, it will also need to install a global hgrc pointing to Homebrew's CA certificates file. This is because Mercurial will no longer be able to use the special hacks in Apple's Python to load the OS X CA store.

The requirement for configuring the CA store in Mercurial packages is documented at the bottom of https://www.mercurial-scm.org/wiki/SecureConnections

[indygreg]

If Homebrew switches away from Apple's Python, it will also need to install a global hgrc pointing to Homebrew's CA certificates file. This is because Mercurial will no longer be able to use the special hacks in Apple's Python to load the OS X CA store.

The requirement for configuring the CA store in Mercurial packages is documented at the bottom of https://www.mercurial-scm.org/wiki/SecureConnections

[DomT4]

I forgot about that discussion!

Your name seemed vaguely familiar so I went and checked for prior mercurial issues in the old repo, heh.

Not only do I think the Mercurial package should use Homebrew's Python, but I also think all Homebrew Python packages should be using Homebrew's Python so they can use modern security technology.

This would be a hilariously big change to the way Homebrew's Python handling currently works. @tdsmith is our 🐍 expert, I'll leave it to Tim to comment on the issues you've raised here that relate to the Python elements specifically.

If Homebrew switches away from Apple's Python, it will also need to install a global hgrc pointing to Homebrew's CA certificates file.

If by global you mean $HOME we wouldn't be able to do that automatically; it'd violate our build sandbox. If you mean HOMEBREW_PREFIX/etc that would be manageable.

[DomT4]

I forgot about that discussion!

Your name seemed vaguely familiar so I went and checked for prior mercurial issues in the old repo, heh.

Not only do I think the Mercurial package should use Homebrew's Python, but I also think all Homebrew Python packages should be using Homebrew's Python so they can use modern security technology.

This would be a hilariously big change to the way Homebrew's Python handling currently works. @tdsmith is our 🐍 expert, I'll leave it to Tim to comment on the issues you've raised here that relate to the Python elements specifically.

If Homebrew switches away from Apple's Python, it will also need to install a global hgrc pointing to Homebrew's CA certificates file.

If by global you mean $HOME we wouldn't be able to do that automatically; it'd violate our build sandbox. If you mean HOMEBREW_PREFIX/etc that would be manageable.

[tdsmith]

This affects versions of Python < 2.7.9; El Capitan ships 2.7.10, so this is not a concern for users of recent operating systems. Concerned users should consider upgrading OS X. If Mercurial intends to support pre-2.7.9 versions of Python, Mercurial could add support for using PyOpenSSL instead of the stdlib's ssl module.

[tdsmith]

This affects versions of Python < 2.7.9; El Capitan ships 2.7.10, so this is not a concern for users of recent operating systems. Concerned users should consider upgrading OS X. If Mercurial intends to support pre-2.7.9 versions of Python, Mercurial could add support for using PyOpenSSL instead of the stdlib's ssl module.

[indygreg]

/usr/bin/python on El Capitan and on the latest beta of MacOS Sierra do not support TLS 1.1+. Python 2.7.9+ is not the mechanism by which you determine security support. (Another counterpoint: RHEL/CentOS 7 ship Python 2.7.5 but have backported ssl module goodies and support TLS 1.1+ along with SNI, etc.)

This demonstrates that El Capitan's Python does not support TLS 1.1+:

$ /usr/bin/python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.PROTOCOL_TLSv1_1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_1'

[indygreg]

/usr/bin/python on El Capitan and on the latest beta of MacOS Sierra do not support TLS 1.1+. Python 2.7.9+ is not the mechanism by which you determine security support. (Another counterpoint: RHEL/CentOS 7 ship Python 2.7.5 but have backported ssl module goodies and support TLS 1.1+ along with SNI, etc.)

This demonstrates that El Capitan's Python does not support TLS 1.1+:

$ /usr/bin/python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.PROTOCOL_TLSv1_1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_1'

[indygreg]

How will adding PyOpenSSL support to Mercurial improve things for Homebrew? Homebrew will install PyOpenSSL against the Homebrew Python. So Mercurial would need to use the Homebrew Python. At that point, it can just use the ssl module directly, since Homebrew's Python has a full featured ssl module with TLS 1.1+ support. (Or am I missing somewhere where Homebrew's Python may not support TLS 1.1+?)

[indygreg]

How will adding PyOpenSSL support to Mercurial improve things for Homebrew? Homebrew will install PyOpenSSL against the Homebrew Python. So Mercurial would need to use the Homebrew Python. At that point, it can just use the ssl module directly, since Homebrew's Python has a full featured ssl module with TLS 1.1+ support. (Or am I missing somewhere where Homebrew's Python may not support TLS 1.1+?)

[tdsmith]

Fair; thanks for the correction. 2.7.9+ is necessary but not sufficient: python also needs to be linked against a sufficiently new OpenSSL and the system Python is not.

tim@rocketman:lib-dynload$ otool -L /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_ssl.so 
_ssl.so:
    /usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
    /usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

Apple is being lobbied to link Python against a newer SSL library, so we'll see if anything changes in the Sierra builds.

How will adding PyOpenSSL support to Mercurial improve things for Homebrew? Homebrew will install PyOpenSSL against the Homebrew Python.

No, we can install PyOpenSSL against a generic 2.7 python; it will work with system python.

[tdsmith]

Fair; thanks for the correction. 2.7.9+ is necessary but not sufficient: python also needs to be linked against a sufficiently new OpenSSL and the system Python is not.

tim@rocketman:lib-dynload$ otool -L /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_ssl.so 
_ssl.so:
    /usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
    /usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

Apple is being lobbied to link Python against a newer SSL library, so we'll see if anything changes in the Sierra builds.

How will adding PyOpenSSL support to Mercurial improve things for Homebrew? Homebrew will install PyOpenSSL against the Homebrew Python.

No, we can install PyOpenSSL against a generic 2.7 python; it will work with system python.

[indygreg]

One can only hope that Apple will fix their Python in Sierra. I too have filed a rdar about the issue to add weight to the cause :)

But it seems to me that by waiting on Apple to maybe fix Python TLS support in Sierra is a bit... not ideal. Homebrew today is shipping packages running with ancient crypto. Shouldn't Homebrew take a more proactive approach to secure its users [by steering them away from Apple's Python]?

I understand that would be an invasive change to perform globally and your hands are probably tied. It's a tough position to be in :/

Can we at least have the conversation about switching the Mercurial package to use Homebrew's Python? Ideally, yes, Mercurial would support PyOpenSSL. However, the way I see it is that Mercurial works with a fully-configured ssl module. And we can get that if Homebrew switches Mercurial to Homebrew Python. That seems like an obvious, straightforward change. I'm not sure why there was/is resistance to it.

[indygreg]

One can only hope that Apple will fix their Python in Sierra. I too have filed a rdar about the issue to add weight to the cause :)

But it seems to me that by waiting on Apple to maybe fix Python TLS support in Sierra is a bit... not ideal. Homebrew today is shipping packages running with ancient crypto. Shouldn't Homebrew take a more proactive approach to secure its users [by steering them away from Apple's Python]?

I understand that would be an invasive change to perform globally and your hands are probably tied. It's a tough position to be in :/

Can we at least have the conversation about switching the Mercurial package to use Homebrew's Python? Ideally, yes, Mercurial would support PyOpenSSL. However, the way I see it is that Mercurial works with a fully-configured ssl module. And we can get that if Homebrew switches Mercurial to Homebrew Python. That seems like an obvious, straightforward change. I'm not sure why there was/is resistance to it.

[MikeMcQuaid]

Shouldn't Homebrew take a more proactive approach to secure its users [by steering them away from Apple's Python]?

It's worth noting you could make this argument for basically every system dependency we rely on. There are other package managers that take this approach but we do not.

[MikeMcQuaid]

Shouldn't Homebrew take a more proactive approach to secure its users [by steering them away from Apple's Python]?

It's worth noting you could make this argument for basically every system dependency we rely on. There are other package managers that take this approach but we do not.

[tdsmith]

I am warming to the idea that we should ship hg against Homebrew's python by default; OpenSSL 0.9.8 is real bad. This is a problem for mercurial more than other apps since a lot of web-touching stuff uses requests, which uses PyOpenSSL if it's available. I still think Mercurial should consider at least optionally using PyOpenSSL but I'm not volunteering to implement it. :) We can always leave an escape hatch for Python devs who are annoyed that we dump another python into $PATH.

[tdsmith]

I am warming to the idea that we should ship hg against Homebrew's python by default; OpenSSL 0.9.8 is real bad. This is a problem for mercurial more than other apps since a lot of web-touching stuff uses requests, which uses PyOpenSSL if it's available. I still think Mercurial should consider at least optionally using PyOpenSSL but I'm not volunteering to implement it. :) We can always leave an escape hatch for Python devs who are annoyed that we dump another python into $PATH.

[tdsmith]

cf #3548

[tdsmith]

cf #3548

[jaraco]

We can always leave an escape hatch for Python devs who are annoyed that we dump another python into $PATH.

I'd just like to chime in that while I'm annoyed by this security issue, I appreciate not having multiple versions of Python 2.7 on my system. I run Homebrew Mercurial and prefer not to install Homebrew Python.

[jaraco]

We can always leave an escape hatch for Python devs who are annoyed that we dump another python into $PATH.

I'd just like to chime in that while I'm annoyed by this security issue, I appreciate not having multiple versions of Python 2.7 on my system. I run Homebrew Mercurial and prefer not to install Homebrew Python.

[dimpase]

They haven't fixed Python in Sierra; they still bundle old (patched) openssl-0.9.8. Moreover, if you build new openssl from source and python from source, with the up to date openssl, to support TLS 1.2, you still have a problem of not having root certs available to python.

[dimpase]

They haven't fixed Python in Sierra; they still bundle old (patched) openssl-0.9.8. Moreover, if you build new openssl from source and python from source, with the up to date openssl, to support TLS 1.2, you still have a problem of not having root certs available to python.

[RokerHRO]

So I have to edit /usr/local/bin/hg manually so it uses /usr/local/bin/python2.7 instead of /usr/bin/python to get a secure connection to the hg repo server? Seriously?

I am very disappointed. :-( Perhaps I should switch to MacPorts, where it all runs out of the box. sigh

[RokerHRO]

So I have to edit /usr/local/bin/hg manually so it uses /usr/local/bin/python2.7 instead of /usr/bin/python to get a secure connection to the hg repo server? Seriously?

I am very disappointed. :-( Perhaps I should switch to MacPorts, where it all runs out of the box. sigh

[ilovezfs]

Our mercurial has a hard dependency on Homebrew Python now. If it's still using system Python, that would be a bug.

[ilovezfs]

Our mercurial has a hard dependency on Homebrew Python now. If it's still using system Python, that would be a bug.

[RokerHRO]

Since when? I just "brew update ; brew upgrade" before and my Homebrew hg client was not able to speak with our hg server.

I just got TLS 1.0 error messages when I want to pull from our hg repo at the server. And the shebang in /usr/local/bin/hg pointed to /usr/bin/python, that is the system's python, shipped by Apple, right?

Since I changed the shebang manually into /usr/local/bin/python2.7 my local hg client is able to talk with the hg repo server again. Ugly hack but #WorksForMe and I can continue to work. :-)

[RokerHRO]

Since when? I just "brew update ; brew upgrade" before and my Homebrew hg client was not able to speak with our hg server.

I just got TLS 1.0 error messages when I want to pull from our hg repo at the server. And the shebang in /usr/local/bin/hg pointed to /usr/bin/python, that is the system's python, shipped by Apple, right?

Since I changed the shebang manually into /usr/local/bin/python2.7 my local hg client is able to talk with the hg repo server again. Ugly hack but #WorksForMe and I can continue to work. :-)

[ilovezfs]

That's definitely a bug. See #3548.

[ilovezfs]

That's definitely a bug. See #3548.

[ilovezfs]

Looks like it was caused by f93b39e. I will fix it.

[ilovezfs]

Looks like it was caused by f93b39e. I will fix it.

[ilovezfs]

Fixed in #23282.

[ilovezfs]

Fixed in #23282.

[MikeMcQuaid]

Perhaps I should switch to MacPorts, where it all runs out of the box.

@RokerHRO As a general statement: this isn't a helpful or motivating threat to make and if MacPorts makes your life easier: yes, perhaps you should use it instead.

[MikeMcQuaid]

Perhaps I should switch to MacPorts, where it all runs out of the box.

@RokerHRO As a general statement: this isn't a helpful or motivating threat to make and if MacPorts makes your life easier: yes, perhaps you should use it instead.

[RokerHRO]

@MikeMcQuaid : Yes, I know. Sorry. I just was a bit annoyed because all of my colleagues said: "well, I use MacPorts and it runs fine here" and no-one was able nor willing to help me. ;-(
I am just a mercurial user, and a Homebrew user, and it is disappointing when things that ran perfectly, suddently break due to an innocent update, and I have no idea what went wrong there.

Sorry for that. Thanks again for your quick and competent fix. :-)

[RokerHRO]

@MikeMcQuaid : Yes, I know. Sorry. I just was a bit annoyed because all of my colleagues said: "well, I use MacPorts and it runs fine here" and no-one was able nor willing to help me. ;-(
I am just a mercurial user, and a Homebrew user, and it is disappointing when things that ran perfectly, suddently break due to an innocent update, and I have no idea what went wrong there.

Sorry for that. Thanks again for your quick and competent fix. :-)

[MikeMcQuaid]

@RokerHRO No worries, appreciate the apology and thanks for the kind words.

[MikeMcQuaid]

@RokerHRO No worries, appreciate the apology and thanks for the kind words.

[fanzeyi]

Hello all,

I noticed this python@2 dependency when I install mercurial on my local machine today, and I thought it was unnecessary so I dig to this issue. I understand the security concern but I want to point out that the python in my macOS High Sierra (10.13.3) does ship with TLS1.1+ supports. It is practical to depend on the system Python. Although I could not find any news on this update nor Apple mentioned such update in their release notes. I asked a few friends to test if their system's Python has TLS1.1+ supports as well and the results are positive. (I also have a friend who did not upgrade to 10.13 from 10.12, and the Python on his machine does not support TLS1.1+) [edit: I found one person talking about this on twitter]

Also here is a proof:

As far as I know, only Apple has the permission to write to /System/ if the System Integrity Protection is not manually turned off. I think this shows enough proofs that the Python comes with High Sierra supports TLS1.1+.

So, I wonder if it is possible to drop the dependency to python@2 for High Sierra users. (or at least adding an option to make it optional?)

Thanks!

[fanzeyi]

Hello all,

I noticed this python@2 dependency when I install mercurial on my local machine today, and I thought it was unnecessary so I dig to this issue. I understand the security concern but I want to point out that the python in my macOS High Sierra (10.13.3) does ship with TLS1.1+ supports. It is practical to depend on the system Python. Although I could not find any news on this update nor Apple mentioned such update in their release notes. I asked a few friends to test if their system's Python has TLS1.1+ supports as well and the results are positive. (I also have a friend who did not upgrade to 10.13 from 10.12, and the Python on his machine does not support TLS1.1+) [edit: I found one person talking about this on twitter]

Also here is a proof:

As far as I know, only Apple has the permission to write to /System/ if the System Integrity Protection is not manually turned off. I think this shows enough proofs that the Python comes with High Sierra supports TLS1.1+.

So, I wonder if it is possible to drop the dependency to python@2 for High Sierra users. (or at least adding an option to make it optional?)

Thanks!

[indygreg]

Yes, High Sierra ships with a /usr/bin/python that supports modern TLS versions (finally). So it is acceptable for Homebrew Python applications to use /usr/bin/python on 10.13 without sacrificing security.

[indygreg]

Yes, High Sierra ships with a /usr/bin/python that supports modern TLS versions (finally). So it is acceptable for Homebrew Python applications to use /usr/bin/python on 10.13 without sacrificing security.

[MikeMcQuaid]

So, I wonder if it is possible to drop the dependency to python@2 for High Sierra users. (or at least adding an option to make it optional?)

Sorry, it's more important to use that we're consistent with our Python usage across supported platforms. Making it optional means it's not tested by our CI system which means if (although in my experience it's more: when) it breaks we'll have no idea. Please ignore the dependency; it's a Homebrew implementation detail.

[MikeMcQuaid]

So, I wonder if it is possible to drop the dependency to python@2 for High Sierra users. (or at least adding an option to make it optional?)

Sorry, it's more important to use that we're consistent with our Python usage across supported platforms. Making it optional means it's not tested by our CI system which means if (although in my experience it's more: when) it breaks we'll have no idea. Please ignore the dependency; it's a Homebrew implementation detail.

[ilovezfs]

@indygreg also please note #26287. Using Homebrew Python is no longer specific to mercurial or a subset of formulae, and is instead a policy across all formulae using Python in https://github.com/Homebrew/homebrew-core

[ilovezfs]

@indygreg also please note #26287. Using Homebrew Python is no longer specific to mercurial or a subset of formulae, and is instead a policy across all formulae using Python in https://github.com/Homebrew/homebrew-core