GitHub switched to accepting TLS v1.2 only

[fuzzylogiq]

https://githubengineering.com/crypto-removal-notice/

This is causing errors with GitHubReleasesInfoProvider e.g.

Error opening URL: https://api.github.com/repos/munki/munki/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Unexpected GitHub API status code None. Error opening URL: https://api.github.com/repos/chilcote/outset/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Error opening URL: https://api.github.com/repos/clburlison/pinpoint/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Unexpected GitHub API status code None.

EDIT: urllib2 in system Python does not support TLS v1.2 on any version of macOS earlier than 10.13 currently.

EDIT2: This also breaks autopkg search on <10.13

(previous post said no version of macOS Python supported TLS v1.2)

[hjuutilainen]

Thanks for reporting. I'm not seeing this here with latest autopkg running on macOS 10.13. What macOS version are you running on?

$ autopkg run -v pinpoint.download
Processing pinpoint.download...
WARNING: pinpoint.download is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
GitHubReleasesInfoProvider
GitHubReleasesInfoProvider: Selected asset 'pinpoint-2.0.0.87.pkg' from release 'v2.0.0'
URLDownloader
URLDownloader: Item at URL is unchanged.

[soberhofer]

Seeing the same issue here with autopkg 1.0.3 on 10.12

autopkg run -v local.munki.outset
Processing local.munki.outset...
GitHubReleasesInfoProvider
Error opening URL: https://api.github.com/repos/chilcote/outset/releases
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
Unexpected GitHub API status code None.
Failed.
Receipt written to /Users/administrator/Library/AutoPkg/Cache/local.munki.outset/receipts/local.munki-receipt-20180223-144742.plist

The following recipes failed:
    local.munki.outset
        Error in local.munki.outset: Processor: GitHubReleasesInfoProvider: Error: Unexpected GitHub API status code None.

Nothing downloaded, packaged or imported.

[clburlison]

10.13 will support TLS 1.2 since it isn’t linked against an outdated OpenSSL (it is linked against boringSSL). So any OS less than 10.13 will start showing this error.

[timsutton]

I knew this day would come - a little surprised it took this long, although I never saw the deprecation notice in September. I'm assuming this would affect any use of the API on 10.12 and under, so autopkg search as well as the admin scripts we have for creating new teams/repos and doing AutoPkg releases.

On my 10.13 system here, trying to specify TLS 1.1 with curl seems to return an error:

curl --tlsv1.1 https://api.github.com/users/timsutton
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

But specifying --tlsv1 returns the expected results, so not sure that's an adequate test.

[timsutton]

Nevermind, --tlsv1 on my 10.13 system (still /usr/bin/curl) seems to actually use 1.2. Maybe this is expected behaviour.

$ curl -v --tlsv1 https://api.github.com/users/timsutton
*   Trying 192.30.253.117...
* TCP_NODELAY set
* Connected to api.github.com (192.30.253.117) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):

[childrss]

So... moving forward, AutoPkg will only work on 10.13, or will AutoPkg be updated with a work-around, or will it be recommended to install over the system version of Python on <10.13 ....

Had to do some upgrades on our server running autopkg anyhoo.... and going to 10.13 anyway, just asking for those that might not have time to do the upgrade.

[grahampugh]

It would be good to know if this will be patched or if there is a workaround for those of us running AutoPkg on 10.11 (because Xserve).

[clburlison]

@childrss At this time, it looks like converting the GH processor to use curl it the method going forward. However someone has to write and test this code. No one has stepped forward yet saying they have the time to test and implement this change.

@grahampugh If you want a workaround you can:

1. Link autopkg against a custom python 2.7.14 install that has a newer version of openssl
2. You can patch the openssl that the stock system python uses a newer version of openssl. I've done some work around this but never tested 10.11. If you install the vendored_tlsssl.pkg clburlison/vendored release v2 it should start working (only works for 10.12 machines).

^ I don't really recommend doing either of these as they are bandaids but if that's what you need. 🤷‍♂️

All that said upgrading your autopkg machine(s) to 10.13 is the best solution right now as you'll also get security patches from Apple, more supported, etc.

---

UPDATE: I got bored and started playing with the second option I presented above about overriding the openssl version with the tls hack. It works but kind of sucks because the patch must be built on the OS that you wish to deploy to IE - build on 10.12 to support 10.12, build 10.11 to support 10.11. So that makes it even more of a pain. The first option about linking to a different version of python (so long as you include all the correct "batteries") is a much better bandaid and from my tests works. However as Tim mentions below you'll want to point all autopkg code to your custom python which is another step.

[timsutton]

@childrss Using a different Python distribution also brings another concern: that this Python will also need access to a PyObjC library, and the only one that's ever been used by AutoPkg is the version(s) that Apple has shipped with OS X/macOS.

This is required for some core functionality like accessing AutoPkg's preferences. AutoPkg contains some workarounds in case a different Python distribution is used, which was added for the purposes of running on Windows. However, those workarounds in their current bare-bones state removes quite a bit of the tool's usability on macOS (IMO).

@clburlison mentioned in Slack that his vendored Python solution has the ability to include a recent PyObjC. If you included a custom Python from, for example, Homebrew, you'd need to install PyObjC yourself. AutoPkg might also still make assumptions about exactly which Python it is running as well, and this would all of course be untested (except possibly by @clburlison!).

[tallfunnyjew]

autopkg: 1.0.3
macOS: 10.12.6

Everything changed for me yesterday when I installed Python 3.6 via Homebrew. Now same error. Re-installed PyObjC and no dice. Re-installed autopkg as well: no dice. What's maddening is that default python hasn't changed: still 2.7.10. I'm entering a virtenv to use Python 3.6, so I'm unclear as to why this is happening. I'm also EXTREMELY new to python, so that may also explain some shiz.

[clburlison]

@tallfunnyjew It looks like you missed the first comment on this thread. It has a link explaining that this change was on Github’s server side. Also, autopkg is written for python 2 not python 3. My recommendation would be to upgrade your machine to 10.13.

[tallfunnyjew]

No, I read it. Then I read the rest of the posts where a few other suggestions were suggested, including from you, to re-install PyObjectC and/or download/install vendored_tlsssl.pkg. Just reporting back on those suggestions that they aren't working for me. And I'm not prepared to update to 10.13 on my autopkg Macs at this time.

[hjuutilainen]

I made some initial changes and tests to switch all GitHub API calls to use curl instead and they seem to work with 10.11, 10.12 and 10.13 (didn't test earlier). This might be the easiest initial fix for this issue.

If you're interested in testing, I've pushed the changes to https://github.com/autopkg/autopkg/tree/github-urllib2-to-curl

[tcinbis]

@hjuutilainen maybe we could also add the changes of PR #404 , because the github module in autopkglib is changing too?