GitHub switched to accepting TLS v1.2 only

[fuzzylogiq]

https://githubengineering.com/crypto-removal-notice/

This is causing errors with GitHubReleasesInfoProvider e.g.

Error opening URL: https://api.github.com/repos/munki/munki/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Unexpected GitHub API status code None. Error opening URL: https://api.github.com/repos/chilcote/outset/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Error opening URL: https://api.github.com/repos/clburlison/pinpoint/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Unexpected GitHub API status code None.

EDIT: urllib2 in system Python does not support TLS v1.2 on any version of macOS earlier than 10.13 currently.

EDIT2: This also breaks autopkg search on <10.13

(previous post said no version of macOS Python supported TLS v1.2)

[hjuutilainen]

Thanks for reporting. I'm not seeing this here with latest autopkg running on macOS 10.13. What macOS version are you running on?

$ autopkg run -v pinpoint.download
Processing pinpoint.download...
WARNING: pinpoint.download is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
GitHubReleasesInfoProvider
GitHubReleasesInfoProvider: Selected asset 'pinpoint-2.0.0.87.pkg' from release 'v2.0.0'
URLDownloader
URLDownloader: Item at URL is unchanged.

[soberhofer]

Seeing the same issue here with autopkg 1.0.3 on 10.12

autopkg run -v local.munki.outset
Processing local.munki.outset...
GitHubReleasesInfoProvider
Error opening URL: https://api.github.com/repos/chilcote/outset/releases
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
Unexpected GitHub API status code None.
Failed.
Receipt written to /Users/administrator/Library/AutoPkg/Cache/local.munki.outset/receipts/local.munki-receipt-20180223-144742.plist

The following recipes failed:
    local.munki.outset
        Error in local.munki.outset: Processor: GitHubReleasesInfoProvider: Error: Unexpected GitHub API status code None.

Nothing downloaded, packaged or imported.

[clburlison]

10.13 will support TLS 1.2 since it isn’t linked against an outdated OpenSSL (it is linked against boringSSL). So any OS less than 10.13 will start showing this error.

[timsutton]

I knew this day would come - a little surprised it took this long, although I never saw the deprecation notice in September. I'm assuming this would affect any use of the API on 10.12 and under, so autopkg search as well as the admin scripts we have for creating new teams/repos and doing AutoPkg releases.

On my 10.13 system here, trying to specify TLS 1.1 with curl seems to return an error:

curl --tlsv1.1 https://api.github.com/users/timsutton
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

But specifying --tlsv1 returns the expected results, so not sure that's an adequate test.

[timsutton]

Nevermind, --tlsv1 on my 10.13 system (still /usr/bin/curl) seems to actually use 1.2. Maybe this is expected behaviour.

$ curl -v --tlsv1 https://api.github.com/users/timsutton
*   Trying 192.30.253.117...
* TCP_NODELAY set
* Connected to api.github.com (192.30.253.117) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):

[childrss]

So... moving forward, AutoPkg will only work on 10.13, or will AutoPkg be updated with a work-around, or will it be recommended to install over the system version of Python on <10.13 ....

Had to do some upgrades on our server running autopkg anyhoo.... and going to 10.13 anyway, just asking for those that might not have time to do the upgrade.

[grahampugh]

It would be good to know if this will be patched or if there is a workaround for those of us running AutoPkg on 10.11 (because Xserve).

[clburlison]

@childrss At this time, it looks like converting the GH processor to use curl it the method going forward. However someone has to write and test this code. No one has stepped forward yet saying they have the time to test and implement this change.

@grahampugh If you want a workaround you can:

1. Link autopkg against a custom python 2.7.14 install that has a newer version of openssl
2. You can patch the openssl that the stock system python uses a newer version of openssl. I've done some work around this but never tested 10.11. If you install the vendored_tlsssl.pkg clburlison/vendored release v2 it should start working (only works for 10.12 machines).

^ I don't really recommend doing either of these as they are bandaids but if that's what you need. 🤷‍♂️

All that said upgrading your autopkg machine(s) to 10.13 is the best solution right now as you'll also get security patches from Apple, more supported, etc.

---

UPDATE: I got bored and started playing with the second option I presented above about overriding the openssl version with the tls hack. It works but kind of sucks because the patch must be built on the OS that you wish to deploy to IE - build on 10.12 to support 10.12, build 10.11 to support 10.11. So that makes it even more of a pain. The first option about linking to a different version of python (so long as you include all the correct "batteries") is a much better bandaid and from my tests works. However as Tim mentions below you'll want to point all autopkg code to your custom python which is another step.

[timsutton]

@childrss Using a different Python distribution also brings another concern: that this Python will also need access to a PyObjC library, and the only one that's ever been used by AutoPkg is the version(s) that Apple has shipped with OS X/macOS.

This is required for some core functionality like accessing AutoPkg's preferences. AutoPkg contains some workarounds in case a different Python distribution is used, which was added for the purposes of running on Windows. However, those workarounds in their current bare-bones state removes quite a bit of the tool's usability on macOS (IMO).

@clburlison mentioned in Slack that his vendored Python solution has the ability to include a recent PyObjC. If you included a custom Python from, for example, Homebrew, you'd need to install PyObjC yourself. AutoPkg might also still make assumptions about exactly which Python it is running as well, and this would all of course be untested (except possibly by @clburlison!).

[tallfunnyjew]

autopkg: 1.0.3
macOS: 10.12.6

Everything changed for me yesterday when I installed Python 3.6 via Homebrew. Now same error. Re-installed PyObjC and no dice. Re-installed autopkg as well: no dice. What's maddening is that default python hasn't changed: still 2.7.10. I'm entering a virtenv to use Python 3.6, so I'm unclear as to why this is happening. I'm also EXTREMELY new to python, so that may also explain some shiz.

[clburlison]

@tallfunnyjew It looks like you missed the first comment on this thread. It has a link explaining that this change was on Github’s server side. Also, autopkg is written for python 2 not python 3. My recommendation would be to upgrade your machine to 10.13.

[tallfunnyjew]

No, I read it. Then I read the rest of the posts where a few other suggestions were suggested, including from you, to re-install PyObjectC and/or download/install vendored_tlsssl.pkg. Just reporting back on those suggestions that they aren't working for me. And I'm not prepared to update to 10.13 on my autopkg Macs at this time.

[hjuutilainen]

I made some initial changes and tests to switch all GitHub API calls to use curl instead and they seem to work with 10.11, 10.12 and 10.13 (didn't test earlier). This might be the easiest initial fix for this issue.

If you're interested in testing, I've pushed the changes to https://github.com/autopkg/autopkg/tree/github-urllib2-to-curl

[tcinbis]

@hjuutilainen maybe we could also add the changes of PR #404 , because the github module in autopkglib is changing too?

[hjuutilainen]

Yes, in some form. All GitHub API calls should use the token if available.

[poundbangbash]

I ran this commit against some recipes and searches I had failing on 10.12.6 due to the github changes and they run clean now.

Run with 1.0.3

holtam:Code homeadmin$ autopkg version
1.0.3

holtam:Code homeadmin$ autopkg run outset.munki
Processing outset.munki...
Error opening URL: https://api.github.com/repos/chilcote/outset/releases
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
Unexpected GitHub API status code None.
Failed.

The following recipes failed:
    outset.munki
        Error in local.munki.outset: Processor: GitHubReleasesInfoProvider: Error: Unexpected GitHub API status code None.

Nothing downloaded, packaged or imported.

Run with 1.0.4

holtam:Code homeadmin$ ./autopkg version
1.0.4

holtam:Code homeadmin$ ./autopkg run outset.munki
Processing outset.munki...

Nothing downloaded, packaged or imported.

Search with 1.0.3

holtam:Code homeadmin$ autopkg search outset
Error opening URL: https://api.github.com/search/code?q=outset+extension:recipe+user:autopkg+in:path,file&per_page=100
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
A GitHub API error occurred!

Search with 1.0.4

holtam:Code homeadmin$ ./autopkg search outset

Name                                           Repo                      Path                                    
----                                           ----                      ----                                    
outset.jss.recipe                              chalcahuite-recipes       Outset/outset.jss.recipe                
outset.download.recipe                         grahamgilbert-recipes     outset/outset.download.recipe           
outset.install.recipe                          grahamgilbert-recipes     outset/outset.install.recipe            
outset.munki.recipe                            grahamgilbert-recipes     outset/outset.munki.recipe              
BlackboardCollaborateLauncher.pkg.recipe       grahamgilbert-recipes     Blackboard/BlackboardCollaborateLauncher.pkg.recipe
BlackboardCollaborateLauncher.munki.recipe     grahamgilbert-recipes     Blackboard/BlackboardCollaborateLauncher.munki.recipe
outset.ds.recipe                               jazzace-recipes           outset/outset.ds.recipe                 
outset.absolute.recipe                         patgmac-recipes           outset/outset.absolute.recipe           
outset.pkg.recipe                              patgmac-recipes           outset/outset.pkg.recipe                

-Eric

[fuzzylogiq]

Initial testing on outset and osxfuse looks good. Autopkg search also behaving correctly for me (all on 10.12.6 with system Python)

[sramdeen]

This seems to be working well for me under 10.11 and autopkgr. Searching and downloading now working as expected. Thank you @hjuutilainen!

[tallfunnyjew]

I'm on 10.12.6. I did a clean re-install of autopkg as I see that @hjuutilainen merged these changes into the main branch. I'm still having the same search errors (TLSV1_ALERT). Python is version 2.7.10. Am I missing something?

[tcinbis]

@tallfunnyjew did you install the latest release (pkg) or did you download the repository and built your own package?

[mattiasvdm]

@tcinbis I always use the pkg. My guess is to built the package yourself to have the latest changes? How do I accomplish this?

[sramdeen]

You don't have to build anything, just download the linked file https://github.com/autopkg/autopkg/archive/github-urllib2-to-curl.zip and then cd into the Scripts folder and run sudo ./install.sh
That's all I had to do.

[tcinbis]

You could also just clone or download the current repository (master branch) and use the install Script @sramdeen mentioned.

[hjuutilainen]

Version 1.0.4 has been released:
https://github.com/autopkg/autopkg/releases/tag/v1.0.4

[gregneagle]

Closing this since addressed with 1.0.4 release.