Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub switched to accepting TLS v1.2 only #408

Closed
fuzzylogiq opened this issue Feb 23, 2018 · 25 comments
Closed

GitHub switched to accepting TLS v1.2 only #408

fuzzylogiq opened this issue Feb 23, 2018 · 25 comments

Comments

@fuzzylogiq
Copy link

@fuzzylogiq fuzzylogiq commented Feb 23, 2018

https://githubengineering.com/crypto-removal-notice/

This is causing errors with GitHubReleasesInfoProvider e.g.

Error opening URL: https://api.github.com/repos/munki/munki/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Unexpected GitHub API status code None. Error opening URL: https://api.github.com/repos/chilcote/outset/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Error opening URL: https://api.github.com/repos/clburlison/pinpoint/releases [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) Unexpected GitHub API status code None.

EDIT: urllib2 in system Python does not support TLS v1.2 on any version of macOS earlier than 10.13 currently.

EDIT2: This also breaks autopkg search on <10.13

(previous post said no version of macOS Python supported TLS v1.2)

@hjuutilainen
Copy link
Contributor

@hjuutilainen hjuutilainen commented Feb 23, 2018

Thanks for reporting. I'm not seeing this here with latest autopkg running on macOS 10.13. What macOS version are you running on?

$ autopkg run -v pinpoint.download
Processing pinpoint.download...
WARNING: pinpoint.download is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
GitHubReleasesInfoProvider
GitHubReleasesInfoProvider: Selected asset 'pinpoint-2.0.0.87.pkg' from release 'v2.0.0'
URLDownloader
URLDownloader: Item at URL is unchanged.
@soberhofer
Copy link

@soberhofer soberhofer commented Feb 23, 2018

Seeing the same issue here with autopkg 1.0.3 on 10.12

autopkg run -v local.munki.outset
Processing local.munki.outset...
GitHubReleasesInfoProvider
Error opening URL: https://api.github.com/repos/chilcote/outset/releases
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
Unexpected GitHub API status code None.
Failed.
Receipt written to /Users/administrator/Library/AutoPkg/Cache/local.munki.outset/receipts/local.munki-receipt-20180223-144742.plist

The following recipes failed:
    local.munki.outset
        Error in local.munki.outset: Processor: GitHubReleasesInfoProvider: Error: Unexpected GitHub API status code None.

Nothing downloaded, packaged or imported.
@clburlison
Copy link
Member

@clburlison clburlison commented Feb 23, 2018

10.13 will support TLS 1.2 since it isn’t linked against an outdated OpenSSL (it is linked against boringSSL). So any OS less than 10.13 will start showing this error.

@timsutton
Copy link
Member

@timsutton timsutton commented Feb 23, 2018

I knew this day would come - a little surprised it took this long, although I never saw the deprecation notice in September. I'm assuming this would affect any use of the API on 10.12 and under, so autopkg search as well as the admin scripts we have for creating new teams/repos and doing AutoPkg releases.

On my 10.13 system here, trying to specify TLS 1.1 with curl seems to return an error:

curl --tlsv1.1 https://api.github.com/users/timsutton
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

But specifying --tlsv1 returns the expected results, so not sure that's an adequate test.

@timsutton
Copy link
Member

@timsutton timsutton commented Feb 23, 2018

Nevermind, --tlsv1 on my 10.13 system (still /usr/bin/curl) seems to actually use 1.2. Maybe this is expected behaviour.

$ curl -v --tlsv1 https://api.github.com/users/timsutton
*   Trying 192.30.253.117...
* TCP_NODELAY set
* Connected to api.github.com (192.30.253.117) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
@childrss
Copy link

@childrss childrss commented Feb 23, 2018

So... moving forward, AutoPkg will only work on 10.13, or will AutoPkg be updated with a work-around, or will it be recommended to install over the system version of Python on <10.13 ....

Had to do some upgrades on our server running autopkg anyhoo.... and going to 10.13 anyway, just asking for those that might not have time to do the upgrade.

@grahampugh
Copy link
Contributor

@grahampugh grahampugh commented Feb 23, 2018

It would be good to know if this will be patched or if there is a workaround for those of us running AutoPkg on 10.11 (because Xserve).

@clburlison
Copy link
Member

@clburlison clburlison commented Feb 23, 2018

@childrss At this time, it looks like converting the GH processor to use curl it the method going forward. However someone has to write and test this code. No one has stepped forward yet saying they have the time to test and implement this change.

@grahampugh If you want a workaround you can:

  1. Link autopkg against a custom python 2.7.14 install that has a newer version of openssl
  2. You can patch the openssl that the stock system python uses a newer version of openssl. I've done some work around this but never tested 10.11. If you install the vendored_tlsssl.pkg clburlison/vendored release v2 it should start working (only works for 10.12 machines).

^ I don't really recommend doing either of these as they are bandaids but if that's what you need. 🤷‍♂️

All that said upgrading your autopkg machine(s) to 10.13 is the best solution right now as you'll also get security patches from Apple, more supported, etc.


UPDATE: I got bored and started playing with the second option I presented above about overriding the openssl version with the tls hack. It works but kind of sucks because the patch must be built on the OS that you wish to deploy to IE - build on 10.12 to support 10.12, build 10.11 to support 10.11. So that makes it even more of a pain. The first option about linking to a different version of python (so long as you include all the correct "batteries") is a much better bandaid and from my tests works. However as Tim mentions below you'll want to point all autopkg code to your custom python which is another step.

@timsutton
Copy link
Member

@timsutton timsutton commented Feb 23, 2018

@childrss Using a different Python distribution also brings another concern: that this Python will also need access to a PyObjC library, and the only one that's ever been used by AutoPkg is the version(s) that Apple has shipped with OS X/macOS.

This is required for some core functionality like accessing AutoPkg's preferences. AutoPkg contains some workarounds in case a different Python distribution is used, which was added for the purposes of running on Windows. However, those workarounds in their current bare-bones state removes quite a bit of the tool's usability on macOS (IMO).

@clburlison mentioned in Slack that his vendored Python solution has the ability to include a recent PyObjC. If you included a custom Python from, for example, Homebrew, you'd need to install PyObjC yourself. AutoPkg might also still make assumptions about exactly which Python it is running as well, and this would all of course be untested (except possibly by @clburlison!).

@tallfunnyjew
Copy link

@tallfunnyjew tallfunnyjew commented Feb 24, 2018

autopkg: 1.0.3
macOS: 10.12.6

Everything changed for me yesterday when I installed Python 3.6 via Homebrew. Now same error. Re-installed PyObjC and no dice. Re-installed autopkg as well: no dice. What's maddening is that default python hasn't changed: still 2.7.10. I'm entering a virtenv to use Python 3.6, so I'm unclear as to why this is happening. I'm also EXTREMELY new to python, so that may also explain some shiz.

@clburlison
Copy link
Member

@clburlison clburlison commented Feb 24, 2018

@tallfunnyjew It looks like you missed the first comment on this thread. It has a link explaining that this change was on Github’s server side. Also, autopkg is written for python 2 not python 3. My recommendation would be to upgrade your machine to 10.13.

@tallfunnyjew
Copy link

@tallfunnyjew tallfunnyjew commented Feb 24, 2018

No, I read it. Then I read the rest of the posts where a few other suggestions were suggested, including from you, to re-install PyObjectC and/or download/install vendored_tlsssl.pkg. Just reporting back on those suggestions that they aren't working for me. And I'm not prepared to update to 10.13 on my autopkg Macs at this time.

@hjuutilainen
Copy link
Contributor

@hjuutilainen hjuutilainen commented Feb 26, 2018

I made some initial changes and tests to switch all GitHub API calls to use curl instead and they seem to work with 10.11, 10.12 and 10.13 (didn't test earlier). This might be the easiest initial fix for this issue.

If you're interested in testing, I've pushed the changes to https://github.com/autopkg/autopkg/tree/github-urllib2-to-curl

@tcinbis
Copy link

@tcinbis tcinbis commented Feb 26, 2018

@hjuutilainen maybe we could also add the changes of PR #404 , because the github module in autopkglib is changing too?

@hjuutilainen
Copy link
Contributor

@hjuutilainen hjuutilainen commented Feb 26, 2018

Yes, in some form. All GitHub API calls should use the token if available.

@poundbangbash
Copy link
Contributor

@poundbangbash poundbangbash commented Feb 26, 2018

I ran this commit against some recipes and searches I had failing on 10.12.6 due to the github changes and they run clean now.

Run with 1.0.3

holtam:Code homeadmin$ autopkg version
1.0.3

holtam:Code homeadmin$ autopkg run outset.munki
Processing outset.munki...
Error opening URL: https://api.github.com/repos/chilcote/outset/releases
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
Unexpected GitHub API status code None.
Failed.

The following recipes failed:
    outset.munki
        Error in local.munki.outset: Processor: GitHubReleasesInfoProvider: Error: Unexpected GitHub API status code None.

Nothing downloaded, packaged or imported.

Run with 1.0.4

holtam:Code homeadmin$ ./autopkg version
1.0.4

holtam:Code homeadmin$ ./autopkg run outset.munki
Processing outset.munki...

Nothing downloaded, packaged or imported.

Search with 1.0.3

holtam:Code homeadmin$ autopkg search outset
Error opening URL: https://api.github.com/search/code?q=outset+extension:recipe+user:autopkg+in:path,file&per_page=100
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)
A GitHub API error occurred!

Search with 1.0.4

holtam:Code homeadmin$ ./autopkg search outset

Name                                           Repo                      Path                                    
----                                           ----                      ----                                    
outset.jss.recipe                              chalcahuite-recipes       Outset/outset.jss.recipe                
outset.download.recipe                         grahamgilbert-recipes     outset/outset.download.recipe           
outset.install.recipe                          grahamgilbert-recipes     outset/outset.install.recipe            
outset.munki.recipe                            grahamgilbert-recipes     outset/outset.munki.recipe              
BlackboardCollaborateLauncher.pkg.recipe       grahamgilbert-recipes     Blackboard/BlackboardCollaborateLauncher.pkg.recipe
BlackboardCollaborateLauncher.munki.recipe     grahamgilbert-recipes     Blackboard/BlackboardCollaborateLauncher.munki.recipe
outset.ds.recipe                               jazzace-recipes           outset/outset.ds.recipe                 
outset.absolute.recipe                         patgmac-recipes           outset/outset.absolute.recipe           
outset.pkg.recipe                              patgmac-recipes           outset/outset.pkg.recipe                

-Eric

@fuzzylogiq
Copy link
Author

@fuzzylogiq fuzzylogiq commented Feb 26, 2018

Initial testing on outset and osxfuse looks good. Autopkg search also behaving correctly for me (all on 10.12.6 with system Python)

@sramdeen
Copy link

@sramdeen sramdeen commented Feb 27, 2018

This seems to be working well for me under 10.11 and autopkgr. Searching and downloading now working as expected. Thank you @hjuutilainen!

@tallfunnyjew
Copy link

@tallfunnyjew tallfunnyjew commented Mar 4, 2018

I'm on 10.12.6. I did a clean re-install of autopkg as I see that @hjuutilainen merged these changes into the main branch. I'm still having the same search errors (TLSV1_ALERT). Python is version 2.7.10. Am I missing something?

@tcinbis
Copy link

@tcinbis tcinbis commented Mar 4, 2018

@tallfunnyjew did you install the latest release (pkg) or did you download the repository and built your own package?

@mattiasvdm
Copy link

@mattiasvdm mattiasvdm commented Mar 5, 2018

@tcinbis I always use the pkg. My guess is to built the package yourself to have the latest changes? How do I accomplish this?

@sramdeen
Copy link

@sramdeen sramdeen commented Mar 5, 2018

You don't have to build anything, just download the linked file https://github.com/autopkg/autopkg/archive/github-urllib2-to-curl.zip and then cd into the Scripts folder and run sudo ./install.sh
That's all I had to do.

@tcinbis
Copy link

@tcinbis tcinbis commented Mar 5, 2018

You could also just clone or download the current repository (master branch) and use the install Script @sramdeen mentioned.

@hjuutilainen
Copy link
Contributor

@hjuutilainen hjuutilainen commented Mar 5, 2018

@gregneagle
Copy link
Contributor

@gregneagle gregneagle commented Mar 12, 2018

Closing this since addressed with 1.0.4 release.

@gregneagle gregneagle closed this Mar 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet