[#9561] Check remote certificates for XMPP TLS #1147

ralphm · 2019-05-27T11:32:13Z

https://twistedmatrix.com/trac/ticket/9561
Replaces #1084.

While setting up XMPP connections, during StartTLS, an unconfigured CertificationOptions instance is passed. These changes properly use optionsForClientTLS instead, using the remote server's domain (XmlStream.otherJID, taken from the JID the client is connecting with) to verify the remote certificate using the platform trust root and that domain.

Additionally, you can now pass custom certificate options to modify this new default, e.g. for using client certificates with SASL External.

This adds an option `required` argument to the inits of initializers deriving from BaseFeatureInitiatingInitializer, to simplify setup. Additionally it changes the requiredness of two initializers used by XMPPAuthenticator: * Setup of TLS is now required by default. This ensures that if StartTLS is not advertized by the server, initialization fails instead of silently proceeding to authentication without encryption. * Binding a resource is required by default, because without it servers will not allow any further meaningful interaction.

This adds an optional `contextFactory` argument to `XMPPClientFactory` that is passed on to `XMPPAuthenticator`, which in turn passes it to `TLSInitiatingInitializer`.

glyph

Minor feedback inline; the biggest deal here is the public-interface changes (the naming and public-ness of contextFactory). Address as you see fit, and file follow-ups for anything you think should be dealt with later, then land!

src/twisted/words/protocols/jabber/client.py

src/twisted/words/protocols/jabber/xmlstream.py

glyph · 2019-05-27T22:08:32Z

src/twisted/words/test/test_jabberclient.py

@@ -16,6 +16,14 @@
 from twisted.words.protocols.jabber.sasl import SASLInitiatingInitializer
 from twisted.words.xish import utility

+try:
+    from twisted.internet import ssl
+except ImportError:


There are lots of other places which do this conditional import. Can we piggyback on one of them rather than adding bespoke ImportError-handling logic? My concern here is that this type of exception handling is likely to mask cases where there's an actual import-time bug in twisted.internet.ssl and I don't want to have to go looking for 20 instances of this if we ever manage to go in and fix it to only fail on defined import errors (i.e. those where cryptography isn't available).

Worst case, I'd rather literally import it from another test_ module for the time being.

I suppose this is due to the unconditional import of the OpenSSL module in twisted.internet.ssl and twisted.internet._sslverify. The former module does have a supported attribute, but it seems to (at least now) always be set to True, even though for example twisted.words.protocols.jabber.xmlstream checks for that.

If the OpenSSL module cannot be imported, the whole import fails, wheres I think my tests probably don't even use it.

Can we not make twisted.internet.ssl do this conditional import and have it set supported to False, and SSL to None?

@glyph I'm not sure if this should be addressed as part of this ticket?

src/twisted/words/test/test_jabberclient.py

alex · 2019-06-15T22:41:10Z

FYI I have requested a CVE for this.

alex · 2019-06-16T13:11:35Z

Please use CVE-2019-12855

alex · 2019-06-16T16:43:39Z

@ralphm can you include the CVE in the newsfragment?

ralphm · 2019-06-16T17:15:43Z

@alex Done! Thanks for that.

alex · 2019-06-16T17:16:59Z

Thank you!

…

On Sun, Jun 16, 2019 at 1:15 PM Ralph Meijer ***@***.***> wrote: @alex <https://github.com/alex> Done! Thanks for that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1147?email_source=notifications&email_token=AAAAGBHWZPD5Q5NZF6YVJQLP2ZYMPA5CNFSM4HP3CCCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXZRJDY#issuecomment-502469775>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAAGBDSUAXWH6BABGEW7K3P2ZYMPANCNFSM4HP3CCCA> .

-- All that is necessary for evil to succeed is for good people to do nothing.

This reverts commit 05556b6.

greysteil · 2019-07-18T10:35:19Z

I work on GitHub's security workflows team and am looking into how we can make it easier for maintainers to publicise security vulnerabilities like this one. I'd love your feedback if you have 5 mins?

Specifically:

GitHub sent alerts to Twisted users about this vulnerability, and in some cases automated security fix PRs. Could we have done anything better there?
did you consider using the "Maintainer Security Advisories" feature to publicise this vulnerability (it's in the "Security" tab on this repo)? What would make that feature useful to you in future?
do you have any other thoughts on the process of handling a security vulnerability, and how GitHub can help?

We're trying to make the process easier so if you have any feedback at all please let me know. You can email me on greysteil@github.com if you'd like to discuss anything privately.

alex · 2019-07-18T12:02:47Z

Hi Grey,

Here's some off the cuff thoughts:

I'd be surprised if GitHub could send a fix PR for this, I don't believe this PR has been included in a release yet.
We need to obtain CVEs for vulnerabilities, so unless GitHub can obtain them for us, using the Maintainer Security Advisories feature is duplicative.
Twisted is a large project and only a fraction of its users use XMPP, if there was some way to only notify users who were impacted by this specific vulnerability, as opposed to using an effected version, that'd be fantastic.

In general I'm super thrilled about GitHub automatically issuing notifications on insecure dependencies.

greysteil · 2019-07-18T12:52:27Z

Really useful feedback - thanks!

I'd be surprised if GitHub could send a fix PR for this, I don't believe this PR has been included in a release yet.

Oops! Thought it had been. We'll create fix PRs when it is :-)

We need to obtain CVEs for vulnerabilities, so unless GitHub can obtain them for us, using the Maintainer Security Advisories feature is duplicative.

Working on it!

Twisted is a large project and only a fraction of its users use XMPP, if there was some way to only notify users who were impacted by this specific vulnerability, as opposed to using an effected version, that'd be fantastic.

Interesting. I'm super keen to reduce our false positive rate - it's a way off but we'd like to be able to do that.

glyph · 2019-07-18T17:41:39Z

Oops! Thought it had been. We'll create fix PRs when it is :-)

One thing that would make this kind of thing easier to track is explicit "present in releases" state on issues and PRs. If even people who work at github can't tell whether a PR has made it into a release or not, that says something about the UI for this 😄 .

(In fairness, Twisted's processes predate like, neolithic information technology so we don't lean particularly heavily on the Github notion of a release, or use Github issues for tracking in this project; still, if the ability to track what releases an issue was fixed in or a PR was present in, that would be a big motivation to migrate.)

glyph · 2019-07-18T17:45:18Z

Could we have done anything better there?

For projects hosted on github, or even just with an official mirror here, it would be helpful if whatever notification-automation were integrated directly into the Maintainer Security Advisories tab. For example, if Github were to misidentify some element of the state associated with the PR (as you potentially did with the mistake around this being included in a release), we could go in and edit the advisory directly.

Based on the workflow I've seen so far, what I think I am asking for more specifically here is "please create a draft maintainer security advisory that we can see and comment on with at least some window before you put us on blast to our entire user base".

greysteil · 2019-07-18T17:48:05Z

Really helpful feedback, thanks @glyph. That's on our list of potential improvements for the next 3 months.

alex · 2019-07-18T17:51:52Z

"The earliest tag that includes this PR is X" would be amazing. When I was at Mozilla the Firefox bug tracker had this and it was super handy.

You can always figure it out by looking at the merge commit page of course, but highlighting it would be dope.

greysteil · 2019-07-18T17:52:58Z

Agreed - have passed that one on to the PR team

glyph · 2019-07-18T17:58:59Z

(If this functionality went both ways — "what are all the PRs this tag (release) is the first to include" — that would go a long way to automating the production of changelogs…)

ralphm · 2019-08-22T17:33:50Z

For completeness. Twisted 19.7.0 (2019-07-28) contains this fix.

glyph · 2019-08-23T00:29:35Z

Thanks for the note, ralph!

ralphm added 8 commits May 7, 2019 12:26

Use optionsForClientTLS to verify server certificate by default

488bdd0

Fix client example to print disconnection reason

0ff32b1

Allow for custom contextFactory to TLS initializer

89954df

Add docstrings for new contextFactory attribute

4759e27

Provide a way to use custom certificate options for XMPP clients

cadf08f

This adds an optional `contextFactory` argument to `XMPPClientFactory` that is passed on to `XMPPAuthenticator`, which in turn passes it to `TLSInitiatingInitializer`.

Adjust tests to TLSInitiatingInitializer being required by default

5ed194c

Add news fragments

a1f4390

ralphm changed the title ~~(#9561) Check remote certificates for XMPP TLS~~ [#9561] Check remote certificates for XMPP TLS May 27, 2019

ralphm and others added 4 commits May 27, 2019 13:53

Fix skipping renamed test when SSL is not available

0a93949

Skip TLS tests if OpenSSL is not available

751ac6f

Fix indents

672a633

Merge branch 'trunk' into 9561-xmpp-tls-verify-cert

fa1bc1b

glyph approved these changes May 27, 2019

View reviewed changes

Better docstring for BasicAuthenticatorTests

a649757

Rename contextFactory to configurationForTLS, make private vars

ea2d28f

ralphm added 3 commits June 16, 2019 19:02

Move check for configurationTLS being None to __init__

05556b6

Document configurationForTLS being None directly

7caf8ac

Mention CVE-2019-12855 in news fragment

a66878c

ralphm added 2 commits June 16, 2019 19:50

Revert "Move check for configurationTLS being None to __init__"

abbf0fd

This reverts commit 05556b6.

Merge branch 'trunk' into 9561-xmpp-tls-verify-cert

d99ce02

ralphm merged commit cf8ed69 into trunk Jul 4, 2019

ralphm mentioned this pull request Jul 5, 2019

Verify TLS certificate during XMPP connection. #1084

Closed

4 tasks

exarkun deleted the 9561-xmpp-tls-verify-cert branch July 18, 2019 12:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#9561] Check remote certificates for XMPP TLS #1147

[#9561] Check remote certificates for XMPP TLS #1147

ralphm commented May 27, 2019

glyph left a comment

glyph May 27, 2019

ralphm May 28, 2019

ralphm Jun 16, 2019

alex commented Jun 15, 2019

alex commented Jun 16, 2019

alex commented Jun 16, 2019

ralphm commented Jun 16, 2019

alex commented Jun 16, 2019 via email

greysteil commented Jul 18, 2019

alex commented Jul 18, 2019

greysteil commented Jul 18, 2019

glyph commented Jul 18, 2019

glyph commented Jul 18, 2019

greysteil commented Jul 18, 2019

alex commented Jul 18, 2019

greysteil commented Jul 18, 2019

glyph commented Jul 18, 2019

ralphm commented Aug 22, 2019

glyph commented Aug 23, 2019

[#9561] Check remote certificates for XMPP TLS #1147

[#9561] Check remote certificates for XMPP TLS #1147

Conversation

ralphm commented May 27, 2019

glyph left a comment

Choose a reason for hiding this comment

glyph May 27, 2019

Choose a reason for hiding this comment

ralphm May 28, 2019

Choose a reason for hiding this comment

ralphm Jun 16, 2019

Choose a reason for hiding this comment

alex commented Jun 15, 2019

alex commented Jun 16, 2019

alex commented Jun 16, 2019

ralphm commented Jun 16, 2019

alex commented Jun 16, 2019 via email

greysteil commented Jul 18, 2019

alex commented Jul 18, 2019

greysteil commented Jul 18, 2019

glyph commented Jul 18, 2019

glyph commented Jul 18, 2019

greysteil commented Jul 18, 2019

alex commented Jul 18, 2019

greysteil commented Jul 18, 2019

glyph commented Jul 18, 2019

ralphm commented Aug 22, 2019

glyph commented Aug 23, 2019