Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SSL support to Netty transport layer for Client/Node-to-Node communication #2105

Closed
wants to merge 1 commit into from

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Jul 18, 2012

This pull request modifies the Netty transport layer in order to support SSL for client/node to node communication within an elasticsearch cluster.

Configuration

SSL encryption is activated with a new configuration entry transport.tcp.compress. If set to true, the keystore and trustore of the node must also be configured:

# Enable SSL/TLS encryption for all communication between nodes:
#
# transport.tcp.ssl: true

# Settings for SSL/TLS encryption
#
# transport.tcp.ssl.keystore: /path/to/the/keystore
# transport.tcp.ssl.keystore_password: password
# transport.tcp.ssl.keystore_algorithm: SunX509
#
# transport.tcp.ssl.truststore: /path/to/the/truststore
# transport.tcp.ssl.truststore_password: password
# transport.tcp.ssl.truststore_algorithm: PKIX

If keystores settings are not set in configuration file, it looks for JVM options (-Djavax.net.ssl.keyStore=..., -Djavax.net.ssl.trustStore=...)

Before starting the cluster, make sure that all nodes recognize each other (by importing the certificates in other nodes' trustores)

A generate.sh script can be used to create keystores for testing:

$./generate.sh 3     // will create 3 keystores and 3 certificates

Transport Client

SSL can be configured for a TransportClient with the Java API:

TransportClient client = new TransportClient(settingsBuilder()
                .put("client.transport.nodes_sampler_interval", "30s")
                .put("transport.tcp.ssl", true)
                .put("transport.tcp.ssl.keystore", "/opt/certificates/esnode1.jks")
                .put("transport.tcp.ssl.keystore_password", "mypassword")
                .put("transport.tcp.ssl.truststore", "/opt/certificates/esnode1.jks")
                .put("transport.tcp.ssl.truststore_password", "mypassword")
                .put("client.transport.sniff", false).build());
        
        client.addTransportAddress(new InetSocketTransportAddress("localhost", 9300));

Use cases

  1. If a node tries to join a cluster but is not identified (certificate is unknown): discovery/recovery will fail, the node will start but won't join the cluster. The error message in log file is "org.elasticsearch.transport.SSLTransportException: SSL / TLS handshake failed, closing the channel" on cluster side.
  2. If a non-SSL node tries to join a cluster configured for SSL: discovery/recovery will fail, the node will start but won't join the cluster. The error message in log file is "StreamCorruptedException: invalid data length".
  3. If a transport client tries to connect to a cluster but is not identified: a NoNodeAvailableException is thrown.

Perf

One should pay attention to performance :/

NettyEchoBenchmark:

Warming up...
Warmed up
Ran 50000, TPS 9069.47215672048
Ran 50000, TPS 8939.746111210441
Ran 50000, TPS 10322.047894302228
Ran 50000, TPS 9238.728750923872
Ran 50000, TPS 10181.225819588679
Ran 50000, TPS 13319.126265316996
Ran 50000, TPS 9823.18271119843
Ran 50000, TPS 9529.25481227368
Ran 50000, TPS 9072.763563781527
Ran [500000] iterations, payload [100]: took [51], TPS: 9803.921568627451

NettyEchoSSLBenchmark:

Warming up...
Handshake completed on server side
Handshake completed on client side
Warmed up
Ran 50000, TPS 5730.002292000916
Ran 50000, TPS 6375.924509053813
Ran 50000, TPS 5571.651437486071
Ran 50000, TPS 6209.6373571783415
Ran 50000, TPS 5787.0370370370365
Ran 50000, TPS 5855.486590935707
Ran 50000, TPS 5726.720879624328
Ran 50000, TPS 5262.050094716901
Ran 50000, TPS 5755.726948313572
Ran [500000] iterations, payload [100]: took [87], TPS: 5747.126436781609

Code review and comments are welcome! :)

@dadoonet
Copy link
Member

dadoonet commented Jul 19, 2012

Very nice. I love the idea.

@skade skade mentioned this pull request Jul 20, 2012
@kimchy
Copy link
Member

kimchy commented Jul 21, 2012

Thanks for providing this pull request. The story around SSL on the transport layer is a bit more complicated though. If we add this, we need to think about allowing for cases where node to node communication does not use SSL, while clients do.

The overhead of node to node communication over SSL is huge, I don't really think people understand that ... . The value security wise is close to 0, to be honest, if you have proper firewall and other security measures in place.

@tlrx
Copy link
Member Author

tlrx commented Jul 23, 2012

Thanks for the feedback. I understand the complexity that represents node to node communication over SSL. This pull request was a quick and easy way to get SSL within a cluster, as requested by many users.

@tlrx tlrx closed this Jul 23, 2012
@thejohnfreeman
Copy link

thejohnfreeman commented Jun 25, 2013

This seemed like the most promising proposal, but it was rejected. What does it take to get SSL into ES? Is it enough to have separate options for requiring SSL for connections initiated inside the node versus outside the node? What are your requirements, @kimchy ?

@mbarrien
Copy link

mbarrien commented Jul 12, 2013

Quick note: Even though security value may be 0, having node-to-node communication be encrypted may still be required. Our specific case is that to store HIPAA information in Amazon and still be considered in compliance for auditing purposes, all patient data must be encrypted when sent over the wire. It's not strictly required by pure HIPAA regulations, but a HIPAA related contract (BAA) signed with Amazon does require it. Because this pull request was rejected, Elasticsearch fails to meet this requirement and thus can't be used.

HIPAA won't be the last place we'll see this requirement (I suspect a lot of government related Amazon work will be similarly covered). And since it's a bunch of legalese, no amount of pointing out how little actual security is gained or how firewalls and port isolation are enough will suffice to pass the lawyers and auditors scrutiny.

Please consider resurrecting this pull request!

@robottaway
Copy link

robottaway commented Aug 3, 2013

@mbarrien in the same boat here. Firewall rules alone won't cut it. We've been audited by expensive HIPAA experts and I can tell you the actual price of not having SSL transport layer, and it's definitely not 0.

@kimchy is the pull request going to hurt anyone if they actually implement it? At this point I may just have to clone ES and roll my own solution even if it's ugly. BTW the overhead looks completely acceptable for our setup if the figures @tlrx provided are in the ballpark.

@fatlotus
Copy link

fatlotus commented Aug 4, 2013

@robottaway Yeah, I'd second that — seems silly to throw away @tlrx's efforts just because we think it was difficult to do.

@danielberlinger
Copy link

danielberlinger commented Jan 7, 2014

@kimchy I'd like to add a vote for this feature (despite the age of the issue). It would be incredibly helpful from a HIPPA/HITECH compliance perspective.

@MartinHatas
Copy link

MartinHatas commented Mar 10, 2014

+1000 for this feature

@anandsomani
Copy link

anandsomani commented Mar 26, 2014

+1000 for this one!

@Mrc0113
Copy link

Mrc0113 commented May 4, 2014

+100000....Also looking to have node to node encryption for HIPAA information in Amazon

@robottaway
Copy link

robottaway commented May 7, 2014

We have a repo containing the SSL enabled version of the 1.1.0 code here:

https://gitlab.devero.com/public/projects

there is also a SSL and QOS enabled RabbitMQ river plugin. On my radar to
cut releases in our Nexus so they can simply be dowloaded but for now you
have to "mvn package" to get the built product.

We plan to keep SSL branches as we move up versions of Elasticsearch or
until SSL transport becomes part if the project proper.

-rob

On Saturday, May 3, 2014, Marc DiPasquale notifications@github.com wrote:

+100000....Also looking to have node to node encryption for HIPAA
information in Amazon


Reply to this email directly or view it on GitHubhttps://github.com//pull/2105#issuecomment-42123510
.

@RobbieHer
Copy link

RobbieHer commented May 16, 2014

@robottaway 👍

Also, could you please confirm if the v1.1.0_ssl_work branch contains functionality to SSL protect inter-node communications on the transport layer (port 9300)? If so, could you please share some configuration settings, java client API usage and expected performance metrics?

Thanks!

@robottaway
Copy link

robottaway commented May 17, 2014

@RobbieHer encryption on 9300 is what this branch of ours provides. If you want TLS/SSL on the 9200 you will likely setup NGINX, Apache or another proxy in front which will allow setting up encryption. Note if you are going to use the Java API in your app you will need to configure encryption for 9300 there also since you cannot have both secured and unsecured.

@RobbieHer
Copy link

RobbieHer commented May 19, 2014

@robottaway Thanks!

Any idea on when this change can/will make it into the official ElasticSearch branch?

@robottaway
Copy link

robottaway commented May 19, 2014

You are very welcome! I would hope this could get into the official branch(es) some day! Past conversations being an indicator it doesn't seem likely though. It def is a bit of work to have to patch every branch we want to use, and it certainly keeps us from upgrading as quickly as we could.

@skurfuerst
Copy link

skurfuerst commented Jul 9, 2014

👍 for this feature! Checking it out right now, it would be awesome if this could be implemented in ES or an "official" module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet