Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL/TLS Support #17

Closed
okkez opened this issue Jul 14, 2017 · 12 comments
Closed

SSL/TLS Support #17

okkez opened this issue Jul 14, 2017 · 12 comments

Comments

@okkez
Copy link
Contributor

@okkez okkez commented Jul 14, 2017

It is very useful to support SSL/TLS by influent.
Because Fluentd v0.14.x in/out_forward supports SSL/TLS.
I want to use SSL/TLS in https://github.com/fluent/kafka-connect-fluentd.

I will investigate about SSL/TLS support to influent.
Do you have any idea? @okumin

@okumin
Copy link
Owner

@okumin okumin commented Jul 14, 2017

@okkez I'm planning to implement all the feature defined in Forward Protocol Specification v1.
But I have no idea abount TLS now, so it is very happy if you help this project!

@okkez
Copy link
Contributor Author

@okkez okkez commented Jul 19, 2017

@okumin Thank you replying! I'm investigating for this.

We need to implement NioSslAcceptor implements NioAttachment and NioSslChannel implements AutoClosable. And then we can add related parameters to NioForwardServer.
Is this right? If wrong, could you help me?

FYI: Sample code TLS server client https://github.com/alkarn/sslengine.example

@okkez
Copy link
Contributor Author

@okkez okkez commented Jul 21, 2017

@okumin
Copy link
Owner

@okumin okumin commented Jul 22, 2017

@okkez Looks good!
Are you going to complete this work?
If so, I hope for you to PR!
I'm learning TLS on nio and waiting that.

Now Influent's test codes are implemented in Scala, so I don't require you to write tests.

@okkez okkez mentioned this issue Jul 22, 2017
@okkez
Copy link
Contributor Author

@okkez okkez commented Jul 22, 2017

Are you going to complete this work?

I want to complete this work in several days.
I've created new PR #22, but it is work in progress.
@okumin Thank you for your offer. Feel free to comment or create PRs.
I'm not Java expert so I want your comment and PRs.

I've trying to run TLS support with Fluentd v0.14.x, but it has not work yet...
I tried to create keys and certs using following command (not work):

$ keytool -genkey -keystore ./keystore.jks -storepass password -alias fluentd
$ keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.pkcs12 -srcstoretype jks -deststoretype pkcs12
$ openssl pkcs12 -in keystore.pkcs12 -clcerts -nokeys -nodes -out keystore_cert.pem
$ openssl pkcs12 -in keystore.pkcs12 -nocerts -nodes -out keystore_key.pem
@okumin
Copy link
Owner

@okumin okumin commented Jul 22, 2017

OK, I wrote some comments.

I have read some documents and alkarn/sslengine.example) and I'm feeling some problems.

  1. Implementing SSL/TLS functions to (mainly) influent-transport
  2. Providing SSL/TLS configuration API to ForwardServer.Builder
  3. Performance optimizations

The first, I want to focus 1 and 2.
Since SSL/TLS requires some extra buffer and memory copy, it may be possible to optimize.
I have a plan to resolve such a problem at #14.

But only thread-blocking has big impact to performance and API design and should not be compromised.

@okumin
Copy link
Owner

@okumin okumin commented Jul 22, 2017

I've trying to run TLS support with Fluentd v0.14.x, but it has not work yet...

I'll try that.

@okumin
Copy link
Owner

@okumin okumin commented Aug 6, 2017

@okkez I published .
This version provides SSL/TLS support.
But notice that 0.4.0-M1 is not stable and TLS implementation has bugs and performance issues with very high probability.
This implementation is built on top of the previous structure and not good yet.

Usage

1. Generate private CA

Execute bin/generate-ca.sh.
out/ca_key.pem and out/ca_cert.pem are generated.

./bin/generate-ca.sh

2. Generate a key store for Influent

Execute bin/generate-server-keystore.sh.
Note that server's common name should be the host name(localhost is useful if client coexists with server).
out/influent-server.jks is generated.

./bin/generate-server-keystore.sh

3. Configure fluentd

tls_cert_path is required.

<source>
  @type dummy
  tag mofu
  dummy {"hello":"world"}
</source>

<match *>
  @type forward
  flush_interval 1s
  require_ack_response true

  transport tls
  tls_cert_path /path/to/ca_cert.pem

  <server>
    name localhost
    host localhost
    port 24224
  </server>
</match>

4. Start server

sbt "project influentJavaSample" "runMain sample.TLSPrint 1000"

5. Start client

fluentd -c /path/to/fluent.conf
@okumin
Copy link
Owner

@okumin okumin commented Aug 6, 2017

I have upgraded kafka-connect-fluentd.
fluent/kafka-connect-fluentd#4

I'm sorry I'm late.

@okkez
Copy link
Contributor Author

@okkez okkez commented Aug 7, 2017

Thanks! I will test SSL/TLS support.

@okkez
Copy link
Contributor Author

@okkez okkez commented Aug 8, 2017

It works on my local environment.

@okumin
Copy link
Owner

@okumin okumin commented Aug 8, 2017

Thanks for your confirmation!

@okumin okumin closed this Aug 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.