New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

having issues with SSL options - what formats are allowed? #197

Open
paul42 opened this Issue Apr 5, 2017 · 11 comments

Comments

Projects
None yet
6 participants
@paul42

paul42 commented Apr 5, 2017

9:37:08.516 [nioEventLoopGroup-4-1] ERROR logstash.inputs.beats - Looks like you either have an invalid key or your private key was not in PKCS8 format. {:exception=>java.lang.IllegalArgumentException: File does not contain valid certificates: /etc/pki/tls/certs/{REDACTED}.der.crt}

why is it saying my key is invalid and then saying my cert file doesn't contain valid certificates? what forms are required for options (DER or PEM?)

@joelio

This comment has been minimized.

Show comment
Hide comment
@joelio

joelio Jun 19, 2017

I've had a similar issue when reusing puppet's CA infrastructure - it can be converted with something like

openssl pkcs8 -topk8 -inform PEM -outform PEM -in {old}.pem -out {new}.pem -nocrypt

joelio commented Jun 19, 2017

I've had a similar issue when reusing puppet's CA infrastructure - it can be converted with something like

openssl pkcs8 -topk8 -inform PEM -outform PEM -in {old}.pem -out {new}.pem -nocrypt

@jstoja

This comment has been minimized.

Show comment
Hide comment
@jstoja

jstoja Dec 5, 2017

@jordansissel I'm currently struggling with the certificates as mentioned in the just linked issue on logstash-core.
I'm ready to help improving the documentation if I find a way to make it work with self signed, but for now, even when specifying my own CA cert, it fails complaining with the same error as described here...
I can reproduce by creating the cert and key with:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout logstash.key -out logstash.crt
openssl pkcs8 -in logstash.key -topk8 -nocrypt -out logstash.p8

and

./bin/logstash -e "input {beats {port => 5000 ssl => true ssl_verify_mode => 'none' ssl_certificate => '/etc/ssl/logstash/logstash.crt' ssl_key => '/etc/ssl/logstash/logstash.p8' tags => ['beats']}}"

Any idea?

jstoja commented Dec 5, 2017

@jordansissel I'm currently struggling with the certificates as mentioned in the just linked issue on logstash-core.
I'm ready to help improving the documentation if I find a way to make it work with self signed, but for now, even when specifying my own CA cert, it fails complaining with the same error as described here...
I can reproduce by creating the cert and key with:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout logstash.key -out logstash.crt
openssl pkcs8 -in logstash.key -topk8 -nocrypt -out logstash.p8

and

./bin/logstash -e "input {beats {port => 5000 ssl => true ssl_verify_mode => 'none' ssl_certificate => '/etc/ssl/logstash/logstash.crt' ssl_key => '/etc/ssl/logstash/logstash.p8' tags => ['beats']}}"

Any idea?

@jstoja

This comment has been minimized.

Show comment
Hide comment
@jstoja

jstoja Dec 11, 2017

Sorry for pushing, I'm sure many people also had issues like this and I think adding the solution to the doc will help! @ph ?

jstoja commented Dec 11, 2017

Sorry for pushing, I'm sure many people also had issues like this and I think adding the solution to the doc will help! @ph ?

@jstoja

This comment has been minimized.

Show comment
Hide comment
@jstoja

jstoja Dec 18, 2017

I got confirmation from the code that pkcs8 is used by Netty's library:

SslContextBuilder builder = SslContextBuilder.forServer(sslCertificateFile, sslKeyFile, passPhrase);

This is the attached documentation that matches what I've done...

https://netty.io/wiki/sslcontextbuilder-and-private-key.html

I solved my case, it was a very simple unix rights issue. This is sad, the error message let me thought that it was an openssl format issue... ( I hope this will help someone).

jstoja commented Dec 18, 2017

I got confirmation from the code that pkcs8 is used by Netty's library:

SslContextBuilder builder = SslContextBuilder.forServer(sslCertificateFile, sslKeyFile, passPhrase);

This is the attached documentation that matches what I've done...

https://netty.io/wiki/sslcontextbuilder-and-private-key.html

I solved my case, it was a very simple unix rights issue. This is sad, the error message let me thought that it was an openssl format issue... ( I hope this will help someone).

@jordansissel

This comment has been minimized.

Show comment
Hide comment
@jordansissel

jordansissel Dec 18, 2017

Contributor

@jstoja when I am doing lab testing/experiments, I use one command for generating my cert/key:

openssl req -x509  -batch -nodes -newkey rsa:2048 -keyout lumberjack.key -out lumberjack.crt -subj /CN=localhost
Contributor

jordansissel commented Dec 18, 2017

@jstoja when I am doing lab testing/experiments, I use one command for generating my cert/key:

openssl req -x509  -batch -nodes -newkey rsa:2048 -keyout lumberjack.key -out lumberjack.crt -subj /CN=localhost
@jstoja

This comment has been minimized.

Show comment
Hide comment
@jstoja

jstoja Dec 22, 2017

@jordansissel How can this be compatible with Netty's requirement to have a key with PKCS8 format?

jstoja commented Dec 22, 2017

@jordansissel How can this be compatible with Netty's requirement to have a key with PKCS8 format?

@jordansissel

This comment has been minimized.

Show comment
Hide comment
@jordansissel

jordansissel Dec 28, 2017

Contributor

@jstoja I don't understand your question. The command I gave produces a certificate and private key that is usable for testing (again, this is how I test).

% openssl req -x509  -batch -nodes -newkey rsa:2048 -keyout lumberjack.key -out lumberjack.crt -subj /CN=localhost
Generating a 2048 bit RSA private key
........+++
..................+++
writing new private key to 'lumberjack.key'
-----
% bin/logstash -e 'input { beats { port => 12345 ssl_certificate => "lumberjack.crt" ssl_key => "lumberjack.key" ssl => true } }'
[2017-12-23T08:19:21,039][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.1.0"}
[2017-12-23T08:19:24,921][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>40, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>5000, :thread=>"#<Thread:0x75f6f4be run>"}
[2017-12-23T08:19:25,390][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:12345"}
[2017-12-23T08:19:25,480][INFO ][logstash.pipeline        ] Pipeline started {"pipeline.id"=>"main"}
...

The above demonstrates success with Logstash 6.1.0 and my one-liner key+cert generator (which I use for quick one-shot tests).

Contributor

jordansissel commented Dec 28, 2017

@jstoja I don't understand your question. The command I gave produces a certificate and private key that is usable for testing (again, this is how I test).

% openssl req -x509  -batch -nodes -newkey rsa:2048 -keyout lumberjack.key -out lumberjack.crt -subj /CN=localhost
Generating a 2048 bit RSA private key
........+++
..................+++
writing new private key to 'lumberjack.key'
-----
% bin/logstash -e 'input { beats { port => 12345 ssl_certificate => "lumberjack.crt" ssl_key => "lumberjack.key" ssl => true } }'
[2017-12-23T08:19:21,039][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.1.0"}
[2017-12-23T08:19:24,921][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>40, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>5000, :thread=>"#<Thread:0x75f6f4be run>"}
[2017-12-23T08:19:25,390][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:12345"}
[2017-12-23T08:19:25,480][INFO ][logstash.pipeline        ] Pipeline started {"pipeline.id"=>"main"}
...

The above demonstrates success with Logstash 6.1.0 and my one-liner key+cert generator (which I use for quick one-shot tests).

@jstoja

This comment has been minimized.

Show comment
Hide comment
@jstoja

jstoja Dec 29, 2017

Tested it and it works well. I'm a bit confused with the key formats required and how I generated my previous keypairs.
The only issue this test shows is the -e/-f option that cannot be used together and fail in some way.

Thank you very much for your answers, I'll use that from now on.

jstoja commented Dec 29, 2017

Tested it and it works well. I'm a bit confused with the key formats required and how I generated my previous keypairs.
The only issue this test shows is the -e/-f option that cannot be used together and fail in some way.

Thank you very much for your answers, I'll use that from now on.

@peksilli

This comment has been minimized.

Show comment
Hide comment
@peksilli

peksilli Apr 26, 2018

@jstoja "it was a very simple unix rights issue" - can you be more specific? Struggling with the same just now.

peksilli commented Apr 26, 2018

@jstoja "it was a very simple unix rights issue" - can you be more specific? Struggling with the same just now.

@jstoja

This comment has been minimized.

Show comment
Hide comment
@jstoja

jstoja Apr 26, 2018

@peksilli I finally used the one-liner of @jordansissel to generate it.
The things to watch out are:

  • the CN
  • the unix rights of the file
  • the format of the keypair

I honestly didn't take time to dig into why, I was a bit in a rush and used the solution above.

jstoja commented Apr 26, 2018

@peksilli I finally used the one-liner of @jordansissel to generate it.
The things to watch out are:

  • the CN
  • the unix rights of the file
  • the format of the keypair

I honestly didn't take time to dig into why, I was a bit in a rush and used the solution above.

@sufiyanghori1

This comment has been minimized.

Show comment
Hide comment
@sufiyanghori1

sufiyanghori1 May 30, 2018

Could it possibly be the same issue as spujadas/elk-docker#112 ?

sufiyanghori1 commented May 30, 2018

Could it possibly be the same issue as spujadas/elk-docker#112 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment