ssl_key_passphrase does not work

[nkakouros]

• Version: 7.1.1
• Operating System: Ubuntu 18.04

# beats input config:
input {
  beats {
    id => 'beat-input'
    port => 5044
    host => '0.0.0.0'
    ssl => true
    ssl_certificate => '/etc/logstash/certs/beat_input/all.crt'
    ssl_certificate_authorities => '/etc/logstash/certs/beat_input/ca.crt'
    ssl_key => '/etc/logstash/certs/beat_input/all.key'
    ssl_key_passphrase => '${beat_input_ssl_key_passphrase}'
    }
}

The beat_input_ssl_key_passphrase is the variable name I used to store the password securely in the logstash keystore. I also tried adding the password in plaintext. Logstash fails to start with the error output here: https://pastebin.com/BdsqiWcH

There are other people that seem to have the same issue.

[berglh]

I am also experiencing this problem in Version 7.3.0 in the official docker container.

The same error exists in the HTTP input plugin, which also is unable to decrypt passphrase protected PKCS8 keys.

[berglh]

@ph @robbavey This is a pretty far reaching problem when using encrypted private keys, which should be preferred particularly with the function of the logstash keystore. My understanding is that it's looking at the first line of the file to determine if it's pkcs8, if it doesn't see the right header, it throws it away. I'm also noticing that bug occurs on other input plugins like logstash-input-http and logstash-input-courier. The later is a third party plugin, but I wonder if it's just using the standard SSL libraries and config as per beats.

If there is any additional information to add to troubleshooting, please ask me to provide.

[kares]

the underlying error for this one is very probably the same as identified at #391 (comment):

java.security.NoSuchAlgorithmException: 1.2.840.113549.1.5.13 SecretKeyFactory not available

(the error message/type gets a bit lost with Netty exception handling)

the problem is standard JDK libraries have issues reading PKCS8 encrypted (PEM) keys generated by openssl. there's no easy work-around that (on Java 8/11) other than using a separate library to read the files manually and than somehow supply the key, if possible, to the library (in this case Netty).

[larry-s]

This still does not work in version 7.8. I just wasted a couple of hours trying to get my encrypted certificate to work.

If you are not going to fix it then please al least put a note in the documentation that it doesn't work.

[frankfoti]

Does anyone have ssl working between filebeat and logstash? Ihave followed all the documentation and posts with no luck. I know my certs and keys are working between logstash and Elasticsearch.

[larry-s]

Does anyone have ssl working between filebeat and logstash? Ihave followed all the documentation and posts with no luck. I know my certs and keys are working between logstash and Elasticsearch.

It works for me if I use a certificate that doesn't have a password, so is not encrypted.

[frankfoti]

Still not working for me, getting an error in the elogstash log when I use the ssl config below:
input {
beats {
port => 5046
ssl => true
ssl_certificate_authorities => ["/etc/logstash/ca.crt"]
ssl_certificate => "/etc/logstash/siem-logstash-01.crt"
ssl_key => "/etc/logstash/siem-logstash-01.key"
ssl_verify_mode => "force_peer"
}
}

curl -XGET -u elastic:******** --cacert /etc/logstash/ca.crt --cert /etc/logstash/siem-logstash-01.crt --key /etc/logstash/siem-logstash-01.key 'https://siem-elasticsearch-01:9200/_cluster/health?pretty'
{
"cluster_name" : "siem-poc",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 62,
"active_shards" : 62,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}

Error in logstash log:
[2020-07-30T19:04:50,184][ERROR][logstash.agent ] Failed to execute action {:id=>:beats, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}

[Rl-0ne]

I had similar issue , unable to use passhprase protected key on beats. Service not starts since unable to read the key.
My workaround was to convert key to Triple DES using following command :
openssl rsa -in client.key -des3 -out client_des.key

In this case key header starts with ""-----BEGIN RSA PRIVATE KEY-----Proc-Type: 4,ENCRYPTED" and it works.

When I tried to use pkcs8 format that starts with "-----BEGIN ENCRYPTED PRIVATE KEY-----" it didn't worked for me.