Certificates

Manually Generating Certificates

We first need to get a Certificate Authority (CA) Certificate from the elasticsearch container.

Run the docker-compose.setup.yml with the following:

docker-compose -f docker-compose.setup.yml up -d

Get CA Certificate

Once this container is running then we need to exec into the container:

docker-compose exec certs bash

Once in the container we then invoke the built-in executable in the bin directory to generate our CA certificate:

CA_ZIP=/secrets/ca.zip
bin/elasticsearch-certutil ca -s --pass "" --pem --out $CA_ZIP

Getting Certificates

Please note that I am creating a certificate for all other services (e.g. kibana, logstash) but depending on your setup you probably should create one for each

Let's use our recently generated CA certificate to generate a certificate. You should still be in the same container we were already in to generate the CA certificate:

bin/elasticsearch-certutil cert -s --ca-cert /secrets/ca/ca.crt --ca-key /secrets/ca/ca.key --ca-pass "" --pem --in /usr/share/elasticsearch/config/instances.yml --out /secrets/bundle.zip

While we are still in this container, let's set passwords for all user accounts

Converting Logstash Key to PKCS8 format

Now that we have these files, let's now generate an actual .pem file needed by logstash. You do this using openssl:

yum install unzip openssl -y
openssl pkcs8 -in $OUTPUT_DIR/logstash/logstash.key -topk8 -nocrypt -out /secrets/logstash/logstash.pkcs8.key

Optional Steps

The following are optional steps and have not been fully tested but definitely should work :)

Set Passwords for all users

Probably best to use the same password for all users when in a demo enviornment only but you can find more information about setting passwords here.

Get PEM for Kibana

This is actually outputted as a crt and key in a zip file

Run the following command to generate a PEM file for Kibana

bin/elasticsearch-certutil cert --pem -ca elastic-stack-ca.p12

Copying Files to local system

Now that we have generated the necessary files, let's exit the container by typing exit and while in the same folder as your docker-compose.setup.yml let's run the following:

docker cp {CONTAINER_ID}:/usr/share/elasticsearch/elastic-certificates.p12 secrets/elastic-certificates.p12
docker cp {CONTAINER_ID}:/usr/share/elasticsearch/elastic-stack-ca.p12 secrets/elastic-stack-ca.p12
docker cp {CONTAINER_ID}:/usr/share/elasticsearch/certificate-bundle.zip secrets/certificate-bundle.zip

# Finally let's unzip the contents of the certificate-bundle.zip and put them in the secrets folder
unzip secrets/certificate-bundle.zip -d ./secrets

Finish

That's it - well it's a pain in the butt and took awhile to figure this out but for you that's it :)

You should have the following files in your secrets directory:

📦secrets
 ┣ 📂certificate_authority
 ┃ ┣ 📂ca
 ┃ ┃ ┣ 📜ca.crt
 ┃ ┃ ┗ 📜ca.key
 ┃ ┣ 📜ca.zip
 ┃ ┗ 📜elastic-stack-ca.p12
 ┣ 📂certificates
 ┃ ┣ 📂elasticsearch
 ┃ ┃ ┣ 📜elasticsearch.crt
 ┃ ┃ ┗ 📜elasticsearch.key
 ┃ ┣ 📂filebeat
 ┃ ┃ ┣ 📜filebeat.crt
 ┃ ┃ ┗ 📜filebeat.key
 ┃ ┣ 📂kibana
 ┃ ┃ ┣ 📜kibana.crt
 ┃ ┃ ┗ 📜kibana.key
 ┃ ┣ 📂logstash
 ┃ ┃ ┣ 📜logstash.crt
 ┃ ┃ ┣ 📜logstash.key
 ┃ ┃ ┗ 📜logstash.pkcs8.key
 ┃ ┣ 📂metricbeat
 ┃ ┃ ┣ 📜metricbeat.crt
 ┃ ┃ ┗ 📜metricbeat.key
 ┃ ┗ 📂packetbeat
 ┃ ┃ ┣ 📜packetbeat.crt
 ┃ ┃ ┗ 📜packetbeat.key
 ┣ 📂keystores
 ┃ ┣ 📂elasticsearch
 ┃ ┃ ┗ 📜elasticsearch.p12
 ┃ ┣ 📂filebeat
 ┃ ┃ ┗ 📜filebeat.p12
 ┃ ┣ 📂kibana
 ┃ ┃ ┗ 📜kibana.p12
 ┃ ┣ 📂logstash
 ┃ ┃ ┗ 📜logstash.p12
 ┃ ┣ 📂metricbeat
 ┃ ┃ ┗ 📜metricbeat.p12
 ┃ ┗ 📂packetbeat
 ┃ ┃ ┗ 📜packetbeat.p12
 ┣ 📜bundle.zip
 ┣ 📜cert_keystores.zip
 ┗ 📜elasticsearch.keystore