Environment variable configuration should use the same syntax across all the stack images

[outcoldman]

Comparing elasticsearch and kibana:

• kibana does not have x-pack basic image
• kibana does not allow to configure similarly how elasticsearch does with just specifying configuration options from elasticsearch.yml

Hi @outcoldman,

There's no need for separate X-Pack Basic and Platinum images for Kibana. The only difference between those two images for Elasticsearch is the licencing. They contain the same software. Kibana doesn't define the licence level of an Elastic Stack, it just follows along with whatever Elasticsearch says, so we only need one image with X-Pack for all subscription types, and one without for open source only installations.

I don't think I understand the second bullet-point. The Kibana image provides the same configuration mechanisms as the other images, including providing kibana.yml.

[outcoldman]

@Jarpy makes sense about x-pack

about configurations, if you will look on elasticsearch image https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_a_present_the_parameters_via_docker_environment_variables

it allows you to just use environment variable "cluster.name=mynewclustername", but in case of kibana it requires you to upper case everything and replace dots with _.

Oh yes. That is different, we know, and we're sorry. Both solutions have problems, unfortunately.

The dotted notation, for example, means you are creating environment variables that are not valid identifiers in Bash, which is problematic for some people. They are also invalid in Kubernetes < 1.8, which is probably more serious.

The capitalized, uppercased notation is more compatible but has a much higher maintenance burden for us, the image maintainers. With the recent release of Kubernetes 1.8, we can probably (finally) come up with a unified syntax for environment variable configuration, but I wouldn't expect to see it before version 7.0.

[gpthome]

Good to see this issue. +1 from me.

I am trying to deploy and use a simple elasticsearch Docker container in OpenShift. I am following the ES documentation here: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docker.html#docker-cli-run

OpenShift 3.5 follows specific syntax regarding environment variable names. It requires that a variable name be an alphanumeric (a-z and 0-9) string beginning with a letter that may contain underscores. "discovery.type" does not follow these rules, as you all mentioned above.

I have also seen many articles describe that the syntax of environment variables should follow alpha-numeric and underscores as well (no periods).

I look forward to watching this issue and hopefully see it resolved.

Thanks.

[Constantin07]

Having run into this confusion around discrepancy in Kibana & ES with environment variables too.
It would be nice if it was the same convention in whole stack.

[nikolay]

All top-level settings such as processors cannot be set for Elasticsearch, but the Kibana approach is more compatible and allows for it. Now, I'm forced to have a mixture of non-compliant environment variables with dots and my own variables used within elasticsearch.yml. Please, adopt the Kibana approach replacing . with _ and making the variable uppercase! Another approach is to do the reverse - look for an ES_ prefix, and then replace _DOT_ with a dot and lowercase the rest, i.e. processors would be ES_PROCESSORS and host.name would be ES_HOST_DOT_NAME.

[nikolay]

@gpthome I agree. What Elasticsearch uses now is a dirty hack. It's not compliant and leads to toxic inconsistencies. Also, it's limited as it does not allow you to pass top-level settings such as processors.

[nikolay]

By the way, even the Kibana's approach is not perfect - all environment variables need to be prefixed with KIBANA_ or KBN_ to avoid collisions as some parameter names could be pretty generic. If Elasticsearch makes a move, all environment variables should have an ES_ prefix!

Kibana's approach also has a high maintenance burden, since every potential setting has to be explicitly declared in the entrypoint script. That's historically been a big source of bugs for the image.

Regardless, we'll aim to have a unified approach of some kind in version 7.0.

[nikolay]

@Jarpy The drawbacks of the Kibana approach are pretty obvious but having all settings in a structured format would be of great help and could then automatically populate the Dockerfile template instead of having to keep both in sync all the time.

[rjernst]

It looks like there is a long history here, but the burden of keeping an alternate form of environment variable passing is likely too much. The preferred method of passing settings is through elasticsearch.yml. Additionally, maintenance of the docker files for elasticsearch has moved to the elasticsearch repo. As we will be archiving this repository, I am going to close this issue.