Skip to content
This repository has been archived by the owner. It is now read-only.

Environment variable configuration should use the same syntax across all the stack images #135

Closed
outcoldman opened this issue Nov 18, 2017 · 11 comments

Comments

@outcoldman
Copy link

@outcoldman outcoldman commented Nov 18, 2017

Comparing elasticsearch and kibana:

  • kibana does not have x-pack basic image
  • kibana does not allow to configure similarly how elasticsearch does with just specifying configuration options from elasticsearch.yml
@jarpy

This comment has been minimized.

Copy link
Contributor

@jarpy jarpy commented Nov 21, 2017

Hi @outcoldman,

There's no need for separate X-Pack Basic and Platinum images for Kibana. The only difference between those two images for Elasticsearch is the licencing. They contain the same software. Kibana doesn't define the licence level of an Elastic Stack, it just follows along with whatever Elasticsearch says, so we only need one image with X-Pack for all subscription types, and one without for open source only installations.

I don't think I understand the second bullet-point. The Kibana image provides the same configuration mechanisms as the other images, including providing kibana.yml.

@outcoldman

This comment has been minimized.

Copy link
Author

@outcoldman outcoldman commented Nov 22, 2017

@jarpy makes sense about x-pack

about configurations, if you will look on elasticsearch image https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_a_present_the_parameters_via_docker_environment_variables

it allows you to just use environment variable "cluster.name=mynewclustername", but in case of kibana it requires you to upper case everything and replace dots with _.

@jarpy

This comment has been minimized.

Copy link
Contributor

@jarpy jarpy commented Nov 22, 2017

Oh yes. That is different, we know, and we're sorry. Both solutions have problems, unfortunately.

The dotted notation, for example, means you are creating environment variables that are not valid identifiers in Bash, which is problematic for some people. They are also invalid in Kubernetes < 1.8, which is probably more serious.

The capitalized, uppercased notation is more compatible but has a much higher maintenance burden for us, the image maintainers. With the recent release of Kubernetes 1.8, we can probably (finally) come up with a unified syntax for environment variable configuration, but I wouldn't expect to see it before version 7.0.

@jarpy jarpy changed the title Kibana should align with elasticsearch on configuration, images convension Environment variable configuration should use the same syntax across all the stack images Nov 22, 2017
@gpthome

This comment has been minimized.

Copy link

@gpthome gpthome commented Mar 8, 2018

Good to see this issue. +1 from me.

I am trying to deploy and use a simple elasticsearch Docker container in OpenShift. I am following the ES documentation here: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docker.html#docker-cli-run

OpenShift 3.5 follows specific syntax regarding environment variable names. It requires that a variable name be an alphanumeric (a-z and 0-9) string beginning with a letter that may contain underscores. "discovery.type" does not follow these rules, as you all mentioned above.

I have also seen many articles describe that the syntax of environment variables should follow alpha-numeric and underscores as well (no periods).

I look forward to watching this issue and hopefully see it resolved.

Thanks.

@Constantin07

This comment has been minimized.

Copy link

@Constantin07 Constantin07 commented May 2, 2018

Having run into this confusion around discrepancy in Kibana & ES with environment variables too.
It would be nice if it was the same convention in whole stack.

@nikolay

This comment has been minimized.

Copy link

@nikolay nikolay commented Jun 25, 2018

All top-level settings such as processors cannot be set for Elasticsearch, but the Kibana approach is more compatible and allows for it. Now, I'm forced to have a mixture of non-compliant environment variables with dots and my own variables used within elasticsearch.yml. Please, adopt the Kibana approach replacing . with _ and making the variable uppercase! Another approach is to do the reverse - look for an ES_ prefix, and then replace _DOT_ with a dot and lowercase the rest, i.e. processors would be ES_PROCESSORS and host.name would be ES_HOST_DOT_NAME.

@nikolay

This comment has been minimized.

Copy link

@nikolay nikolay commented Jun 25, 2018

@gpthome I agree. What Elasticsearch uses now is a dirty hack. It's not compliant and leads to toxic inconsistencies. Also, it's limited as it does not allow you to pass top-level settings such as processors.

@nikolay

This comment has been minimized.

Copy link

@nikolay nikolay commented Jun 25, 2018

By the way, even the Kibana's approach is not perfect - all environment variables need to be prefixed with KIBANA_ or KBN_ to avoid collisions as some parameter names could be pretty generic. If Elasticsearch makes a move, all environment variables should have an ES_ prefix!

@jarpy

This comment has been minimized.

Copy link
Contributor

@jarpy jarpy commented Jun 26, 2018

Kibana's approach also has a high maintenance burden, since every potential setting has to be explicitly declared in the entrypoint script. That's historically been a big source of bugs for the image.

Regardless, we'll aim to have a unified approach of some kind in version 7.0.

@nikolay

This comment has been minimized.

Copy link

@nikolay nikolay commented Jun 26, 2018

@jarpy The drawbacks of the Kibana approach are pretty obvious but having all settings in a structured format would be of great help and could then automatically populate the Dockerfile template instead of having to keep both in sync all the time.

@rjernst

This comment has been minimized.

Copy link
Member

@rjernst rjernst commented Jun 26, 2019

It looks like there is a long history here, but the burden of keeping an alternate form of environment variable passing is likely too much. The preferred method of passing settings is through elasticsearch.yml. Additionally, maintenance of the docker files for elasticsearch has moved to the elasticsearch repo. As we will be archiving this repository, I am going to close this issue.

@rjernst rjernst closed this Jun 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
6 participants
You can’t perform that action at this time.