Environment variables not set in logstash config

[mrcnc]

According to the docs I should be able to specify a configuration file that can use environment variables for the pipeline config. But when I try to set environment variables and mount this config file, it doesn't appear to work.

input { stdin { } }
output {
  elasticsearch {
    hosts => "${ES_HOSTS}"
    user => "${ES_USERNAME}"
    password => "${ES_PASSWORD}"
    index => "logs"
  }
  stdout { codec => rubydebug }
}

You can reproduce this by mounting the config file above with a command like this:

docker run --rm -it \
-v /path/to/my-logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
-e ES_HOSTS=http://my.elastic.node:9200 \
-e ES_USERNAME=elastic \
-e ES_PASSWORD=mypwd \
 docker.elastic.co/logstash/logstash:5.3.1

I see the error [2017-04-26T19:03:02,580][ERROR][logstash.agent ] Cannot load an invalid configuration {:reason=>"bad URI(is not URI?): ${ES_HOSTS}"} so the config file is copied to the correct location but the environment variables aren't replaced.

Am I missing something, or should this be working?

[untergeek]

Have you tried encapsulating the value in single-quotes?

-e ES_HOSTS='http://my.elastic.node:9200' \

or even including single-quotes or double-quotes inside the outer quotes?

-e ES_HOSTS="'http://my.elastic.node:9200'" \

[jordansissel]

This is likely this bug in Logstash: elastic/logstash#6696

…

On Wed, Apr 26, 2017 at 7:41 PM Aaron Mildenstein ***@***.***> wrote: Have you tried encapsulating the value in single-quotes? -e ES_HOSTS='http://my.elastic.node:9200' \ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIC6pmRiOoDlRhqWQJ03BX7eT_cO8wNks5r0ABPgaJpZM4NJTC8> .

[mrcnc]

@untergeek Still have the same issue regardless of the quotes

@jordansissel Thx for pointing out that bug....I was able to use docker.elastic.co/logstash/logstash:5.1.1 and it's working as expected.

[kazizi-swe]

@mrcnc did you manage to solve it?

Edit:

Fixed the issue temporarily by putting ENV ES_HOSTS in the Dockerfile, but it is not the best practice. If anyone knows how to fix this by using -e in docker run -e <ES_HOSTS> command, please let me know.