Environment variables in the configuration does not work for client url

[dorinand]

From the DOC, I understand, that I can use environment variables in promtail config if I configure promtail with -config.expand-env=true and use variable as ${VAR}.

I configure Loki client url with environment variable as described below:

clients:
  - url: ${LOKI_ADDR}/loki/api/v1/push

I run it as Swarm service:

  promtail:
    image: grafana/promtail:2.6.1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./promtail/config/config.yaml:/etc/promtail/config.yml
      - ./promtail/positions:/positions
    command: -config.file=/etc/promtail/config.yml -config.expand-env=true
    environment:
      - NODE_HOSTNAME={{.Node.Hostname}}
      - LOKI_ADDR="http://loki:3100"

But when I run it, container will not start and raise error:

Unable to parse config: /etc/promtail/config.yml: yaml: line 9: did not find expected key

This is exactly, where - url: ${LOKI_ADDR}/loki/api/v1/push is (I am able to run it when replace env variable for url).

I would expect, I am able to configure client url with environment variable.

I am using environment variables in other places in configuration and its working:

  - static_labels:
      hostname: ${NODE_HOSTNAME}

Promtail version 2.6.1.

I cannot tell, if this is some bug or feature - environment variable can be used in scrape_configs perhaps?

[Cosm0]

@dorinand did you manage to solve this? For other grafana stack apps which do not support env variables in configs I normally use a custom docker image and I copy a config template from my local filesystem to some temp catalogue inside the image. Then I use sed during image build process to substitude variables and copy the resulting config to /etc/promtail/.

I also confirm that it still does not work in 2023/01/22 even though I do this exactly the same as for grafana/agent which also supports env variables substitution in config. It seems to be a bug in promtail. I also neet to mention that it does not work no matter where in config I place the variable. Below is an example of how I tried to use it:

docker-compose.yml

promtail:
    image: grafana/promtail:2.8.7
    container_name: promtail
    environment:
      - LOKI_URL=${LOKI_URL}
      - LOGS_LABELS_PARSE_EXPR=${LOGS_LABELS_PARSE_EXPR}
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers
      - ./promtail/config.yml:/etc/promtail/config.yml
    command:
      - '-config.expand-env=true'
      - '-config.file=/etc/promtail/config.yml'
    networks:
      - monitoring-node-net
    logging: *base-logging

config.yml

server:
  http_listen_address: 0.0.0.0
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: "${LOKI_URL}"

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log

- job_name: containers
  static_configs:
  - targets:
      - localhost
    labels:
      job: containerlogs
      __path__: /var/lib/docker/containers/*/*log

  pipeline_stages:
  - json:
      expressions:
        output: log
        stream: stream
        attrs:
  - json:
      expressions:
        tag:
      source: attrs
  - regex:
      expression: "${LOGS_LABELS_PARSE_EXPR}"
      source: tag
  - timestamp:
      format: RFC3339Nano
      source: time
  - labels:
      tag:
      stream:
      host:
      instance:
      container_name:
  - output:
      source: output

Please note that it makes no difference if I put the env variables in ", ' or in nothing at all.

For those looking for a workaround I paste my solution below. This is for prometheus, but it can also be applied for promtail.

FROM prom/prometheus:v2.48.0

ARG ADMIN_PASSWORD
ARG AGENT_PASSWORD
ARG GRAFANA_PASSWORD
RUN mkdir -p /home/temp/alerts
COPY ./prometheus/*.yml /home/temp/
COPY ./prometheus/alerts/* /home/temp/alerts/
RUN  sed -E "s|\\$\\{ADMIN_PASSWORD\\}|$ADMIN_PASSWORD|g" -i /home/temp/web.yml
RUN  sed -E "s|\\$\\{AGENT_PASSWORD\\}|$AGENT_PASSWORD|g" -i /home/temp/web.yml
RUN  sed -E "s|\\$\\{GRAFANA_PASSWORD\\}|$GRAFANA_PASSWORD|g" -i /home/temp/web.yml
RUN cp /home/temp/*.yml /etc/prometheus/
RUN cp -r /home/temp/alerts /etc/prometheus/

[JStickler]

Questions have a better chance of being answered if you ask them on the community forums.

Closing due to the age of this question.