URL rewriting functionality

[mbaig25]

Can there possible be a label to specifiy that the backend config that generated should rewrite the base path from the frontend.

Currently having an issue where I'm trying to put kibana behind the docker-flow-poxy but apparently it requires a url rewrite.

[hamburml]

You can use regPathSearch and regPathReplace for that. See http://proxy.dockerflow.com/swarm-mode-auto/#rewriting-paths.

I use that for grafana. Here is an example:

 docker service create \
 --name grafana \
 --network proxy \
 --env GF_SERVER_ROOT_URL="%(protocol)s://%(domain)s:%(http_port)s/grafana/" \
 --label com.df.notify=true \
 --label com.df.distribute=true \
 --label com.df.reqPathSearch=/grafana/ \
 --label com.df.reqPathReplace=/ \
 --label com.df.servicePath=/grafana/,/grafana/public,/grafana/api \
 --label com.df.serviceDomain=subdomain.pitforest.de \
 --label com.df.httpsOnly=true \
 --label com.df.port=3000 \
 grafana/grafana:4.1.1

Grafana can be accessed via https://subdomain.pitforest.de/grafana/. When I remember correctly grafana tries to load urls like https://subdomain.pitforest.de/grafana/something.js but something.js can only reached via url https://subdomain.pitforest.de/something.js. regPathSearch and regPathReplace should help you with your task.

[vfarcic]

@mbaig25 You can find a similar example @hamburml mentioned in https://github.com/vfarcic/docker-flow-stacks/blob/master/metrics/prometheus-grafana-df-proxy.yml#L49

[mbaig25]

@hamburml @vfarcic thank you for responding so quickly you guys are awesome.

I've been trying your suggestions and the URL does get replaced but I keep getting 503 from docker-flow-proxy.

Here are the commands I tried with same result:

docker service create \
    --name kibana \
    --network proxy \
    --network log_net \
    --label com.df.notify=true \
    --label com.df.distribute=true \
    --label com.df.servicePath=/kibana \
    --label com.df.port=5601 \
    --label com.df.reqPathSearch='/kibana' \
    --label com.df.reqPathReplace='/' \
    --constraint 'node.labels.logger == true' \
    kibana

docker service create \
    --name kibana \
    --network proxy \
    --network log_net \
    --label com.df.notify=true \
    --label com.df.distribute=true \
    --label com.df.servicePath=/kibana/ \
    --label com.df.port=5601 \
    --label com.df.reqPathSearch='/kibana/' \
    --label com.df.reqPathReplace='/' \
    --constraint 'node.labels.logger == true' \
    kibana

docker service create \
    --name kibana \
    --network proxy \
    --network log_net \
    --env SERVER_BASEPATH="%(protocol)s://%(domain)s:%(http_port)s/kibana" \
    --label com.df.notify=true \
    --label com.df.distribute=true \
    --label com.df.servicePath=/kibana \
    --label com.df.port=5601 \
    --label com.df.reqPathSearch='/kibana' \
    --label com.df.reqPathReplace='/' \
    --constraint 'node.labels.logger == true' \
    kibana

docker service create \
    --name kibana \
    --network proxy \
    --network log_net \
    --env SERVER_BASEPATH="%(protocol)s://%(domain)s:%(http_port)s/app/kibana" \
    --label com.df.notify=true \
    --label com.df.distribute=true \
    --label com.df.servicePath=/kibana \
    --label com.df.port=5601 \
    --label com.df.reqPathSearch='/kibana' \
    --label com.df.reqPathReplace='/' \
    --constraint 'node.labels.logger == true' \
    kibana

I'm not sure why its not being passed through. Then again I've been at this for the past couple days so I might be missing something completely obvious.

Here is the haproxy config:

global
    pidfile /var/run/haproxy.pid
    tune.ssl.default-dh-param 2048

    #disable sslv3, prefer modern ciphers
    ssl-default-bind-options no-sslv3
    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS

    ssl-default-server-options no-sslv3
    ssl-default-server-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS

defaults
    mode    http
    balance roundrobin

    option  dontlognull
    option  dontlog-normal
    option  http-server-close
    option  forwardfor
    option  redispatch

    errorfile 400 /errorfiles/400.http
    errorfile 403 /errorfiles/403.http
    errorfile 405 /errorfiles/405.http
    errorfile 408 /errorfiles/408.http
    errorfile 429 /errorfiles/429.http
    errorfile 500 /errorfiles/500.http
    errorfile 502 /errorfiles/502.http
    errorfile 503 /errorfiles/503.http
    errorfile 504 /errorfiles/504.http

    maxconn 5000
    timeout connect 5s
    timeout client  20s
    timeout server  20s
    timeout queue   30s
    timeout tunnel  3600s
    timeout http-request 5s
    timeout http-keep-alive 15s

    stats enable
    stats refresh 30s
    stats realm Strictly\ Private
    stats auth admin:admin
    stats uri /admin?stats

frontend services
    bind *:80
    bind *:443
    mode http

    acl url_kibana5601 path_beg /kibana
    use_backend kibana-be5601 if url_kibana5601




backend kibana-be5601
    mode http
    http-request set-path %[path,regsub(/kibana,/)]
    server kibana kibana:5601

Thanks again.

[vfarcic]

Here are the commands I executed. I put comments before each.

# Clone the repo with Docker stacks.
git clone https://github.com/vfarcic/docker-flow-stacks

cd docker-flow-stacks/logging

docker network create --driver overlay proxy

docker network create --driver overlay log_net

# Deploy Docker Flow Proxy and Docker Flow Swarm Listener
docker stack deploy -c ../proxy/docker-flow-proxy.yml proxy

# Kibana is special. It does not have a unique path so multiple need to be defined as the servicePath label. Those used below are the paths used by Kibana.
docker service create \
    --name kibana \
    --network proxy \
    --network log_net \
    --label com.df.notify=true \
    --label com.df.distribute=true \
    --label com.df.servicePath=/app,/elasticsearch,/api,/ui,/bundles,/plugins \
    --label com.df.port=5601 \
    kibana

open "http://localhost/app/kibana"

# UI works but it shows a warning since ElasticSearch is not running.

docker service rm kibana

# The same thing but through a stack file together with ElasticSearch, LogSpout, and LogStash
docker stack deploy -c logging-df-proxy.yml logging

open "http://localhost/app/kibana"

# UI works

docker stack rm logging

docker stack rm proxy

The problem with having all those paths (e.g. /app, /api, and so on) is that there might be a different application that uses the same paths. Ideally, Kibana (and any other app), should have a single (potentially unique) path or have it customizable through environment variables. Unfortunately, Kibana is not one of those :( So, all those paths need to be present.

Normally, I would run Kibana on a separate domain. In that case, it would not matter what the path is and labels would be something like:

...
    --label com.df.notify=true \
    --label com.df.serviceDomain=kibana.acme.com \
    --label com.df.distribute=true \
    --label com.df.servicePath=/ \
    --label com.df.port=5601 \
...

Please note that the path is, this time, /. Normally, that would hijack any other service in your cluster (/ matches everything). When combined with a subdomain it is unique so it would not produce a conflict.

Please let me know whether one of those solutions fits your use case. If they don't we can explore other options.

[mbaig25]

@vfarcic Thank you. The paths in your command and option with the service domain solved the issue. I'll close it now.

Just curious though, how did you go about finding out the paths that are required by Kibana?
Is there documentation that I couldn't find? or Was it trial and error when you were in a similar position as me?

Would just like to know. 😄

Thanks again to both of guys.

[vfarcic]

I find out about the paths by inspecting network traffic through Chrome :)

[mbaig25]

@vfarcic Ah simpler than I thought