Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker-compose setup #28

Merged
merged 15 commits into from
Jun 3, 2020
Merged

Add docker-compose setup #28

merged 15 commits into from
Jun 3, 2020

Conversation

ytjohn
Copy link
Contributor

@ytjohn ytjohn commented May 6, 2020

Docker-compose deployment based on one-service-per-container Docker images (https://github.com/StackStorm/st2-dockerfiles) with Ubuntu 18.04 LTS as a base OS and Python 3.

@arm4b arm4b added feature help wanted Extra attention is needed labels May 8, 2020
Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks for sharing!

Yes, more 👀 and manual testing for this PR would be appreciated.

If everything goes well with the functionality & working state of this one, docker-compose will be moved to st2-docker project and eventually replace the https://github.com/stackstorm/st2-docker repo contents.

This way we could give a second life for the https://github.com/stackstorm/st2-docker/ which had no contributor for a long time with unsupported Ubuntu Trusty.

@arm4b
Copy link
Member

arm4b commented May 8, 2020

As #16 merged, this deployment is based on new Ubuntu Bionic images now.

@winem
Copy link
Contributor

winem commented May 8, 2020

Sorry, had to post this comment again as I used the wrong account before. This one was originally posted as direct reply to the issue and before armab's first reply.

"I really like the idea and would like to contribute to it. I was just working on an extension of st2-docker that adds prometheus and grafana with preprovisioned dashboards, etc. so that the user has a full st2 environment incl. monitoring & visualization which can be helpful during the evaluation.

Can we summarize here what it takes to get this merged? I'll test the PR in my lab on the weekend and am happy to contribute and push the docker-compose setup. :)"

@winem
Copy link
Contributor

winem commented May 8, 2020

I spent some time this evening to play around with the setup and it works just great. No flaws or issues so far. Things I tried:

  • un-/installing packages
  • running a bunch of actions from the core, linux and github packs
  • create and enable rules

I'll spend some more time with this setup in the next days and keep you posted.

Copy link
Contributor

@winem winem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a few notes while most of them are basically about the wording. The most important one is the one regarding the exposed st2web HTTP port already being bound to 0.0.0.0.

stackstorm-compose/README.md Outdated Show resolved Hide resolved
stackstorm-compose/README.md Outdated Show resolved Hide resolved
stackstorm-compose/st2.docker.conf Outdated Show resolved Hide resolved
stackstorm-compose/st2.user.conf Outdated Show resolved Hide resolved
@winem
Copy link
Contributor

winem commented May 9, 2020

I found an issue with docker-compose restart. (You can scroll down to the bottom as I found the solution while writing down my troubleshooting notes if you want).

docker-compose restart restarts all containers as expected and they are running afterwards but I have the issue on ~2/5 restarts that the st2web container is still using the old IP addresses of the st2stream and or st2api containers. Therefore requests to st2web return a 503:

Selection_118

Example

Running containers

(If you want to try it on your own, the command to get the output as md table is docker inspect $(docker ps -q) | jq -r '["|", "Container name", "|", " private IP ", "|", "public IP", "|"], (.[] | ["|", .Name, "|", .NetworkSettings.Networks["stackstorm-compose_private"].IPAddress, "|", .NetworkSettings.Networks["stackstorm-compose_public"].IPAddress, "|"]) | join(" ")')

Container name Private IP Public IP
"/stackstorm-compose_st2web_1" "172.20.0.15" "172.22.0.2"
"/stackstorm-compose_st2resultstracker_1" "172.20.0.6" null
"/stackstorm-compose_st2timersengine_1" "172.20.0.7" null
"/stackstorm-compose_st2actionrunner_1" "172.20.0.8" null
"/stackstorm-compose_st2workflowengine_1" "172.20.0.10" null
"/stackstorm-compose_st2garbagecollector_1" "172.20.0.13" null
"/stackstorm-compose_st2stream_1" "172.20.0.11" null
"/stackstorm-compose_st2scheduler_1" "172.20.0.9" null
"/stackstorm-compose_st2auth_1" "172.20.0.14" null
"/stackstorm-compose_st2sensorcontainer_1" "172.20.0.17" null
"/stackstorm-compose_st2rulesengine_1" "172.20.0.16" null
"/stackstorm-compose_st2notifier_1" "172.20.0.12" null
"/stackstorm-compose_st2api_1" "172.20.0.5" null
"/stackstorm-compose_rabbitmq_1" "172.20.0.2" null
"/stackstorm-compose_redis_1" "172.20.0.3" null
"/stackstorm-compose_mongo_1" "172.20.0.4" null

docker-compose restart

Works as expected and ends up in:

marcel@PWS2608:~/Sources/github/pr-validation/st2-dockerfiles/stackstorm-compose$ docker-compose restart 
Restarting stackstorm-compose_st2web_1              ... done
Restarting stackstorm-compose_st2resultstracker_1   ... done
Restarting stackstorm-compose_st2timersengine_1     ... done
Restarting stackstorm-compose_st2actionrunner_1     ... done
Restarting stackstorm-compose_st2workflowengine_1   ... done
Restarting stackstorm-compose_st2garbagecollector_1 ... done
Restarting stackstorm-compose_st2stream_1           ... done
Restarting stackstorm-compose_st2scheduler_1        ... done
Restarting stackstorm-compose_st2auth_1             ... done
Restarting stackstorm-compose_st2sensorcontainer_1  ... done
Restarting stackstorm-compose_st2rulesengine_1      ... done
Restarting stackstorm-compose_st2notifier_1         ... done
Restarting stackstorm-compose_st2api_1              ... done
Restarting stackstorm-compose_rabbitmq_1            ... done
Restarting stackstorm-compose_redis_1               ... done
Restarting stackstorm-compose_st2client_1           ... done
Restarting stackstorm-compose_mongo_1               ... done

This is with a delay of 32 minutes but it shows that all containers were restarted at the same time:

marcel@ITS0873:~/Sources/github/pr-validation/st2-dockerfiles/stackstorm-compose$ docker ps 
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS              PORTS                                NAMES
fc4e420759fb        stackstorm/st2web:3.3dev                "/bin/bash -c 'if [ …"   24 hours ago        Up 32 minutes       443/tcp, 0.0.0.0:8000->80/tcp        stackstorm-compose_st2web_1
e6607d6a9ede        stackstorm/st2resultstracker:3.3dev     "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2resultstracker_1
bd1cead1f1d8        stackstorm/st2timersengine:3.3dev       "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2timersengine_1
5a19d36f6c9a        stackstorm/st2actionrunner:3.3dev       "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2actionrunner_1
4bc6cc5cf047        stackstorm/st2workflowengine:3.3dev     "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2workflowengine_1
66d211889ea6        stackstorm/st2garbagecollector:3.3dev   "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2garbagecollector_1
4473a569d799        stackstorm/st2stream:3.3dev             "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes       9102/tcp                             stackstorm-compose_st2stream_1
a94416c1dc61        stackstorm/st2scheduler:3.3dev          "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2scheduler_1
a3d7dd930ba2        stackstorm/st2auth:3.3dev               "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes       9100/tcp                             stackstorm-compose_st2auth_1
6cdd71034ec3        stackstorm/st2sensorcontainer:3.3dev    "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2sensorcontainer_1
28aaa55a30e8        stackstorm/st2rulesengine:3.3dev        "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2rulesengine_1
6f493ac28c47        stackstorm/st2garbagecollector:3.3dev   "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes                                            stackstorm-compose_st2notifier_1
99389e6dfa5e        stackstorm/st2api:3.3dev                "/opt/stackstorm/st2…"   24 hours ago        Up 32 minutes       9101/tcp                             stackstorm-compose_st2api_1
42a9d4adc74b        rabbitmq:3.8                            "docker-entrypoint.s…"   24 hours ago        Up 32 minutes       4369/tcp, 5671-5672/tcp, 25672/tcp   stackstorm-compose_rabbitmq_1
3c0d0c3e4637        redis:6.0                               "docker-entrypoint.s…"   24 hours ago        Up 32 minutes       6379/tcp                             stackstorm-compose_redis_1
38e3cedf2913        mongo:3.4                               "docker-entrypoint.s…"   24 hours ago        Up 32 minutes       27017/tcp                            stackstorm-compose_mongo_1

Now running containers

Container name Private IP Public IP
"/stackstorm-compose_st2web_1" "172.20.0.15" "172.22.0.2"
"/stackstorm-compose_st2resultstracker_1" "172.20.0.6" null
"/stackstorm-compose_st2timersengine_1" "172.20.0.11" null
"/stackstorm-compose_st2actionrunner_1" "172.20.0.9" null
"/stackstorm-compose_st2workflowengine_1" "172.20.0.10" null
"/stackstorm-compose_st2garbagecollector_1" "172.20.0.14" null
"/stackstorm-compose_st2stream_1" "172.20.0.16" null
"/stackstorm-compose_st2scheduler_1" "172.20.0.8" null
"/stackstorm-compose_st2auth_1" "172.20.0.13" null
"/stackstorm-compose_st2sensorcontainer_1" "172.20.0.17" null
"/stackstorm-compose_st2rulesengine_1" "172.20.0.5" null
"/stackstorm-compose_st2notifier_1" "172.20.0.7" null
"/stackstorm-compose_st2api_1" "172.20.0.12" null
"/stackstorm-compose_rabbitmq_1" "172.20.0.2" null
"/stackstorm-compose_redis_1" "172.20.0.3" null
"/stackstorm-compose_mongo_1" "172.20.0.4" null

So the IPs changed.

The error

st2web access log if the upstreams aren't resolved to the new IP:

2020/05/09 22:23:18 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.20.0.1, server: , request: "GET /stream/stream HTTP/1.1", upstream: "http://172.20.0.11:9102/stream", host: "localhost:8000", referrer: "http://localhost:8000/"
2020/05/09 22:23:19 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.20.0.1, server: , request: "GET /stream/stream HTTP/1.1", upstream: "http://172.20.0.11:9102/stream", host: "localhost:8000", referrer: "http://localhost:8000/"

172.20.0.11 is still the old IP.

Verification that the DNS resolution itself works

docker exec -it stackstorm-compose_st2web_1 sh
root@fc4e420759fb:~# curl -v  st2stream
* Rebuilt URL to: st2stream/
*   Trying 172.20.0.16...
* TCP_NODELAY set
* connect to 172.20.0.16 port 80 failed: Connection refused
* Failed to connect to st2stream port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to st2stream port 80: Connection refused

Solution

Initially I assumed an issue with the order of the containers being restarted or something like that which takes more time but I actually now the solution already because the same issue hit us at the company like 6 months ago. So it was pretty helpful to write everything down. :)

I'll keep the content above in the comment as this might be helpful for others when troubleshooting something.

So, the issue is that nginx resolves all the upstream hosts on startup and caches these IP addresses forever with the default configuration.

The fix

  1. configure the resolver to 127.0.0.11 valid=1s which is the default Docker DNS resolver and tells nginx to resolve the upstream hosts every second
  2. define the upstream host in a variable like set api_upstream http://st2api:9101/;, set stream_upstream http://st2stream:9102/ and set auth_upstream http://st2auth:9100/
  3. use the variables in the proxy_pass configuration

So the new /etc/nginx/conf.d/st2.conf would look like this:

#
# nginx configuration to expose st2 webui, and reverse-proxy the st2api and
# st2auth API endpoints.
# To enable:
#    cp ${LOCATION}/st2.conf /etc/nginx/sites-available
#    ln -l /etc/nginx/sites-available/st2.conf /etc/nginx/sites-enabled/st2.conf
# see https://docs.stackstorm.com/install.html for details

server {
  listen *:80 default_server;

  add_header X-Content-Type-Options nosniff;

  index  index.html;

  access_log /proc/self/fd/1 combined;
  error_log stderr;

  resolver 127.0.0.11 valid=1s;

  location @apiError {
    add_header Content-Type application/json always;
    return 503 '{ "faultstring": "Nginx is unable to reach st2api. Make sure service is running." }';
  }

  location /api/ {
    error_page 502 = @apiError;

    rewrite ^/api/(.*)  /$1 break;

    set api_upstream http://st2api:9101/;

    proxy_pass            $api_upstream;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;

    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;

    proxy_set_header Connection '';
    chunked_transfer_encoding off;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Host $host;
  }

  location @streamError {
    add_header Content-Type text/event-stream;
    return 200 "retry: 1000\n\n";
  }

  # For backward compatibility reasons, rewrite requests from "/api/stream"
  # to "/stream/v1/stream" and "/api/v1/stream" to "/stream/v1/stream"
  rewrite ^/api/stream/?$ /stream/v1/stream break;
  rewrite ^/api/(v\d)/stream/?$ /stream/$1/stream break;
  location /stream/ {
    error_page 502 = @streamError;

    rewrite ^/stream/(.*)  /$1 break;

    set stream_upstream http://st2stream:9102/;

    proxy_pass  $stream_upstream;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    # Disable buffering and chunked encoding.
    # In the stream case we want to receive the whole payload at once, we don't
    # want multiple chunks.
    proxy_set_header Connection '';
    chunked_transfer_encoding off;
    proxy_buffering off;
    proxy_cache off;
  }

  location @authError {
    add_header Content-Type application/json always;
    return 503 '{ "faultstring": "Nginx is unable to reach st2auth. Make sure service is running." }';
  }

  location /auth/ {
    error_page 502 = @authError;

    rewrite ^/auth/(.*)  /$1 break;

    set auth_upstream http://st2auth:9100/;

    proxy_pass            $auth_upstream;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_redirect        off;

    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass_header     Authorization;

    proxy_set_header Connection '';
    chunked_transfer_encoding off;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Host $host;
  }

  location / {
    root      /opt/stackstorm/static/webui/;
    index     index.html;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
  }

}

For some reason all 3 changes (setting the resolver and defining + using the variable) are required to enforce that nginx resolves the upstream hosts at runtime. So it's not enough to set just the resolver for example.

Helpful links on this topic:
https://dev.to/danielkun/nginx-everything-about-proxypass-2ona (scroll down to Let nginx start even when not all upstream hosts are available)
https://www.nginx.com/blog/dns-service-discovery-nginx-plus/

I noticed that this issue is not just an issue with the docker-compose branch and will create a PR for st2-dockerfiles, too.

volumes:
- stackstorm-mongodb:/data/db
dns_search: .
rabbitmq:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that rabbitmq starts up delayed and makes st2api fail to start. We need another healtcheck here:

2020-05-14 09:43:40,020 DEBUG [-] Using Python: 2.7.12 (/opt/stackstorm/st2/bin/python)

2020-05-14 09:43:40,020 DEBUG [-] Using config files: /etc/st2/st2.conf,/etc/st2/st2.docker.conf,/etc/st2/st2.user.conf

2020-05-14 09:43:40,021 DEBUG [-] Using logging config: /etc/st2/logging.api.gunicorn.conf

2020-05-14 09:43:40,063 INFO [-] Connecting to database "st2" @ "mongo:27017" as user "None".

2020-05-14 09:43:40,115 INFO [-] Successfully connected to database "st2" @ "mongo:27017" as user "None".

2020-05-14 09:44:06,143 ERROR [-] Connections to RabbitMQ cannot be re-established: [Errno 111] ECONNREFUSED

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing my comment, we don't need a healthcheck. We need the "wait-for-it.sh" here

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, adding restart: on-failure on st2api (and others) solves the problem the as well. But I'd prefer wait-for-it for a cleaner startup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen outright startup failure in my testing. Most of the containers depend on rabbitmq and most will throw ECONNREFUSED during cold (re)start, but once RabbitMQ initializes they all picked right up. Before I setup the "depends on", order the logs were spammed with ECONNREFUSED.

Are you seeing st2api error out and then you have to restart it by hand? We could be seeing different results based on speed and cpu of the host.

stackstorm-ha containers behave in a manner similiar to the suggested restart: on-failure option. When I do a deploy, most of the containers may end up restarting 2-8 times while waiting for rabbitmq to get its act together (and if it doesn't, they go into CrashLoopBackOff).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had an eye on this as well. My oberservations basically confirm what @ytjohn said.

  1. The rabbitmq container restarts faster than the st2api service. Therefore we see ECONNREFUSED during the shutdown of st2api.
  2. if the st2api container can not connect to the rabbitmq host for any reason (I prodoced ECONNREFUSED errors as well as EHOSTUNREACH errors) st2api recovers on it's own as soon as the connection works again

No manual intervention was used on any of the test cases described above.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@winem @ytjohn I don't know why I'm facing another scenario unlike yours. I'm feeling this is an improvement done in st2 3.3-dev.

Let me write my environmental details:

  • ST2_VERSION=3.2.0 (I think thats different from yours)
  • MBP 2017
  • OSX 10.14.6 Mojave
  • Docker for Mac 2.3.2.0 with Engine 19.03.8 and Compose 1.25.5

The interesting part is that, I had to lower the Docker for mac VM settings to reproduce the issue. To clarify, I made 1 CPU, 2 GB Memory and 1 GB of Swap and st2api failed on boot.

After st2api fails on boot, it didn't retry connecting unlike yours, because the container itself is dead, as you can see in the below shell output:

docker-compose ps

                  Name                                Command                   State                         Ports
------------------------------------------------------------------------------------------------------------------------------------
stackstorm-compose_mongo_1                 docker-entrypoint.sh mongod      Up               27017/tcp
stackstorm-compose_rabbitmq_1              docker-entrypoint.sh rabbi ...   Up               25672/tcp, 4369/tcp, 5671/tcp, 5672/tcp
stackstorm-compose_redis_1                 docker-entrypoint.sh redis ...   Up               6379/tcp
stackstorm-compose_st2actionrunner_1       /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2api_1                /opt/stackstorm/st2/bin/st ...   Exit 1
stackstorm-compose_st2auth_1               /opt/stackstorm/st2/bin/st ...   Up               9100/tcp
stackstorm-compose_st2client_1             /bin/bash                        Exit 0
stackstorm-compose_st2garbagecollector_1   /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2notifier_1           /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2resultstracker_1     /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2rulesengine_1        /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2scheduler_1          /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2sensorcontainer_1    /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2stream_1             /opt/stackstorm/st2/bin/st ...   Up               9102/tcp
stackstorm-compose_st2timersengine_1       /opt/stackstorm/st2/bin/st ...   Up
stackstorm-compose_st2web_1                /bin/bash -c if [ ${ST2WEB ...   Up (unhealthy)   443/tcp, 127.0.0.1:8000->80/tcp
stackstorm-compose_st2workflowengine_1     /opt/stackstorm/st2/bin/st ...   Up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried some more, including shutting down mongo. With mongo down, st2scheduler will eventually fail after so many retries.

I set all of the containers to restart: on-failure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TunaYagci, I am wondering if this may be something specific to Mac or your docker version. But this is as unqualified as it gets.

I'll do a bunch of tests with the latest changes from @ytjohn later tonight. The most recent ones on Friday and Saturday evening went fine.

stackstorm-compose/docker-compose.yml Outdated Show resolved Hide resolved
@ytjohn
Copy link
Contributor Author

ytjohn commented May 17, 2020

@TunaYagci Pushed a bunch of changes, can you please review and resolve conversations if they're fixed?

Copy link

@TunaYagci TunaYagci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ytjohn thanks for everything!

@TunaYagci
Copy link

@TunaYagci Pushed a bunch of changes, can you please review and resolve conversations if they're fixed?

I think I don't have the permission to resolve conversations, but all looks good to me!

@TunaYagci
Copy link

TunaYagci commented May 18, 2020

@ytjohn I wanted to let you know that current version doesn't support "Storing Secrets", you're getting "MESSAGE: Crypto key not found" error while storing secrets with secret:true field. Details here. It's small detail and we should convert this into a new issue. No need to solve it here (by solving, I mean, even editing the README for some instructions should be enough for users).

I've looked up the stackstorm-ha build but couldn't find a solution there as well. There are 2 open issues about it, here and there.

editing: The single all-in-one stackstorm image actually thought about this:

# Setup symmetric crypto key for datastore
RUN mkdir -p /etc/st2/keys \
    && st2-generate-symmetric-crypto-key --key-path /etc/st2/keys/datastore_key.json \
    && usermod -a -G st2 st2 && chgrp st2 /etc/st2/keys && chmod o-r /etc/st2/keys \
    && chgrp st2 /etc/st2/keys/datastore_key.json && chmod o-r /etc/st2/keys/datastore_key.json \
    && crudini --set /etc/st2/st2.conf keyvalue encryption_key_path /etc/st2/keys/datastore_key.json \
    && crudini --set /etc/st2/st2.conf auth enable True \
    && crudini --set /etc/st2/st2.conf content packs_base_paths /opt/stackstorm/packs.dev

Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome! 👍

I just tried it and amazed how nicely it integrates with the existing Docker images, follows the best practices and avoids conflicts with the K8s. That was our ideal goal of having one set of images for both orchestrators.

Thank you for putting it all together for docker-compose, @ytjohn!

I just left a few minor comments to improve below.

stackstorm-compose/README.md Outdated Show resolved Hide resolved
stackstorm-compose/README.md Show resolved Hide resolved
stackstorm-compose/docker-compose.yml Outdated Show resolved Hide resolved
@arm4b arm4b self-assigned this May 19, 2020
@mickmcgrath13
Copy link

This is exciting to see!

QQ: Is this intended to replace https://github.com/StackStorm/st2-docker?

@winem
Copy link
Contributor

winem commented May 20, 2020

QQ: Is this intended to replace https://github.com/StackStorm/st2-docker?

Yes, it is. :)

@ytjohn
Copy link
Contributor Author

ytjohn commented May 25, 2020

I wasn't able to follow up on this for a few days, but this morning I pushed a few improvements:

  • created a volume for secret keys and generate those on start (I do burn a container for this because it has to happen prior to st2api)
  • better st2web healthchecks. Container will appear unhealthy while downstream services are unavailable. The container will no only restart if a) downstream services are reachable from the OS and b) nginx is not able to reach the downstream services
  • st2client now has ST2CLIENT=1 environment variable
  • st2client has a startup script that will register packs once downstream services are available

@ytjohn ytjohn requested a review from arm4b May 26, 2020 16:36
printf " It's recommended to use \033[1mst2client\033[0m container to work with StackStorm cluster.\n"
# Is K8s environment
if [ -n "$KUBERNETES_PORT" ]; then
printf " \033[1mWarning!\033[0m Do not edit configs, packs or any content inplace as they will be overridden. Modify Helm values.yaml instead!\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added adjustments for the welcome messages so it works properly in both K8s and docker-compose environments.

Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! 👍

Thanks a lot @ytjohn for dedication to bring docker-compose and @winem @TunaYagci @trstruth for review, feedback and testing. Awesome team effort here!


FYI we're including docker-compose deployment in this repo as a temporary home.
Later it will be moved to the https://github.com/stackstorm/st2-docker and replace entire repository.

@arm4b arm4b merged commit 6bd9fc0 into StackStorm:master Jun 3, 2020
@arm4b arm4b changed the title feat: add docker-compose setup Add docker-compose setup Jun 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants