Update example ctmpl to select services

[tgross]

In TritonDataCenter/containerpilot#80 @donniev noted that our example configuration for Nginx grabs all the services. Because we want to mostly use the Containerbuddy examples as integration tests, I've recommended we move that discussion here.

@donniev has provided a proof-of-concept for generating the list of services we want into the template file: https://gist.github.com/donniev/60cd7c97522c538f6d11 We could use something like this to create the https://github.com/tgross/triton-nginx/blob/master/example/nginx.ctmpl template file prior to injecting it into the Nginx environment.

In adapting this script, we should be careful not to get ourselves into a situation where we can't add a new service to Nginx without a redeploy -- we should be able to add a service with the appropriate tag and have Nginx pick it up automatically with help from Containerbuddy.

[donniev]

Currently it creates the base template(default.ctmpl) before it starts nginx. Nginx wll pick up when you start a service that was in the original docker-compose.yml when it is started but would not pick up any changes if you added to that file after nginx was deployed. It would use the tags of the service to generate the appropriate service blocks but if you had a new service that was not in the original list of services it would never get that far. There would have to be some mechanism that a) generated new base template default.ctmp b) copied it to consul k-v pair c) copied that to files system on nginx where it looks for that template. So basically execute the commands that the start.sh script does before it launches nginx

[donniev]

https://gist.github.com/donniev/728d423bb754eff0012d I have added this script to my menu options. I deployed nginx without the "hexo" service in either that upstreams comment on in docker-compose. I then added the service in docker-compose and added it to the upstreams comment and then ran reloadbase.sh and started hexo. Shelled into nginx and looked at config file and it had picked hexo up.

I did modify the reload-nginx.sh to always pull latest from consul instead of checking for $VIRTUALHOST so this seems to work. Of course it requires a manual step to run the reload but I don't think that is a big deal because you manually edited the docker-compose to add a new service so running that reload script after you are done can just be part of the workflow.

[tgross]

There would have to be some mechanism that a) generated new base template default.ctmp b) copied it to consul k-v pair c) copied that to files system on nginx where it looks for that template.

I think you're getting at the crux of the problem here, which is that adding a new upstream to Nginx requires two things:

• a new Containerbuddy configuration for the Nginx container so that it knows to watch for that service
• a new virtualhost configuration

(Alternately, we could just redeploy a new Nginx container of course but I imagine that's what we're trying to avoid here.)

I did modify the reload-nginx.sh to always pull latest from consul instead of checking for $VIRTUALHOST so this seems to work

It's $NGINX_CONF in this repo but same thing. That was something we had added to give people an option for injecting the template as part of their docker-compose-based deployments without the need for a script to send the Nginx config into Consul. In practice the Nginx config is going to be multi-line and so injecting it into the docker-compose script involves running a script anyways so I think I agree with you that we don't gain much by that.

Of course it requires a manual step to run the reload

What it sounds like we want to do is to have both the Containerbuddy configuration and the Nginx configuration in Consul, and then try to get Nginx to poll the key somehow. Unless we do something like TritonDataCenter/containerpilot#87 maybe this is something we could add to the health handler in the meantime. (Hey @misterbisson note yet another thing we're trying to hack into the health handler as evidence that we probably do want the periodic tasks feature!) So what we could do is make a new script for the health handler that:

• queries Consul for keys for Containerbuddy and Nginx configs
• fires off a SIGHUP to Containerbuddy if that config changes, or fire off a nginx -s reload to Nginx if that config changes
• executes the curl health check and returns the exit code to Containerbuddy

The danger, as alluded to in TritonDataCenter/containerpilot#87, is that if the query on Consul runs long then the heartbeat TTL might expire and the node marked unhealthy. So we'll need to accommodate that.

---

As an aside, I'm going to argue that automatically generating a virtualhost configuration from the list of services only works if the location blocks you want for the services are going to be identical. In my experience, this isn't the case: ServiceA wants to be cached on disk, ServiceB needs CORS headers added, ServiceC wants round-robin load balancing, ServiceD wants least conns, etc. I'm not opposed to doing this for the purposes of demonstration but we should keep in mind that's not going to be the general case.

[misterbisson]

@tgross' concerns about the difficulty of building a generic Nginx image seem to have been born out as we've attempted to integrate this with other projects. I finally gave up hope for it recently and we overhauled this image and our usage example to reflect the following:

1. The best use of this image is as a base for your own image. See that at work in https://github.com/autopilotpattern/wordpress and other Autopilot Pattern implementations
2. Trivial applications, like the example app included in this repo, can be proxied generically

We're maintaining the image for those two usage scenarios. I believe the need expressed in this ticket is best addressed by using this Nginx as a base and building up from there.