Dynamic proxy

[abiosoft]

Dynamic Proxy.

Supports Etcd, Consul and ZooKeeper using github.com/docker/libkv.

Demonstration Video: https://youtu.be/I0Kax0F1XWM

Examples

Etcd

proxy / etcd://127.0.0.1:2379/caddyserver.com

Consul

proxy / consul://127.0.0.1:8500/caddyserver.com

[InAnimaTe]

So after watching your video, this definitely does seem to be shaping up! I'm really glad you've taken the plunge into getting support for dynamic backends at least started!

Based on some experience and seeing what others provide, I have some input that may not exactly apply to this specific PR but can at least spawn discussion and thinking for future additions to this end goal.

1. It would be really cool if Caddy could provide the ability to consume proxy based configuration via container runtime labels or variables. You could consume a label/variable like caddy.port=2015 telling the proxy to utilize the public port entry in etcd mapped to 2015. You could even define http or https backend connectivity i.e caddy.protocol=https or weight caddy.weight=7 for this particular backend.

Note that this depends on what registrator dumps in your K/V store based on metadata it finds from your docker socket! I know for a fact a lot of things are possible with environment variables.

1. Not sure much about the etcd entry with the appended /caddy-2015 but to me, the only data in the actual front-end proxy configuration (relating to the container infra) should be the name of the service. In example, if I have a service bugzilla, I should only really need to reference /bugzilla in etcd. That's, for all intensive purposes, the key for which back-end containers register themselves with a front-end.
2. Global definition of backend store locations is definitely something I'd like to see. A lot of the time, docker infrastructure has multiple redundant proxies all making backend connections to a multitude of container hosts. For an entire deployment like this, the proxies would essentially utilize the same K/V store and service discovery system for the entire cluster. Maybe utilize a variable or something for this i.e. proxy / main_etcd_store://bugzilla with something like dynamic etcd etcd://192.168.1.19:4444 setup globally. Keep reading for more ideas in this realm.
3. Re-iteration on the last point, why don't you allow for sub-service declarations under a specific from and even specify the destination K/V store right in the stanza i.e.

proxy / {
  bugzilla {
    destination: etcd
    max_fails 7
    health_check /bugs
  }
  jira {
    fail_timeout 30
    destination: consul
  }
}

So this says look in the globally defined etcd K/V for bugzilla and health check any backend container endpoints @ /bugs to ensure a proper response is returned. Likewise for jira, just a tad different.

Finally, I haven't used Caddy extensively yet but it seems it doesn't yet support Name Based Proxying? This feature is a must for Caddy to compete long-term as a docker load-balancer and reverse proxy.
Something like this would be very exciting:

proxy / {
  bitbucket {
    name: git.example.com
    destination: zookeeper

I'm subscribed to this thread so post replies and lets work together to figure out the best way to do this!

[faddat]

....and now I am subscribed as well. Is there an IRC or messaging client that the three of us have in common?

[mholt]

@faddat Join us on gitter for dev chat: https://gitter.im/mholt/caddy

[mholt]

After Caddy 0.9 is released (not next week, but the release after -- not sure when that is yet), Caddy will be in a much better position to have this merged. Not quite sure of the details yet but I'll work closely with @abiosoft at that point.

[thalesfsp]

@mholt What will be the performance impact of this PR? Benchmark?

[mholt]

@thalesfsp No clue, honestly - @abiosoft might know better. I would imagine it would only have minimal, if any, impact, since as I understand it, the updating of proxies is done in the background.

[thalesfsp]

@mholt Awesome! I hope and will wait for @abiosoft come with some benchmark :)

[mholt]

You can always perform your own, too.

[abiosoft]

I expect it to have little to no impact

[mholt]

It's amazing this PR can still be merged 😄 That will not be the case for long though.

The 0.9 changes are in a branch and will soon be in master. I don't yet have the proxy middleware brought over to that branch but I will soon, and then these changes can possibly go on top.

[pwFoo]

I use caddy as reverse proxy with docker-gen for my docker host and it works great. But etcd support sounds like a better solution. Can't wait to see this feature and the howto to use it ;)

[thalesfsp]

@mholt Whats the plan to when merge this PR?

[mholt]

I think it depends on @abiosoft's schedule.

[stp-ip]

Would this lay the groundwork for caddy being able to supply an ingress service to kubernetes? As far as I see it... an ingress service is basically a dynamic proxy using the k8s apiserver instead of general key-value stores.

[thalesfsp]

@abiosoft @mholt What's the ETA to be merged?

[abiosoft]

No ETAs, sorry. I plan to resume work on this after 0.9 :)

There is a very good chance it will make it in before 1.0 and could even be earlier depending on Caddy's priorities.

[hemsleyk]

A+

Would really help with Rancher / Kubernetes environments.

[pwFoo]

Yes, would use it with rancher...

[thalesfsp]

@mholt @abiosoft 0.9 released ;)

[abiosoft]

Okay, good news :)

Work is about to resume on this. However, I wanna make this a pluggable feature so proxy will only be making use of it, rather than having it built into proxy. This will make it easy to bring the feature to any other directive or plugin that needs it.

Still, no ETA 😄 but I will keep this thread updated.

[InAnimaTe]

@abiosoft great idea! much better and extensible way to implement.

Have you had anymore thoughts on implementation details? I'm wondering if it's not worth getting this working the original intended way (when you created this PR), and then move on to other ways later (since this PR is quite large both in conversation and commits).

Additionally, I've had some other ideas about how this could work based on other projects that have come along (i.e. Fabio, Traefik, etc..) and attempted to provide LE functionality with dynamic backends.

I would love to talk through some of this directly with you guys @abiosoft @faddat @stp-ip and others..
Seems to be a #caddy on freenode that I just joined. Ping me and we can chat (notifications go to my phone). Or maybe just create a new forum thread; whatever's easier.

[abiosoft]

@InAnimaTe I'm rarely on irc :( What about Gopher's slack ?

[InAnimaTe]

@abiosoft That works! I just joined. Go ahead and make a channel and invite me plz ;)

[hazcod]

Is this PR still going? Also, will this support native docker swarm?

[faddat]

If this supported swarm mode, wow. Also, I think that it isn't too tough to implement. Right now I haven't time, but it's possible that myself or someone on my team will do it. ... Because Caddy is awesome.

…

On Dec 7, 2016 9:25 PM, "HazCod" ***@***.***> wrote: Is this PR still going? Also, will this support native docker swarm? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#564 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGz6iXhD7LaOTu2KYRp7wEQR95ePV4uaks5rFsHggaJpZM4HRNBY> .

[wendigo]

Maybe it is time to write it once again? :)

[hazcod]

I've put some thought in this and isn't it cleaner to do this via docker-gen ?
Altough I am unsure how it works for swarm/multiple hosts.

[mholt]

I think I'm going to close this PR -- it's almost a year old, but I hope it will soon make it to Caddy -- if not into core, then maybe it would do well as a plugin? Thanks for all the work you've put into it, @abiosoft, I hope we can use it someday. :)