proxy: Dynamic placeholders for URL segments

[lvhuat]

Every time i deploy a new micro service,i have to config the Caddyfile again,and at the same time,my caddy service is in the docker,so i have to run caddy container again with new caddy file.

proxy /v1/users/user1/ www.user1.any.com
proxy /v1/users/user2/ www.user2.any.com
proxy /v1/users/user3/ www.user3.any.com
//....

Do there any way to do proxy like:

/v1/users/:user www.:user.any.com

when the user named user1 send a request,it mean:
proxy /v1/users/user1 to www.user1.any.com

[mholt]

Not at this time, sorry. Someone is welcome to implement this, though. Maybe a syntax like this:

proxy /users/((user1)) www.((user1)).any.com

(Double parentheses because single parentheses might be used in URLs? I dunno.)

[abiosoft]

We have {?user} for query params. Maybe {:user}?

Curly brace is already used in caddy for placeholders so I would say something with curly braces should be used for this as well.

[mholt]

@abiosoft True, that could work. I'm okay with that syntax as long as we don't foresee ever having a {key:value} placeholder (not sure what that would be useful for anyway).

If this is implemented, it would have to be efficient in how it matches the URL and does the replacement.

[sahildua2305]

@mholt I'd like to work on this. I do have prior Go experience. Just new to this project. 🙂

[mholt]

Sounds good to me @sahildua2305 !

Note that I am pondering one potential security issue with this, since typically allowing outside requests to dictate proxy destinations is a bad idea. Go ahead and work on it though. :)

[dbritto-dev]

Maybe something like NGINX syntax:

proxy /users/(\d+) www.$1.any.com

[mholt]

@danilobrinu I think we should be consistent with Caddy's placeholder syntax, which is like {something}.

[dbritto-dev]

@mholt i propose something like {?<placeholder>pattern}

proxy /users/{?<user>[0-9]+} www.{user}.any.com

I think is more secure use regex pattern to match values coming in route

[mholt]

@danilobrinu That's pretty good; although do we need a full regexp? Maybe just capturing that label of the path would be enough, for example /users/{:user}/foo -- much simpler IMO.

[dbritto-dev]

@mholt i think is more secure use pattern to match exact values from route, this concept is using in Django routes.

[mholt]

@danilobrinu Why is regexp more secure?

[dbritto-dev]

@mholt is simple u can use a pattern to eval the value, this is more secure than only use a variable and replace this variable.
Example:

proxy /users/{?<user>[0-9]+} www.{user}.any.com
// Test
proxy /users/5 www.5.any.com (is valid)
proxy /users/clients www.clients.any.con (no valid)
proxy /users/page.otherpage www.page.otherpage.any.com (no valid)

proxy /users/{?user} www.{:user}.any.com
// Test
proxy /users/5 www.5.any.com (is valid)
proxy /users/clients www.clients.any.com (is valid)
proxy /users/page.otherpage www.page.otherpage.any.com (is valid)

[mholt]

@danilobrinu I don't think that's what I mean by secure. Consider SSRF instead: https://github.com/yandex/gixy/blob/master/docs/en/plugins/ssrf.md (for more reading: https://www.owasp.org/index.php/Server_Side_Request_Forgery)

[pyankoff]

I tried to use {label1} for that, but it only works for redirects?

[mholt]

Half of this issue is implemented in v2, the other half certainly can be implemented, just needs someone to work on it.

Anyone mind giving it a go? Submit a PR?

The placeholders part is done. v2 has this ability by using matchers. You can match requests using a regex capture group and then that becomes a label you can apply to the proxy upstream.

The only thing left is to define an upstream that uses a placeholder in its address and replaces it at request-time. So, the unfinished part of this issue is actually the same as #990

[mholt]

Well, I just went ahead and finished it in 1e31be8. 😄

Caddy 2 can now proxy to different remotes by using any valid placeholders (variables) in the dial property of an upstream, for example:

"dial": "{http.request.host}:80"

In other words, you'll define a single upstream and its address is evaluated dynamically at request-time. This has implications for health checks, of course, so be wise when configuring health checks, because in this case a single upstream can actually be many arbitrary number of hosts/ports.

Go try it out!