Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Application router configurations (ingress + service-to-service) #111

Open
flaviostutz opened this issue Oct 25, 2020 · 31 comments

Comments

@flaviostutz
Copy link

flaviostutz commented Oct 25, 2020

What is the problem you're trying to solve
It's very common for compose services to be accessed through an application router (ingress) but today we lack a "spec" for indicating some configurations for ingress services such as protocol (http/https), domain names and paths. We have to define and configure the services using proprietary strategies depending on runtime (traefik labels, caddy labels, Kubernetes ingress, AWS LB rules). Some of those runtime configurations cannot be kept inside docker-compose.yml file itself, what is bad for who is trying to use docker-compose as the solely configuration source.

Describe the solution you'd like
Add the attribute service -> ports -> ingress -> urls whose value is a white separated list of urls in format "[protocol]://[domainname]:[port]/[path]" that will be used by the various container service providers in order to route requests to the service port.

Example:

version: '3.7'
services:
  mytest:
    image: mytest:1.0.0
    ports:
      - target: 5000
        ingress: 
           urls: http://mytest.org https://anothertest.com https://mysite.org/mytest http://easytest.net:2000

According to this example, the following ingress rules will be created:

Port 80: forward by header host="mytest.org" --> one instance of service "mytest":5000
Port 80: forward by header host="anothertest.com" --> redirect to https
Port 80: forward by header host="mysite.org" --> redirect to https
Port 443: forward by header host="anothertest.com" --> one instance of service "mytest":5000
Port 443: forward by header host="mysite.org" + path="/mytest/*" --> one instance of service "mytest":5000
Port 2000: forward by header host="easytest.net" --> one instance of service "mytest":5000

A user would see the following:

When accessing "http://mytest.org", the page on this service will be shown immediately
When accessing "http://anothertest.com", the browser will be redirected to https and the page will be shown
When accessing "http://mysite.org", no webpage is shown
When accessing "http://mysite.org/mytest/page2.html", the browser will be redirected to https and page2.html will be shown
When accessing "http://easytest.net:2000", the page on this service will be shown immediately

Future additions of configurations related to "ingress" may be added here (service -> ports -> ingress). For example, for indicating allowed source IPs, rate limits, minimum TLS protocols, request/response header transformations etc (these are not part of this proposal).

Additional context

@EricHripko
Copy link
Collaborator

While I can totally understand that HTTP services are quite ubiquitous today, this use case seems a tad too specific to be ingrained into the spec. My primary concern here is the ability to reproduce this locally.

You provide an example of proprietary strategies to configure this today, but I cannot pinpoint what this proposal does to address this. While platforms like ECS may have a single way to configure ingresses (that suggest a straightforward mapping from Compose), there is no local equivalent. We'd need to either:

  • Sacrifice local UX (undesirable); or
  • Potentially have each Compose implementation bundle a whole HTTP routing stack just for this functionality (seems like an overkill)

@ndeloof suggestion of using domain name sounds more accessible since Compose is already responsible for tweaking some DNS settings.

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 2, 2020

I like the idea we introduce ingress concept in such an abstract way. By defining an URL we clearly don't just focus on http (even this would probably be 99% usage), actual support for other protocols would be implementation/platform specific.

URLs eare obviously well defines and compact notation, so my +1 on using those. I'd suggest we avoid whitespace-separated lists, and prefer plain list. Can still be a one-liner using ["..", ".."] syntax. Doing so we can adopt the "string or struct" approach use in many other places to support both a compact notation and a more detailled one for complex use-cases which would require some additional attributes (and maybe custom extensions).

services:
  mytest:
    image: mytest:1.0.0
    ports:
      - target: 5000
        ingress: 
         # string syntax
          - http://mytest.org  
         # struct syntaxt with `url` pseudo-attribute. url will be parsed when compose file is loaded
          - url: http://mytest.org  
         # full struct syntaxt
          - procotol: http
            host: mysite.org
            path: /test
            port: 2000
            x-traeffic-somethingmagic: "foo"

This would also allow to introduce support for SSL certificate to be used for HTTPS ingress (could be a separate discussion, just want to ensure we don't shoot into our own feet).

services:
  mytest:
    image: mytest:1.0.0
    ports:
      - target: 5000
        ingress: 
          - url: https//acme.com:5000/foo
            certificate: ./server.crt

there is no local equivalent.

Yes indeed, same applied to secrets, but docker-compose comes with a workaround / local-hack to offer this functionnality, whenever "secrets" are plain text files on developer workstation - which make them not such secrets :P

We could offer something comparable for local ingress development, either with some help from Docker desktop to support some ingress capabilities, or by transparently adding an ingress controller to the app being ran by compose up when such attribute is in use.

@justincormack
Copy link

I think this should be presented as a general layer 7 routing layer, versus traditional ports being layer 3. This is very common now and should be protocol specific as per Nicolas's suggestion. This is how much of routing currently works so it makes sense. Not sure I would call it "ingress" there may be layer 7 routing that is not ingress, eg between services.

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 2, 2020

routing might indeed be a more generic noon for this purpose, and we could then have nested attribute to define if those are actually exposed externaly (aka ingress) or just for service-2-service communication (~ service mesh)

@karolz-ms
Copy link

I wonder what are the pros and cons of having ingress as a top-level entity in the Compose file. For example:

ingress:
  rules:
    - protocol: http
      path: /A
      port: 5000
      service: svc-A
    - protocol: https
      host: subsystem-foo.org
      port: 6000
      targetPort: 32000
      service: svc-foo

In the above example

  • http://localhost:5000/A/b/c would be routed to svc-A using the same port (5000), with path /b/c (the portion of the path that is used for routing the request is truncated)
  • https://subsystem-foo.org:6000/a/b/c would be routed to svc-foo on port 32000, with path /a/b/c (no path is specified, in the ingress configuration, so nothing is removed from the path)

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 2, 2020

@karolz-ms this would be moslty equivalent to an agregate of service-scoped ingress attributes.
With profiles discussion in progress it would anyway make more sense too get this declared at service level, so that ingress is configured only for enabled services.

@karolz-ms
Copy link

@ndeloof good point about profiles. The "bring your own ingress config" approach also meshes well with some types of ingress services like traefik that use pod labels for ingress configuration in K8s world.

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 2, 2020

Yes indeed, an abstract ingress declaration I compose file could be either mapped to labels to embrace traeffik (and other) habits, or used by implementation for a plain imperative service configuration using ingress controller api. Both are feasible and would remain implementation decisions.

@EricHripko
Copy link
Collaborator

By defining an URL we clearly don't just focus on http (even this would probably be 99% usage), actual support for other protocols would be implementation/platform specific.

Good point, didn't think of the fact that URLs could be used for other RPC mechanisms. This makes the proposal much more appealing 👍

We could offer something comparable for local ingress development, either with some help from Docker desktop to support some ingress capabilities, or by transparently adding an ingress controller to the app being ran by compose up when such attribute is in use.

Given that Docker Desktop will likely be focused only on HTTP, injecting an ingress controller into the Compose project sounds more flexible. If the spec enables configuring the ingress controller for local world, it'll even facilitate supporting RPC mechanisms that otherwise may be left behind. Maybe for local, the developer could dedicate one of the Compose services to be used as the router?

I agree with @ndeloof and @justincormack regarding the suggestion to call this routing as opposed to ingress since users may want to declare incoming (host->Compose), outgoing (Compose->external resource) and internal (service-to-service) routes.

@karolz-ms suggestion of defining routes at top-level seems more intuitive to me. @ndeloof if routing is per-service, how would the configuration from multiple services be aggregated?

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 3, 2020

a top-level routing definition will require to link to target services. As sole benefit I can see, it offers a central view on routing rules. But is this what we want to focus on ?

service-level routing would better fit with exising model imho, as each service will "own" it's routing rules, and the compose implementation can just aggregate those to buidl the adequate ingress controller configuration.

@hangyan
Copy link
Collaborator

hangyan commented Nov 4, 2020

Similar to netowrk/volume ...., define a top-level routing and let the services to reference the routing object?

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 4, 2020

@hangyan if we apply same approach as volumes/networks, this would look like this:

routing:
  routeA:
    - protocol: http
      path: /A
      port: 5000
      service: svc-A

services:
  foo:
    ...
    routes:
      - routeA

as a route is "just" a definition, not actual resource allocation, this seems to me over-complicated definition. I prefer a service-scoped definition:

services:
  foo:
    ...
    routes:
    - protocol: http
      path: /A
      port: 5000

A top-level routing element would make sense if we used it to define some actual implementation configuration (comparable to volume/network driver/driver_opts), but AFAICT this only would make sense to define the ingress controller, not individual routes, wouldn't it? Also, I don't think we'd like the ingress controller to be hard-coded like this in a compose file, but considered an architecture implementation choice left under platform responsibility.

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 4, 2020

In addition to my previous comment, main reason volumes and networks have a dedicated section, is that those resources are expected to be used by 1_or_more services, while a route will only target a single service (maybe replicated by multiple containers) so a top-level element does not make much sense.

If the request for such a routing element is for compose file readability, then the tooling could offer commands to collect active routes and dump a routing map:
`$ docker compose routes --format yaml
routes:

  • protocol: http
    path: /A
    port: 5000
    service: svc-A
  • protocol: https
    host: subsystem-foo.org
    port: 6000
    targetPort: 32000
    service: svc-foo
    `
    ... but this is out of the scope of this spec :)

@hangyan
Copy link
Collaborator

hangyan commented Nov 4, 2020

In addition to my previous comment, main reason volumes and networks have a dedicated section, is that those resources are expected to be used by 1_or_more services, while a route will only target a single service (maybe replicated by multiple containers) so a top-level element does not make much sense.

If the request for such a routing element is for compose file readability, then the tooling could offer commands to collect active routes and dump a routing map:
`$ docker compose routes --format yaml
routes:

* protocol: http
  path: /A
  port: 5000
  service: svc-A

* protocol: https
  host: subsystem-foo.org
  port: 6000
  targetPort: 32000
  service: svc-foo
  `
  ... but this is out of the scope of this spec :)

This would make more sense. I guess a complete overview of all routes will be needed somehow.

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 4, 2020

@hangyan yes indeed, just like port command can be used to obtain a public port binding, tool can make our life simpler with adequate commands :)

@EricHripko
Copy link
Collaborator

it offers a central view on routing rules

I guess it's quite a nice benefit to have, but I appreciate that this approach is rather inconsistent when compared to how other objects are defined. As you pointed out, this gap in UX could also be filled with appropriate tooling behaviour.

as each service will "own" it's routing rules

One last question: is the idea that internal (service-to-service) routes could be one way?

@flaviostutz
Copy link
Author

routing might indeed be a more generic noon for this purpose, and we could then have nested attribute to define if those are actually exposed externaly (aka ingress) or just for service-2-service communication (~ service mesh)

I totally agree on keeping this more generic, but to be honest I only had the requirements of configuring more advanced routing parameters on "ingress" routings. For service-to-service routing the Swarm/K8S mesh simplicity always served me very well in production scenarios.

Using "routes" in the spec is better because if we need to add more service-to-service configurations in future we will be more prepared. There are lots of more advanced service-to-service cases with Envoy/Istio like rate-limiting, service certification, circuit breaking, metrics and health checks that could be specified in compose files in future too! (but I don't have enough production experience on those requirements yet...).

@flaviostutz
Copy link
Author

services:
  mytest:
    image: mytest:1.0.0
    ports:
      - target: 5000
        ingress: 
         # string syntax
          - http://mytest.org  
         # struct syntaxt with `url` pseudo-attribute. url will be parsed when compose file is loaded
          - url: http://mytest.org  
         # full struct syntaxt
          - procotol: http
            host: mysite.org
            path: /test
            port: 2000
            x-traeffic-somethingmagic: "foo"

I really liked this construction.

@EricHripko
Copy link
Collaborator

EricHripko commented Nov 12, 2020

I understand why we'd want to start with ingress, but we certainly shouldn't stop there. At the very least, it seems like we should to pair the configuration of incoming routes with the concept of outgoing routes.

On the flip side, this could be crossing into the area of service discovery - which (arguably) is better left to a specialised service discovery container 🙈

@flaviostutz
Copy link
Author

On the flip side, this could be crossing into the area of service discovery - which (arguably) is better left to a specialised service discovery container 🙈

AppMesh services are compatible with AWS Service Registries so that Envoy containers use this information for routing services. ;)

@flaviostutz flaviostutz changed the title Proposal: Application router configurations (ingress) Proposal: Application router configurations (ingress + service-to-service) Nov 13, 2020
@mikesir87
Copy link
Contributor

Hello all! Finally chiming in! 😄

I was thinking about the use case in which multiple services might be using the same route definition (like canary deploy or A/B testing). In those situations, the route config might be mostly the same, minus details like weight or some other routing strategies. To me, those extra details feel like extension areas, as that will most likely be provider specific.

So, does it make sense to have that same route config defined once in a top-level config and then referenced with additional weights? Or does it make sense to just inline it all as part of a per-service config? Honestly, I feel I could go either way. Figured I'd at least bring up the use case.

@ndeloof
Copy link
Collaborator

ndeloof commented Nov 19, 2020

@mikesir87 imho canary or A/B deployement should not be expressed within a compose file, such deployment technique should better be used by Compose implementation on compose up for an existing app with some configuration changes/image update.
service.deploy configuration has some definition for update strategy (https://docs.docker.com/compose/compose-file/#update_config) that could host parameters for this.

@flaviostutz
Copy link
Author

flaviostutz commented Dec 22, 2020

Using compose "deploy" configuration is surely the way for automatic rollout scenarios. And this is the most common/simpler way to update a container.

But in production in various cases we really liked the idea to create two compose services (for the same "service"), but each pointing to a different container configuration or different ENV contents because the errors that could arrive from this deploy could not be detected with a container "health" check. We really wanted to point just 5% of the users to the new service definition, wait for the support to give a thumbs up, and then open to 30% of the users and so on. Keeping all those definitions on docker-compose.yml is great for clarity too, so we use a custom feature flag attribute in frontend ENV container that points the UI to different backend addresses. When we have dozens of thousand of concurrent users, 0.1% of an expurious error is a great headache for our users.

AWS AppMesh uses the concept of VirtualRoute for this kind of A/B testing among different versions of a service, but there is no such concept in compose, is there?

@flaviostutz
Copy link
Author

flaviostutz commented Dec 22, 2020

@ndeloof extending your proposal, we could add a "weight" param that could be used to disambiguate routes and handle some A/B load scenarios. For example:

services:
  foo1:
    ...
    routes:
    - protocol: http
      host: foo.example.org
      path: /A
      port: 5000
      weight: 2

  foo2:
    ...
    routes:
    - protocol: http
      host: foo.example.org
      path: /A
      port: 4000
      weight: 1      

In this case, as "host" and "path" are the same for services foo1 and foo2, the route provider (ECS/AppMesh, K8S, ECI etc) will use "weight" to route requests coming from http://foo.example.org/A/*, sending 2:1 requests to foo1:foo2.

All "deploy" features are kept the same when updating the same service over time but we can leverage more advanced scenarios by using the "weight" attribute on routing.

What do you think?

@pchico83
Copy link
Contributor

pchico83 commented Apr 21, 2021

Exposing services using ingress/routes is a very common scenario in production environments today. I think having support for them is a need to make Compose more powerful in production environments.
We are implementing a Kubernetes backend for the Compose specification called Okteto Stacks:
https://okteto.com/docs/reference/stacks

Kubernetes has standardized the use of ingresses as a top-level resource. As a reference, we are using the following notation, but it would be nice to have something similar in the Compose spec:

endpoints:
  endpointA:
    - path: /
       port: 80
       service: frontend
    - path: /api
       port: 8080
       service: API

services:
  frontend:
      image: xxxx
      ports:
         - 80
  api:
      image: xxxx
      ports:
         - 8080

@ndeloof
Copy link
Collaborator

ndeloof commented Apr 21, 2021

@pchico83 quick question
with your notation, how do you know which port enpoint1/api:8080 should be routed to, if service API has more than one port exposed?

@pchico83
Copy link
Contributor

@ndeloof the port in the endpoint definition matches the port in the service definition. If the ports don't match, we throw an error

@ndeloof
Copy link
Collaborator

ndeloof commented Apr 21, 2021

ok, so we can't express port translation using this syntax, i.e. if I want publicly published port to be 443, but my service expose 8080 and I expect the ingress layer to manage SSL termination for HTTPS.

@pchico83
Copy link
Contributor

Normally you already have an ingress controller in your cluster listening in a public port, and the developer doesn't have control over it, it is configured by the ops team. That is why the proposed syntax is enough to match the external port to the container port. You could have several ingress controllers in the same cluster, in this case, you can specify the ingress which should manage this route using annotations. This is why we also have the extended notation:

endpoints:
  annotations:
    kubernetes.io/ingress.class: nginx
  rules:
    endpointA:
      - path: /
         port: 80
         service: frontend
      - path: /api
         port: 8080
         service: API

For things like SSL termination, you can also use annotations. For example, for nginx:

endpoints:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"    
  rules:
    endpointA:
      - path: /
         port: 80
         service: frontend
      - path: /api
         port: 8080
         service: API

@ndeloof
Copy link
Collaborator

ndeloof commented Apr 23, 2021

The annotation you describe is exactly what we should hide behind adequate abstraction in the compose spec. i.e. by definition of my compose application, I'd like I can express "ssl-redirect", which might be implemented on Kubernetes using nginx or other annotations as this is how this infrastructure handles configuration, but on AWS I would tweak the load balancer's listener rules, and running locally with Traefik (keep in mind we need a local developer experience too) this would translate into yet another mechanism.

@pchico83
Copy link
Contributor

@ndeloof I agree. I explained how kubernetes works to provide more context, but all those annotations (or at least the most common ones) should be defined as fields in the compose spec. And then have a generic field to specify specific options for a particular compose spec backend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants