New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ALB: Request for input #1373

Closed
mattmanning opened this Issue Nov 4, 2016 · 40 comments

Comments

Projects
@mattmanning
Contributor

mattmanning commented Nov 4, 2016

A few months ago, AWS announced the Application Load Balancer (ALB). ALB is a new load balancer service that supports exciting features such as content-based routing, HTTP/2, and sticky sessions.

Convox currently only works with ELB, and some of Convox's high-availability features are based on the way ELB works with ECS, such as distributing containers with open ports evenly across the cluster (and therefore across Availability Zones). Because of this, supporting ALB will require some deep changes and careful design.

The purpose of this issue is to solicit feedback from our user community. In your comments, please try to answer the following questions:

  • What ALB features are you most interested in?
  • What problem(s) would ALB solve for you that ELB currently does not?
  • Do you have design/implementation suggestions?

After collecting feedback we will propose a design, break down the changes needed, and provide our best estimate for an implementation timeline.

Some comments from a previous experimental ALB pull request are shared below:


@dlanger commented on Sep 30

For us, the most attractive feature of ALBs is that they enable running more than one copy of a container on a single instance. This greatly simplifies capacity planning and makes it easier to think of an ECS cluster as a pool of compute resources, and means I no longer have to do separate tracking to ensure that max(number of copies of a single container I'd like to run) <= number of instances.


@ddollar commented on Sep 30

So one thing to note is that running those processes on multiple instances is a lot more fault-tolerant. If you stack multiple processes on a single instance then it's a lot easier to take it down with the loss of a single instance.


@mwarkentin commented on Sep 30

@ddollar @dlanger was more talking about if we wanted to scale up to 6 processes, that we could do that at 2x3 instances or whatever, not running a whole bunch on 1 instance. Like he mentioned, it simplifies our resource calculations on one dimension.. another nice benefits is that we could see some cost savings on things like Datadog and ThreatStack where things are billed on a per-instance basis.


@scottmessinger commented 24 days ago

Hi! @ddollar asked for me to leave a comment in this PR with my use case for ALB. We're deploying an Elixir app that uses websockets. From what I understand, ELB doesn't support sticky sessions with websockets. Also, there seem to be some complexity in simply setting up websockets on ELB from the scattering of blog posts I've read. ALB is attractive because, from what I understand, it would just work. (Btw, I could be off here -- I'm coming from Heroku so I haven't had to get into the weeds with AWS deployments)


@idyll commented 24 days ago

That's crazy @scottmessinger -- we are in the exact same position. Some Elixir (other stuff still in Ruby) and we really need the ALB support to remove the need for ProxyProtocol.

We are actively porting more and more ruby to elixir so the pain of missing ALB support is getting worse for us.

@mattmanning

This comment has been minimized.

Show comment
Hide comment
Contributor

mattmanning commented Nov 4, 2016

@nzoschke

This comment has been minimized.

Show comment
Hide comment
@nzoschke

nzoschke Nov 5, 2016

Contributor

Problem Statement

Cost for microservice apps can be expensive. ~$100s / month.

Currently use 1 ELB per service type. $20 x number of services.

Instance count for scaling services can also be inflated. ELB requires 1 container on 1 instance. Means need min 6 instances for 5 web processes.

Goal

Reduce cost of microservice apps to ~$20/month.

Use 1 ALB per app. ~$16 total.

Challenges

ALB supports HTTP, HTTPS, HTTP/2 and Websockets. But no TCP/TLS support.

ELB supports TCP/TLS and Proxy Protocol. These are used widely for HTTP/2 and Websockets.

ALB routing rules are path-based. New concept to Convox, and Docker Compose.

Zero-downtime migration is hard. Would need both running at once.

Design

Add convox-specific path label to docker-compose.yml:

api:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80

Routing path order is configured by process definition order (not alphabetical):

site:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80
docs:
  labels:
    - convox.router.path=/docs/*
  ports:
    - 80:80

And by dependency order as defined by links:

api:
  labels:
    - convox.router.path=/api/*
  ports:
    - 80:80
  links:
    - site
site:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80

HTTPS / HTTP/2 is configured with a label:

api:
  labels:
    - convox.router.path=/*
    - convox.port.443.protocol=https
  ports:
    - 80:80
    - 443:80
api:
  labels:
    - convox.router.path=/*
    - convox.port.443.protocol=http/2
  ports:
    - 80:80
    - 443:80

End-To-End encryption provisions an ELB:

web:
  labels:
    - convox.port.443.secure=true
    - convox.port.443.protocol=https
  ports:
    - 443:5001

TCP / Proxy Protocol provisions an ELB:

web:
  labels:
    - convox.port.443.protocol=tls
    - convox.port.443.proxy=true
  ports:
    - 443:5000

An app can provision an ALB and many ELBs based on docker-compose.yml. Invalid combinations are detected by convox start, convox doctor and convox deploy.

convox/proxy does path-based routing to simulate ALB locally.

Transition of a service from ELB to ALB incurs downtime. Create a new app and use DNS to mitigate.

Resources

ALB Docs
David Dollar's Provisional ALB Support Pull Request
Noah Zoschke's ALB Review
Load Balancers Doc
Balancers Guide

Contributor

nzoschke commented Nov 5, 2016

Problem Statement

Cost for microservice apps can be expensive. ~$100s / month.

Currently use 1 ELB per service type. $20 x number of services.

Instance count for scaling services can also be inflated. ELB requires 1 container on 1 instance. Means need min 6 instances for 5 web processes.

Goal

Reduce cost of microservice apps to ~$20/month.

Use 1 ALB per app. ~$16 total.

Challenges

ALB supports HTTP, HTTPS, HTTP/2 and Websockets. But no TCP/TLS support.

ELB supports TCP/TLS and Proxy Protocol. These are used widely for HTTP/2 and Websockets.

ALB routing rules are path-based. New concept to Convox, and Docker Compose.

Zero-downtime migration is hard. Would need both running at once.

Design

Add convox-specific path label to docker-compose.yml:

api:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80

Routing path order is configured by process definition order (not alphabetical):

site:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80
docs:
  labels:
    - convox.router.path=/docs/*
  ports:
    - 80:80

And by dependency order as defined by links:

api:
  labels:
    - convox.router.path=/api/*
  ports:
    - 80:80
  links:
    - site
site:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80

HTTPS / HTTP/2 is configured with a label:

api:
  labels:
    - convox.router.path=/*
    - convox.port.443.protocol=https
  ports:
    - 80:80
    - 443:80
api:
  labels:
    - convox.router.path=/*
    - convox.port.443.protocol=http/2
  ports:
    - 80:80
    - 443:80

End-To-End encryption provisions an ELB:

web:
  labels:
    - convox.port.443.secure=true
    - convox.port.443.protocol=https
  ports:
    - 443:5001

TCP / Proxy Protocol provisions an ELB:

web:
  labels:
    - convox.port.443.protocol=tls
    - convox.port.443.proxy=true
  ports:
    - 443:5000

An app can provision an ALB and many ELBs based on docker-compose.yml. Invalid combinations are detected by convox start, convox doctor and convox deploy.

convox/proxy does path-based routing to simulate ALB locally.

Transition of a service from ELB to ALB incurs downtime. Create a new app and use DNS to mitigate.

Resources

ALB Docs
David Dollar's Provisional ALB Support Pull Request
Noah Zoschke's ALB Review
Load Balancers Doc
Balancers Guide

@nzoschke

This comment has been minimized.

Show comment
Hide comment
@nzoschke
Contributor

nzoschke commented Nov 5, 2016

@jhspaybar

This comment has been minimized.

Show comment
Hide comment
@jhspaybar

jhspaybar Nov 5, 2016

I saw some concern expressed that the service wouldn't spread tasks without using the ELB. According to this page http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html It will spread tasks across availability zones if possible.

When the service scheduler launches new tasks, it attempts to balance them across the Availability Zones in your cluster with the following logic:

Determine which of the container instances in your cluster can support your service's task definition (for example, they have the required CPU, memory, ports, and container instance attributes).

Sort the valid container instances by the fewest number of running tasks for this service in the same Availability Zone as the instance. For example, if zone A has one running service task and zones B and C each have zero, valid container instances in either zone B or C are considered optimal for placement.

Place the new service task on a valid container instance in an optimal Availability Zone (based on the previous steps), favoring container instances with the fewest number of running tasks for this service.

jhspaybar commented Nov 5, 2016

I saw some concern expressed that the service wouldn't spread tasks without using the ELB. According to this page http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html It will spread tasks across availability zones if possible.

When the service scheduler launches new tasks, it attempts to balance them across the Availability Zones in your cluster with the following logic:

Determine which of the container instances in your cluster can support your service's task definition (for example, they have the required CPU, memory, ports, and container instance attributes).

Sort the valid container instances by the fewest number of running tasks for this service in the same Availability Zone as the instance. For example, if zone A has one running service task and zones B and C each have zero, valid container instances in either zone B or C are considered optimal for placement.

Place the new service task on a valid container instance in an optimal Availability Zone (based on the previous steps), favoring container instances with the fewest number of running tasks for this service.
@dlanger

This comment has been minimized.

Show comment
Hide comment
@dlanger

dlanger Nov 5, 2016

+1 to the interface of defining the path in docker-compose.yml - and thanks for the shout-out to comment on this.

"Transition of a service from ELB to ALB incurs downtime. Create a new app and use DNS to mitigate" sounds reasonable.

Clarifications:

  • One of our use-cases is that within a single app, we'll have an external-facing service and an internal-only service. This sounds like it'll still be possible ("An app can provision an ALB and many ELBs based on docker-compose.yml."), but could you provide an example docker-compose.yml with this?
  • Could you expand on the "And by dependency order as defined by links" example? I don't follow what you're illustrating.
  • Any thoughts about how this would play with auto-scaling?

dlanger commented Nov 5, 2016

+1 to the interface of defining the path in docker-compose.yml - and thanks for the shout-out to comment on this.

"Transition of a service from ELB to ALB incurs downtime. Create a new app and use DNS to mitigate" sounds reasonable.

Clarifications:

  • One of our use-cases is that within a single app, we'll have an external-facing service and an internal-only service. This sounds like it'll still be possible ("An app can provision an ALB and many ELBs based on docker-compose.yml."), but could you provide an example docker-compose.yml with this?
  • Could you expand on the "And by dependency order as defined by links" example? I don't follow what you're illustrating.
  • Any thoughts about how this would play with auto-scaling?
@idyll

This comment has been minimized.

Show comment
Hide comment
@idyll

idyll Nov 7, 2016

"Transition of a service from ELB to ALB incurs downtime. Create a new app and use DNS to mitigate"
I agree with this as well. I think it is totally reasonable that you need to create a new app to switch between the two.

I think adding the config to docker-compose.yml with labels makes sense.

At some point it would be great if compose understood path and protocol so that labels weren't required. (but that's a Docker thing, not Convox...)

idyll commented Nov 7, 2016

"Transition of a service from ELB to ALB incurs downtime. Create a new app and use DNS to mitigate"
I agree with this as well. I think it is totally reasonable that you need to create a new app to switch between the two.

I think adding the config to docker-compose.yml with labels makes sense.

At some point it would be great if compose understood path and protocol so that labels weren't required. (but that's a Docker thing, not Convox...)

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Nov 7, 2016

Contributor

I think it would be good to keep in mind (and plan for) the different routing options that will be available. It was confirmed after the launch of ALBs that you would eventually be able to route on other keys, like subdomains (instead of paths) [1].

For the docker-compose, that seems like it could be pretty straightforward:

api:
  labels:
    - convox.router.subdomain=api
  ports:
    - 80:80
  links:
    - site
site:
  labels:
    - convox.router.subdomain=www
  ports:
    - 80:80

Probably worth keeping in mind from an implementation perspective as well.

Contributor

mwarkentin commented Nov 7, 2016

I think it would be good to keep in mind (and plan for) the different routing options that will be available. It was confirmed after the launch of ALBs that you would eventually be able to route on other keys, like subdomains (instead of paths) [1].

For the docker-compose, that seems like it could be pretty straightforward:

api:
  labels:
    - convox.router.subdomain=api
  ports:
    - 80:80
  links:
    - site
site:
  labels:
    - convox.router.subdomain=www
  ports:
    - 80:80

Probably worth keeping in mind from an implementation perspective as well.

@ddollar

This comment has been minimized.

Show comment
Hide comment
@ddollar

ddollar Nov 8, 2016

Member

@jhspaybar It is my understanding that ALB will try to balance things when possible but would still allow two Tasks of the same Service to colocate on one instance if that was the only place that had room for them? It seems like this breaks some possible guarantees around HA.

Member

ddollar commented Nov 8, 2016

@jhspaybar It is my understanding that ALB will try to balance things when possible but would still allow two Tasks of the same Service to colocate on one instance if that was the only place that had room for them? It seems like this breaks some possible guarantees around HA.

@dlanger

This comment has been minimized.

Show comment
Hide comment
@dlanger

dlanger Nov 8, 2016

@ddollar My read is similar to @jhspaybar: ALB will try and avoid colocating multiple copies of the same container on one instance (following various rules for picking), but will do so if it needs to.

Incidentally, this is the feature I'm most looking forwards to in ALB (my use-case is ~10 of the same container spread over 5-8 hosts).

dlanger commented Nov 8, 2016

@ddollar My read is similar to @jhspaybar: ALB will try and avoid colocating multiple copies of the same container on one instance (following various rules for picking), but will do so if it needs to.

Incidentally, this is the feature I'm most looking forwards to in ALB (my use-case is ~10 of the same container spread over 5-8 hosts).

@nzoschke

This comment has been minimized.

Show comment
Hide comment
@nzoschke

nzoschke Nov 9, 2016

Contributor

@dlanger

External-facing service and an internal-only service is something like:

``yaml
api:
ports:
- 8000
links:
- site
site:
labels:
- convox.router.path=/*
ports:
- 80:80


The single `8000` indicates an internal ALB. This app would have 2 ALBs, one internal for all the non-exposed services and one external for all the exposed services. 

For dependency order, in my example I have an `/api/*` and `/*` path:

```yaml
api:
  labels:
    - convox.router.path=/api/*
  ports:
    - 80:80
  links:
    - site
site:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80

You have to configure the ALB to know to try to match /api/* first and then the catchall /* next. If you defined these paths in the opposite order, the catchall would always win.

We have a few ways to specify order... It could be alphabetical, i.e. api comes before site. It could be sequential, i.e. api is defined before site. Or it could be explicit with links, i.e. api depends on site.

This is the design I'm least sure about. All options are kind of magical ways to configure routing behavior.

Another strategy would be to have a - convox.router.priority=n label, and configure them by the order there.

Contributor

nzoschke commented Nov 9, 2016

@dlanger

External-facing service and an internal-only service is something like:

``yaml
api:
ports:
- 8000
links:
- site
site:
labels:
- convox.router.path=/*
ports:
- 80:80


The single `8000` indicates an internal ALB. This app would have 2 ALBs, one internal for all the non-exposed services and one external for all the exposed services. 

For dependency order, in my example I have an `/api/*` and `/*` path:

```yaml
api:
  labels:
    - convox.router.path=/api/*
  ports:
    - 80:80
  links:
    - site
site:
  labels:
    - convox.router.path=/*
  ports:
    - 80:80

You have to configure the ALB to know to try to match /api/* first and then the catchall /* next. If you defined these paths in the opposite order, the catchall would always win.

We have a few ways to specify order... It could be alphabetical, i.e. api comes before site. It could be sequential, i.e. api is defined before site. Or it could be explicit with links, i.e. api depends on site.

This is the design I'm least sure about. All options are kind of magical ways to configure routing behavior.

Another strategy would be to have a - convox.router.priority=n label, and configure them by the order there.

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Nov 9, 2016

Contributor

Another option might be just to go with most-specific first? Eg. count the number of slashes, more wins? Would a route that can't ever be matched ever make sense?

Contributor

mwarkentin commented Nov 9, 2016

Another option might be just to go with most-specific first? Eg. count the number of slashes, more wins? Would a route that can't ever be matched ever make sense?

@dlanger

This comment has been minimized.

Show comment
Hide comment
@dlanger

dlanger Nov 9, 2016

@nzoschke Thanks for the example - that makes sense.

In terms of resolution order, I think I most-like the priority label...but they're all pretty magical, as you say. I don't like alphabetical as I feel like this should be easily-configurable without having to hack the name of your app to get a specific order, and I don't like sequential as we're used to key-order in a dictionary (which is really what a docker-compose.yml file is) not mattering. I don't fully get the links-based idea, but it sounds like I'm going to have to draw out a dependency graph to figure out what happens when.

No matter which option you pick, I think it'd be useful to have a CLI command (in doctor or elsewhere) which outputs the final order you're going to use to enter them into the ALB config.

dlanger commented Nov 9, 2016

@nzoschke Thanks for the example - that makes sense.

In terms of resolution order, I think I most-like the priority label...but they're all pretty magical, as you say. I don't like alphabetical as I feel like this should be easily-configurable without having to hack the name of your app to get a specific order, and I don't like sequential as we're used to key-order in a dictionary (which is really what a docker-compose.yml file is) not mattering. I don't fully get the links-based idea, but it sounds like I'm going to have to draw out a dependency graph to figure out what happens when.

No matter which option you pick, I think it'd be useful to have a CLI command (in doctor or elsewhere) which outputs the final order you're going to use to enter them into the ALB config.

@nzoschke

This comment has been minimized.

Show comment
Hide comment
@nzoschke

nzoschke Nov 9, 2016

Contributor

@mwarkentin I like that idea a lot. It's like CSS the most specific paths get the highest priority.

Contributor

nzoschke commented Nov 9, 2016

@mwarkentin I like that idea a lot. It's like CSS the most specific paths get the highest priority.

@scottmessinger

This comment has been minimized.

Show comment
Hide comment
@scottmessinger

scottmessinger Nov 9, 2016

@nzoschke This looks great! One thing I noticed is the switch from ALB to ELB seems implicit. If I specify TLS or proxy, I get ELB. Especially for people new to deploying on AWS, this forces one to know the capabilities of ALB vs ELB in order to determine which they're specifying. Instead (or, in addition), I'd love to see something explicit. For instance, a label like convox.router.type=alb. One advantage of making the declaration explicit it helps with diagnosing configuration errors. For instance, convox doctor could have a message like, "You specified you want to use ALB but you specified TLS. ALB doesn't support TLS. Instead, use ELB".

scottmessinger commented Nov 9, 2016

@nzoschke This looks great! One thing I noticed is the switch from ALB to ELB seems implicit. If I specify TLS or proxy, I get ELB. Especially for people new to deploying on AWS, this forces one to know the capabilities of ALB vs ELB in order to determine which they're specifying. Instead (or, in addition), I'd love to see something explicit. For instance, a label like convox.router.type=alb. One advantage of making the declaration explicit it helps with diagnosing configuration errors. For instance, convox doctor could have a message like, "You specified you want to use ALB but you specified TLS. ALB doesn't support TLS. Instead, use ELB".

@piotrb

This comment has been minimized.

Show comment
Hide comment
@piotrb

piotrb Nov 21, 2016

+1 here .. we have a particular use case which makes ALBs ideal

I like @scottmessinger 's idea with explicit types, this way you explicitly choose to take the potential HA drawback, but you gain ALB's advantages.

Anyways our use case is that we have several very slow API calls on our apis, (like 30-60 second requests), our more mission critical requests are more in the <5 sec range ...

before ALB, we had a shared pool of servers running all the API requests, what would happen is a few of the slow requests would come in, and get queued up on workers, you just a couple of those requests could bog down the workers, they would eventually finish, but some requests might take +1min to process including queue time and it started causing issues downstream in apps which depended on those calls.

We ended up doing some instrumentation and found that its only a very small number of urls actually cause the slow requests .. so we switched to ALB and created a new target group specifically for slow requests, this way, if the slow requests come in, they stall EACH OTHER, not the other API calls. The fast API calls can process themselves in one target, and the slow ones in another, isolating load balances queues from each other.

piotrb commented Nov 21, 2016

+1 here .. we have a particular use case which makes ALBs ideal

I like @scottmessinger 's idea with explicit types, this way you explicitly choose to take the potential HA drawback, but you gain ALB's advantages.

Anyways our use case is that we have several very slow API calls on our apis, (like 30-60 second requests), our more mission critical requests are more in the <5 sec range ...

before ALB, we had a shared pool of servers running all the API requests, what would happen is a few of the slow requests would come in, and get queued up on workers, you just a couple of those requests could bog down the workers, they would eventually finish, but some requests might take +1min to process including queue time and it started causing issues downstream in apps which depended on those calls.

We ended up doing some instrumentation and found that its only a very small number of urls actually cause the slow requests .. so we switched to ALB and created a new target group specifically for slow requests, this way, if the slow requests come in, they stall EACH OTHER, not the other API calls. The fast API calls can process themselves in one target, and the slow ones in another, isolating load balances queues from each other.

@SonOfBytes

This comment has been minimized.

Show comment
Hide comment
@SonOfBytes

SonOfBytes Dec 14, 2016

Contributor

I'm looking at convox to deploy 11 mircoservices for a service. They are formed into layered internal and external interfaces. So having a route based ALB would be very helpful. Although an outage due to bad placement would be a concern. Having automated rapid restoration mitigates that concern quite significantly compared to the complexity of running 11 ELBs. 2 ALBs (internal & external) would be much better :)

Does ALB support have a direction/implementation path yet?

Contributor

SonOfBytes commented Dec 14, 2016

I'm looking at convox to deploy 11 mircoservices for a service. They are formed into layered internal and external interfaces. So having a route based ALB would be very helpful. Although an outage due to bad placement would be a concern. Having automated rapid restoration mitigates that concern quite significantly compared to the complexity of running 11 ELBs. 2 ALBs (internal & external) would be much better :)

Does ALB support have a direction/implementation path yet?

@ryanhiebert

This comment has been minimized.

Show comment
Hide comment
@ryanhiebert

ryanhiebert Dec 19, 2016

@mwarkentin : I'm not sure that would the right way to go. For instance, you may wish to have a special */long_running_info route, with a whole bunch of prefixes. Counting the slashes would break that. I did like the simplicity of the idea, though.

ryanhiebert commented Dec 19, 2016

@mwarkentin : I'm not sure that would the right way to go. For instance, you may wish to have a special */long_running_info route, with a whole bunch of prefixes. Counting the slashes would break that. I did like the simplicity of the idea, though.

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Dec 19, 2016

Contributor

@ryanhiebert Not sure I fully follow what you mean - if you have a minute to put together an example docker compose file which would show how it would mess things up, that might be helpful.

Contributor

mwarkentin commented Dec 19, 2016

@ryanhiebert Not sure I fully follow what you mean - if you have a minute to put together an example docker compose file which would show how it would mess things up, that might be helpful.

@ryanhiebert

This comment has been minimized.

Show comment
Hide comment
@ryanhiebert

ryanhiebert Dec 19, 2016

@mwarkentin : Making an example docker compose probably isn't worth the time, so let me try explaining it again.

In a hypothetical app, you can load any URL with a TON of extra logging and information (and it takes a long longer to generate) by adding /expanded_info to the end of the url. This is all done by a separate route.

So you might have the following pages:

  • /foo: Show the foo page
  • /bar/baz: Show the bar-baz page
  • /foo/expanded_info: Show the foo page, as well as very detailed logging and information.
  • /bar/baz/expanded_info: Show the bar-baz page, as well as very detailed logging and information.

In this admittedly contrived scenario, /foo and /bar/baz are in different routes for business reasons, but those same reasons don't apply to the special expanded_info routes. So you'd have 3 different routes, and counting the slashes would apply the routes in the wrong order.

  1. */expanded_info: All expanded info requests go to a special route.
  2. /foo/*: Foo requests go to a dedicated route.
  3. /bar/baz/*: bar-baz requests go to a dedicated route.

If the slashes were counted, the expanded_info route would not catch the two routes it was intended to.

ryanhiebert commented Dec 19, 2016

@mwarkentin : Making an example docker compose probably isn't worth the time, so let me try explaining it again.

In a hypothetical app, you can load any URL with a TON of extra logging and information (and it takes a long longer to generate) by adding /expanded_info to the end of the url. This is all done by a separate route.

So you might have the following pages:

  • /foo: Show the foo page
  • /bar/baz: Show the bar-baz page
  • /foo/expanded_info: Show the foo page, as well as very detailed logging and information.
  • /bar/baz/expanded_info: Show the bar-baz page, as well as very detailed logging and information.

In this admittedly contrived scenario, /foo and /bar/baz are in different routes for business reasons, but those same reasons don't apply to the special expanded_info routes. So you'd have 3 different routes, and counting the slashes would apply the routes in the wrong order.

  1. */expanded_info: All expanded info requests go to a special route.
  2. /foo/*: Foo requests go to a dedicated route.
  3. /bar/baz/*: bar-baz requests go to a dedicated route.

If the slashes were counted, the expanded_info route would not catch the two routes it was intended to.

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Dec 19, 2016

Contributor

@ryanhiebert That makes sense - didn't realize that */expanded_info was a valid route.. thanks!

Contributor

mwarkentin commented Dec 19, 2016

@ryanhiebert That makes sense - didn't realize that */expanded_info was a valid route.. thanks!

@SonOfBytes

This comment has been minimized.

Show comment
Hide comment
@SonOfBytes

SonOfBytes Dec 19, 2016

Contributor

I feel an implicit behaviour of priority based on the order of the rules makes sense unless some or all of the services have priorities at which point they take priority and the rest of the rules follow in order of declaration.

I also feel an implied ELB vs ALB based on best practices is reasonable especially if it makes the best practice more accessible, however an explicit declaration can also be defined which should take precedence or generate an incompatibility error.

Contributor

SonOfBytes commented Dec 19, 2016

I feel an implicit behaviour of priority based on the order of the rules makes sense unless some or all of the services have priorities at which point they take priority and the rest of the rules follow in order of declaration.

I also feel an implied ELB vs ALB based on best practices is reasonable especially if it makes the best practice more accessible, however an explicit declaration can also be defined which should take precedence or generate an incompatibility error.

@SonOfBytes

This comment has been minimized.

Show comment
Hide comment
@SonOfBytes

SonOfBytes Dec 21, 2016

Contributor

One question that comes to mind is how to support ephemeral ports on the ECS which is supported with an ALB. This is expressed by using a host port of 0 and then ECS will use an ephemeral port which the ALB will connect too.

From http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html

The port number on the container instance to reserve for your container. You can specify a non-reserved host port for your container port mapping (this is referred to as static host port mapping), or you can omit the hostPort (or set it to 0) while specifying a containerPort and your container will automatically receive a port (this is referred to as dynamic host port mapping) in the ephemeral port range for your container instance operating system and Docker version.

Contributor

SonOfBytes commented Dec 21, 2016

One question that comes to mind is how to support ephemeral ports on the ECS which is supported with an ALB. This is expressed by using a host port of 0 and then ECS will use an ephemeral port which the ALB will connect too.

From http://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html

The port number on the container instance to reserve for your container. You can specify a non-reserved host port for your container port mapping (this is referred to as static host port mapping), or you can omit the hostPort (or set it to 0) while specifying a containerPort and your container will automatically receive a port (this is referred to as dynamic host port mapping) in the ephemeral port range for your container instance operating system and Docker version.

@airways

This comment has been minimized.

Show comment
Hide comment
@airways

airways Dec 30, 2016

I wanted to add a note with a use case that we would really like to have this feature for.

I am responsible for the server architecture of a large national non-profit headquarter's website and we are in the processing of switching our AWS cluster to be fully Convox managed.

We have a number of microsites and remote services that we proxy requests to -- some of which are at subdomains but the majority of which are in sub-paths of our main domain. We would like to deploy each microsite and/or proxying interface to it's own self-contained service via Convox, and have a single place to configure routing for the sub-path based microsites.

It sounds like this feature would support this use case if it were implemented as described in @nzoschke's comment from Nov 5. We would need to manage DNS separately for the subdomain microsites, but the others would be much easier to manage by simply adding path info to docker-compose.yml.

airways commented Dec 30, 2016

I wanted to add a note with a use case that we would really like to have this feature for.

I am responsible for the server architecture of a large national non-profit headquarter's website and we are in the processing of switching our AWS cluster to be fully Convox managed.

We have a number of microsites and remote services that we proxy requests to -- some of which are at subdomains but the majority of which are in sub-paths of our main domain. We would like to deploy each microsite and/or proxying interface to it's own self-contained service via Convox, and have a single place to configure routing for the sub-path based microsites.

It sounds like this feature would support this use case if it were implemented as described in @nzoschke's comment from Nov 5. We would need to manage DNS separately for the subdomain microsites, but the others would be much easier to manage by simply adding path info to docker-compose.yml.

@dlanger

This comment has been minimized.

Show comment
Hide comment
@dlanger

dlanger Dec 30, 2016

I'd also like this to cooperate with convox.deployment.{minimum,maximum} - if I'm running a rack with only one "spare" instance (eg. I have a service with three container on a four-instance rack), I should be able to spin up three new containers while deploying, as opposed to only doing one at a time.

dlanger commented Dec 30, 2016

I'd also like this to cooperate with convox.deployment.{minimum,maximum} - if I'm running a rack with only one "spare" instance (eg. I have a service with three container on a four-instance rack), I should be able to spin up three new containers while deploying, as opposed to only doing one at a time.

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Dec 30, 2016

Contributor

@airways FYI, it sounded like host-based routing would be available "at some point" when ALB was announced.

Contributor

mwarkentin commented Dec 30, 2016

@airways FYI, it sounded like host-based routing would be available "at some point" when ALB was announced.

@SonOfBytes

This comment has been minimized.

Show comment
Hide comment
@SonOfBytes

SonOfBytes Dec 31, 2016

Contributor

I feel the notation should be, and I think is, independent of the implementation.

That is the same notation could result in an ALB/ECS container or an API Gateway/Lambda function, although probably some other criteria could define which implementation is chosen.

Contributor

SonOfBytes commented Dec 31, 2016

I feel the notation should be, and I think is, independent of the implementation.

That is the same notation could result in an ALB/ECS container or an API Gateway/Lambda function, although probably some other criteria could define which implementation is chosen.

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Jan 6, 2017

Contributor

Just came across request tracing that's only available (as far as I can tell) on ALBs.

This would be helpful for tracing requests through our various services.

Contributor

mwarkentin commented Jan 6, 2017

Just came across request tracing that's only available (as far as I can tell) on ALBs.

This would be helpful for tracing requests through our various services.

@mattmanning

This comment has been minimized.

Show comment
Hide comment
@mattmanning
Contributor

mattmanning commented Jan 16, 2017

@nzoschke nzoschke added this to TODO in Routing Feb 6, 2017

@mwisner

This comment has been minimized.

Show comment
Hide comment
@mwisner

mwisner Feb 8, 2017

ALBs now support Amazon's WAF solution. https://aws.amazon.com/about-aws/whats-new/2016/12/AWS-WAF-now-available-on-Application-Load-Balancer/

For our company, our security requirements require a WAF in place for our infrastructure and being able to use this in front of our own existing API gateway would dramatically simplify our setup.

Right now we've got "ELB" -> "WAF" -> "ELB" -> "Api Gateway" ->

Simplifying this to "ALB/WAF" -> "Api Gateway" -> Would be great.

mwisner commented Feb 8, 2017

ALBs now support Amazon's WAF solution. https://aws.amazon.com/about-aws/whats-new/2016/12/AWS-WAF-now-available-on-Application-Load-Balancer/

For our company, our security requirements require a WAF in place for our infrastructure and being able to use this in front of our own existing API gateway would dramatically simplify our setup.

Right now we've got "ELB" -> "WAF" -> "ELB" -> "Api Gateway" ->

Simplifying this to "ALB/WAF" -> "Api Gateway" -> Would be great.

@countergram

This comment has been minimized.

Show comment
Hide comment
@countergram

countergram Feb 16, 2017

ALB+ECS is attractive, but I have found problems while prototyping it. URL routes are limited to 10 per ALB, and per my research this appears to be a "hard" limit (haven't gone and gotten a specific answer from our AWS rep yet). This means a microservices deployment, or simply a large deployment, will need second-level routing to multiple ALBs. Further, each route for an ALB must have its order specified in the config as a distinct number, which means the configs for services cannot be independent. (Edited to add: this is about trying to use the ALB's routing features - if just swapping out ELB-per-service for ALB-per service, this is irrelevant).

countergram commented Feb 16, 2017

ALB+ECS is attractive, but I have found problems while prototyping it. URL routes are limited to 10 per ALB, and per my research this appears to be a "hard" limit (haven't gone and gotten a specific answer from our AWS rep yet). This means a microservices deployment, or simply a large deployment, will need second-level routing to multiple ALBs. Further, each route for an ALB must have its order specified in the config as a distinct number, which means the configs for services cannot be independent. (Edited to add: this is about trying to use the ALB's routing features - if just swapping out ELB-per-service for ALB-per service, this is irrelevant).

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Feb 16, 2017

Contributor

Might be worth looking at how empire added ALB support: remind101/empire#969

Contributor

mwarkentin commented Feb 16, 2017

Might be worth looking at how empire added ALB support: remind101/empire#969

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Apr 6, 2017

Contributor

Host-based routing is now available!

This makes ALB even more attractive for us as our apps are on separate subdomins.

They've also bumped the rule limit from 10 -> 75, so that could help with some of the complexity discussed earlier about how to break things up, as well as enabling further cost savings over ELB.

Contributor

mwarkentin commented Apr 6, 2017

Host-based routing is now available!

This makes ALB even more attractive for us as our apps are on separate subdomins.

They've also bumped the rule limit from 10 -> 75, so that could help with some of the complexity discussed earlier about how to break things up, as well as enabling further cost savings over ELB.

@dalberto

This comment has been minimized.

Show comment
Hide comment
@dalberto

dalberto Apr 6, 2017

In light of the new rule limit, ALBs would also be useful from an agility perspective. Deploying a new app would be significantly faster if new load balancers did not need to be provisioned.

dalberto commented Apr 6, 2017

In light of the new rule limit, ALBs would also be useful from an agility perspective. Deploying a new app would be significantly faster if new load balancers did not need to be provisioned.

@nathany

This comment has been minimized.

Show comment
Hide comment
@nathany
Contributor

nathany commented Apr 6, 2017

@idris

This comment has been minimized.

Show comment
Hide comment
@idris

idris Apr 8, 2017

Contributor

What's the point of host-based routing? Why not use DNS to have the subdomains point to different load balancers?

Contributor

idris commented Apr 8, 2017

What's the point of host-based routing? Why not use DNS to have the subdomains point to different load balancers?

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Apr 8, 2017

Contributor

Save money by replacing a bunch of ELBs with a single ALB, less resources to manage, ALBs don't require warmup like ELB afaik.

Contributor

mwarkentin commented Apr 8, 2017

Save money by replacing a bunch of ELBs with a single ALB, less resources to manage, ALBs don't require warmup like ELB afaik.

@advance512

This comment has been minimized.

Show comment
Hide comment
@advance512

advance512 Apr 9, 2017

What is the current recommended way to use an ALB instead of many ELBs?

This is such a big issue for modern apps and is a serious barrier for new customers. It's surprising this issue is 6 months old with no solution.

advance512 commented Apr 9, 2017

What is the current recommended way to use an ALB instead of many ELBs?

This is such a big issue for modern apps and is a serious barrier for new customers. It's surprising this issue is 6 months old with no solution.

@ddollar

This comment has been minimized.

Show comment
Hide comment
@ddollar

ddollar May 23, 2017

Member

We are going down the ALB path in the new Praxis codebase

https://github.com/convox/praxis

Member

ddollar commented May 23, 2017

We are going down the ALB path in the new Praxis codebase

https://github.com/convox/praxis

@ddollar ddollar closed this May 23, 2017

@anthonyrisinger

This comment has been minimized.

Show comment
Hide comment
@anthonyrisinger

anthonyrisinger Oct 26, 2017

Contributor

Is Praxis still the path forward for ALB integration? I'm being told in #convox that Praxis is deprecated. ALB is a pretty hard requirement for us.

What is the latest on Convox + ALB?

Contributor

anthonyrisinger commented Oct 26, 2017

Is Praxis still the path forward for ALB integration? I'm being told in #convox that Praxis is deprecated. ALB is a pretty hard requirement for us.

What is the latest on Convox + ALB?

@mwarkentin

This comment has been minimized.

Show comment
Hide comment
@mwarkentin

mwarkentin Oct 26, 2017

Contributor

@anthonyrisinger the plan is now to backport the new praxis features like ALB into convox - latest changes are here: https://github.com/convox/rack/tree/refactor

Contributor

mwarkentin commented Oct 26, 2017

@anthonyrisinger the plan is now to backport the new praxis features like ALB into convox - latest changes are here: https://github.com/convox/rack/tree/refactor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment