Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory-scaling a cluster with a single node causes downtime #1173

Open
Qqwy opened this issue Aug 16, 2022 · 2 comments
Open

Memory-scaling a cluster with a single node causes downtime #1173

Qqwy opened this issue Aug 16, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@Qqwy
Copy link

Qqwy commented Aug 16, 2022

Given

  • Any simple Fly app.
  • That runs on a cluster with a single instance. In other words: scale count == 1.
  • No volumes
  • No special scaling settings in the fly.toml nor by calling fly scale .... Everything is at stock settings.

These preconditions are trivially satisfied by fly launching any of the example apps in the Fly documentation.
For instance, with the Rails example app.

When

When running fly scale memory some_other_amount (such as fly scale memory 512)

Expected behaviour

A 'canary'-style deploy is done. That is:

  • A new VM instance is started in the background.
  • Fly waits until it is fully started and its health checks are OK
  • Only now is the current VM instance stopped and traffic redirected to the new VM instance.

Actual behaviour

An 'immediate'-style deploy is done. That is:

  • The current VM instance is stopped immediately, before the new VM instance is operational.
  • This causes a period of downtime. (The downtime takes roughly as long as it takes the new VM instance to boot and pass its health checks.)

After @davissp14 confirmed that this is not intended behaviour in the Forum topic about this problem, formalizing it in an issue seemed like the right thing to do. I hope this aligns with Fly's dev guidelines 😊 .

@Qqwy Qqwy added the bug Something isn't working label Aug 16, 2022
@redjonzaci
Copy link
Contributor

@davissp14 are you still working on this internally? Can I help here?

@rubn-g
Copy link

rubn-g commented Nov 4, 2023

More than a year and no updates about a bug that causes downtime to your users?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants