New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility mode #5684

Merged
merged 2 commits into from Feb 26, 2018

Conversation

Projects
None yet
4 participants
@shin-
Member

shin- commented Feb 17, 2018

Using the --compatibility CLI flag, v3 files can be converted to their v2 equivalent, with deploy options translated when possible. --compatibility can be used with all commands.

The conversion is a "best effort" attempt and shouldn't be relied on for production deployments.

Fixes #4513

Implement compatibility mode,
translating deploy keys to equivalent v2 config if available

Enabled using `--compatibility` CLI flag

Signed-off-by: Joffrey F <joffrey@docker.com>

@shin- shin- added this to the 1.20.0 milestone Feb 17, 2018

@shin- shin- added the format/v3 label Feb 17, 2018

Tests for compatibility mode
Signed-off-by: Joffrey F <joffrey@docker.com>
@shin-

This comment has been minimized.

Show comment
Hide comment
@shin-

shin- Feb 17, 2018

Member

cc @mefyl @vdemeester for review 👓

Member

shin- commented Feb 17, 2018

cc @mefyl @vdemeester for review 👓

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 17, 2018

Member

But doesn’t the V3 format already support memory and cpu limits? ie;

deploy:
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
        reservations:
          cpus: '0.25'
          memory: 20M
Member

thaJeztah commented Feb 17, 2018

But doesn’t the V3 format already support memory and cpu limits? ie;

deploy:
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
        reservations:
          cpus: '0.25'
          memory: 20M
@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 17, 2018

Member

Oh; they were ignored by docker compose? Sorry, missed that

Member

thaJeztah commented Feb 17, 2018

Oh; they were ignored by docker compose? Sorry, missed that

@shin-

This comment has been minimized.

Show comment
Hide comment
@shin-

shin- Feb 20, 2018

Member

@thaJeztah Yes, the point here is to have a way to account for them in Compose as well. It's behind a flag because we can't really guarantee the behavior will match exactly what you would see with stack deploy. Does that make sense?

Member

shin- commented Feb 20, 2018

@thaJeztah Yes, the point here is to have a way to account for them in Compose as well. It's behind a flag because we can't really guarantee the behavior will match exactly what you would see with stack deploy. Does that make sense?

@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Feb 21, 2018

Contributor

Would it make sense make this an option on docker-compose config for converting a config from v3 to v2? That way you could inspect the changes to have some idea of what you're running.

Contributor

dnephin commented Feb 21, 2018

Would it make sense make this an option on docker-compose config for converting a config from v3 to v2? That way you could inspect the changes to have some idea of what you're running.

@shin-

This comment has been minimized.

Show comment
Hide comment
@shin-

shin- Feb 21, 2018

Member

@dnephin it works with config as well - it outputs a valid 2.3 file.

Member

shin- commented Feb 21, 2018

@dnephin it works with config as well - it outputs a valid 2.3 file.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 21, 2018

Member

It's behind a flag because we can't really guarantee the behavior will match exactly what you would see with stack deploy. Does that make sense?

Wondering; things like configs and secrets already have a different implementation in docker-compose; if we document that there are differences, would it make sense to enable it by default (possibly an option to disable it)?

Member

thaJeztah commented Feb 21, 2018

It's behind a flag because we can't really guarantee the behavior will match exactly what you would see with stack deploy. Does that make sense?

Wondering; things like configs and secrets already have a different implementation in docker-compose; if we document that there are differences, would it make sense to enable it by default (possibly an option to disable it)?

@shin-

This comment has been minimized.

Show comment
Hide comment
@shin-

shin- Feb 21, 2018

Member

I want to say no - the secrets emulation is super confusing to users as it is already (see feedback in #4994 for example). I think having users opt into the feature gives us a chance to setup expectations a little better.

Member

shin- commented Feb 21, 2018

I want to say no - the secrets emulation is super confusing to users as it is already (see feedback in #4994 for example). I think having users opt into the feature gives us a chance to setup expectations a little better.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 21, 2018

Member

Ah, yes, then 👍 on the flag

Member

thaJeztah commented Feb 21, 2018

Ah, yes, then 👍 on the flag

@shin- shin- merged commit ec0de7e into master Feb 26, 2018

7 checks passed

ci/circleci: build-linux-binary Your tests passed on CircleCI!
Details
ci/circleci: build-osx-binary Your tests passed on CircleCI!
Details
ci/circleci: test Your tests passed on CircleCI!
Details
continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/jenkins/pr-head This commit looks good
Details
dco-signed All commits are signed

@shin- shin- deleted the compat_mode branch Feb 28, 2018

@SoManyHs SoManyHs referenced this pull request May 2, 2018

Closed

Support docker-compose v3 #218

@thiagolsfortunato

This comment has been minimized.

Show comment
Hide comment
@thiagolsfortunato

thiagolsfortunato Jul 31, 2018

When I executed docker-compose --compatibility config` return this message:

WARNING: The following deploy sub-keys are not supported in compatibility mode and have been ignored: resources.reservations.cpus

deploy:
      resources:
        limits:
          memory: 1G
          cpus: '1'
        reservations:
          memory: 512M
          cpus: '0.50'

Why flag --compatibility does not support resources.reservations.cpus and resources.limits.cpus as memory ?!

thiagolsfortunato commented Jul 31, 2018

When I executed docker-compose --compatibility config` return this message:

WARNING: The following deploy sub-keys are not supported in compatibility mode and have been ignored: resources.reservations.cpus

deploy:
      resources:
        limits:
          memory: 1G
          cpus: '1'
        reservations:
          memory: 512M
          cpus: '0.50'

Why flag --compatibility does not support resources.reservations.cpus and resources.limits.cpus as memory ?!

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 31, 2018

Member

Why flag --compatibility does not support resources.reservations.cpus and resources.limits.cpus equals memory ?!

Those options are scheduling options for swarm-mode services (i.e., they don't actually reserve the resources, only make swarm take those into account when scheduling containers on a node).

Docker Compose doesn't work with swarm mode (it deploys local containers), so there's no orchestrator to take those limits into account

Member

thaJeztah commented Jul 31, 2018

Why flag --compatibility does not support resources.reservations.cpus and resources.limits.cpus equals memory ?!

Those options are scheduling options for swarm-mode services (i.e., they don't actually reserve the resources, only make swarm take those into account when scheduling containers on a node).

Docker Compose doesn't work with swarm mode (it deploys local containers), so there's no orchestrator to take those limits into account

@thiagolsfortunato

This comment has been minimized.

Show comment
Hide comment
@thiagolsfortunato

thiagolsfortunato Jul 31, 2018

For not swarm-mode, how can I manager memory and cpu resources deploying local containers?

thiagolsfortunato commented Jul 31, 2018

For not swarm-mode, how can I manager memory and cpu resources deploying local containers?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 31, 2018

Member

Using the feature described here; the warning that's printed is about cpu reservation;

Using this compose file:

version: '3.5'
services:
  web:
    image: nginx:alpine
    deploy:
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
docker-compose --compatibility up -d
WARNING: The following deploy sub-keys are not supported in compatibility mode and have been ignored: resources.reservations.cpus
Creating network "compat_default" with the default driver
Creating compat_web_1 ... done
docker inspect compat_web_1

...
    "Memory": 52428800,
    "NanoCpus": 500000000,
...
Member

thaJeztah commented Jul 31, 2018

Using the feature described here; the warning that's printed is about cpu reservation;

Using this compose file:

version: '3.5'
services:
  web:
    image: nginx:alpine
    deploy:
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
docker-compose --compatibility up -d
WARNING: The following deploy sub-keys are not supported in compatibility mode and have been ignored: resources.reservations.cpus
Creating network "compat_default" with the default driver
Creating compat_web_1 ... done
docker inspect compat_web_1

...
    "Memory": 52428800,
    "NanoCpus": 500000000,
...
if 'cpus' in resources_dict['limits']:
service_dict['cpus'] = float(resources_dict['limits']['cpus'])
if 'reservations' in resources_dict:
service_dict['mem_reservation'] = resources_dict['reservations'].get('memory')

This comment has been minimized.

@thaJeztah

thaJeztah Jul 31, 2018

Member

@shin- I just noticed this; this part isn't correct, because --memory-reservation (although named similar) is something very different than --reserve-memory on docker service; see moby/moby#14579

@thaJeztah

thaJeztah Jul 31, 2018

Member

@shin- I just noticed this; this part isn't correct, because --memory-reservation (although named similar) is something very different than --reserve-memory on docker service; see moby/moby#14579

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment