Proposal: Docker Client Version Negotiation

[dave-tucker]

There are many times when the Docker version on a client can differ to that on a server.

This is typically because that end users are faster to update their clients whereas hosted Docker services or servers in production are slower to upgrade.

This issue can be much worse when attempting to RC test the new version of Docker as you need to switch Docker Client versions or set DOCKER_API_VERSION. The latter can have consequences e.g newer client features and flags failing against older engine versions.

Today

Attempting to connect to older server:

$ docker ps
Error response from daemon: client is newer than server (client API version: 1.23, server API version: 1.22)

Attempting to create an IPv6 network

$ DOCKER_API_VERSION=1.22 docker network create --ipv6 foo
2d6a8ce8e8303d27fbfdc19cb1d2a73328d5d278a1dea173d31ec4b9d586e8ca
$ DOCKER_API_VERSION=1.22 docker network inspect foo
[
    {
        "Name": "foo",
        "Id": "2d6a8ce8e8303d27fbfdc19cb1d2a73328d5d278a1dea173d31ec4b9d586e8ca",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1/16"
                }
            ]
        },
        "Internal": false,
        "Containers": {},
        "Options": {},
        "Labels": null
    }
]

The command succeeds, but IPv6 wasn't enabled as this is only available in a newer API version!

Tomorrow

Regardless of version, commands should work:

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Handling of unsupported operations

$ docker network create --ipv6 foo
Error: `--ipv6` is not available on your server
Client Version: 1.23 
Server Version: 1.22

Suggestion

If the Docker Client were able to maintain some form of session state, it could detect and preserve the Docker API version for a session. It should also be possible to prevent newer client features from being used on servers that do not support them to avoid inconsistencies like the example noted above

[cpuguy83]

Part of the issue is also that the data structures can also change and potentially cause encode/decode issues.
This should be something we can do, but I don't think is a trivial change to implement.

[AkihiroSuda]

+1, but supporting all the protocol versions is a difficult work.

How about just supporting a single specific version?

[StefanScherer]

Just a spontanous idea. If managing the different API version in the code is difficult. What about delegating the command to the right version of the docker client?

Eg. if the user has older versions of the Docker client installed beside the docker binary (or docker.exe on Windows).

Docker Engine API 1.22

docker (1.11, API 1.24)
docker-1.11.0
docker-1.10.3 (API 1.22)

The docker binary calls docker-1.10.3 with the same arguments to talk with API 1.22 if Docker Engine has different API version.

But, ok there is dvm https://github.com/getcarina/dvm :-)

[justincormack]

@StefanScherer that is strinctly better than dvm as it is automatic rather than manual - people who routinely use multiple endpoints would find it much easier.

[vdemeester]

It's a little bit related to #27745 I think 👼

[AkihiroSuda]

can we close this issue, according to #27745?
cc @vieux

[thaJeztah]

Yes, this was implemented, and available in docker 1.13 and up

[Bharathkumarraju]

I am getting error as below...

'''

docker version

Client:
Version: 1.12.6
API version: 1.24
Package version: docker-1.12.6-48.git0fdc778.el7.x86_64
Go version: go1.8.3
Git commit: 0fdc778/1.12.6
Built: Thu Jul 20 00:06:39 2017
OS/Arch: linux/amd64

Server:
Version: 1.12.6
API version: 1.24
Package version: docker-1.12.6-48.git0fdc778.el7.x86_64
Go version: go1.8.3
Git commit: 0fdc778/1.12.6
Built: Thu Jul 20 00:06:39 2017
OS/Arch: linux/amd64

python

Python 2.7.5 (default, Aug 29 2016, 10:12:21)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import docker
client = docker.APIClient(base_url='unix://var/run/docker.sock')
print client.version()
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/docker/api/daemon.py", line 177, in version
return self._result(self._get(url), json=True)
File "/usr/lib/python2.7/site-packages/docker/api/client.py", line 226, in _result
self._raise_for_status(response)
File "/usr/lib/python2.7/site-packages/docker/api/client.py", line 222, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/usr/lib/python2.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 400 Client Error: Bad Request ("client is newer than server (client API version: 1.30, server API version: 1.24)")
quit()

'''

[thaJeztah]

@Bharathkumarraju that looks to be an issue with the python client you're using, which may not have version-negotiation, or needs to be configured to use a specific API version.

Can you report that in the docker-py issue tracker? That's where the Python SDK is maintained; https://github.com/docker/docker-py (make sure to check the docs and search existing issues before opening a new issue there)

[Bharathkumarraju]

@thaJeztah

Great thanks as of now I am using like this is that okay right?

client = docker.DockerClient(version='1.24')

for i in client.containers.list():
print.id

Works for me

[remram44]

This doesn't work today. Even though I set DOCKER_API_VERSION="1.23", the client fails stating that it is newer:

Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)

The output from docker version is really weird, the client states that it downgraded from 1.23 to 1.24 (which sounds like an upgrade to me):

$ docker version
Client:
 Version:      17.09.0-ce
 API version:  1.24 (downgraded from 1.23)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:40:09 2017
 OS/Arch:      darwin/amd64
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)

[thaJeztah]

@remram44 see #35008 but the DOCKER_API_VERSION environment variable is a different feature; the version negotiation feature that's proposed here (and implemented in #27745) automatically negotiates the version to use based on the response from the daemon. This works with any daemon running docker 1.12 and up. The DOCKER_API_VERSION environment variable disables version negotiation, and manually overrides the version (but see the linked issue for a bug in 17.09)