docker-compose up fails with 'expected string or buffer' when specifying ports #4972

agladysh · 2017-06-28T14:43:10Z

$ cat docker-compose.yml
version: "3"
services:
  bug:
    image: hello-world
    ports:
      - "8000:8000"

$ docker-compose up
ERROR: for bug  expected string or buffer
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python2.7/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python2.7/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
TypeError: expected string or buffer

$ docker-compose --version
docker-compose version 1.14.0, build c7bdf9e

$ docker --version
Docker version 17.05.0-ce, build 89658be

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 17.04
Release:	17.04
Codename:	zesty

The text was updated successfully, but these errors were encountered:

agladysh · 2017-06-28T14:48:16Z

NB: long ports syntax yields the same error message with a different stack:

version: "3.2"
services:
  bug:
    image: hello-world
    ports:
      - target: 8000
        published: 8000
        mode: host
        protocol: tcp

$ docker-compose up
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 115, in perform_command
    project = project_from_options('.', options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/command.py", line 37, in project_from_options
    override_dir=options.get('--project-directory'),
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/command.py", line 91, in get_project
    config_data = config.load(config_details)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/config.py", line 362, in load
    for config_file in config_details.config_files
  File "/usr/local/lib/python2.7/dist-packages/compose/config/config.py", line 528, in process_config_file
    validate_against_config_schema(config_file)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/validation.py", line 378, in validate_against_config_schema
    config_file.filename)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/validation.py", line 435, in handle_errors
    errors = list(sorted(errors, key=str))
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 304, in properties_draft4
    schema_path=property,
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 16, in patternProperties
    v, subschema, path=k, schema_path=pattern,
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 216, in ref
    for error in validator.descend(instance, resolved):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 304, in properties_draft4
    schema_path=property,
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 55, in items
    for error in validator.descend(item, items, path=index):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 341, in oneOf_draft4
    errs = list(validator.descend(instance, subschema, schema_path=index))
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 121, in descend
    for error in self.iter_errors(instance, schema):
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/validators.py", line 105, in iter_errors
    for error in errors:
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_validators.py", line 163, in format
    validator.format_checker.check(instance, format)
  File "/home/agladysh/.local/lib/python2.7/site-packages/jsonschema/_format.py", line 96, in check
    result = func(instance)
  File "/usr/local/lib/python2.7/dist-packages/compose/config/validation.py", line 51, in format_ports
    split_port(instance)
  File "/home/agladysh/.local/lib/python2.7/site-packages/docker/utils/ports.py", line 57, in split_port
    match = PORT_SPEC.match(port)
TypeError: expected string or buffer

agladysh · 2017-06-28T15:19:58Z

NB: Verbose output (for "8000:8000"):

$ docker-compose --verbose up
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.auth.find_config_file: Trying paths: ['/home/agladysh/.docker/config.json', '/home/agladysh/.dockercfg']
docker.auth.find_config_file: No config file found
compose.cli.command.get_client: docker-compose version 1.14.0, build c7bdf9e
docker-py version: 2.4.0
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
compose.cli.command.get_client: Docker base_url: http+docker://localunixsocket
compose.cli.command.get_client: Docker version: KernelVersion=4.10.0-24-generic, Arch=amd64, BuildTime=2017-05-04T22:27:42.481436991+00:00, ApiVersion=1.29, Version=17.05.0-ce, MinAPIVersion=1.12, GitCommit=89658be, Os=linux, GoVersion=go1.7.5
compose.cli.verbose_proxy.proxy_callable: docker info <- ()
compose.cli.verbose_proxy.proxy_callable: docker info -> {u'Architecture': u'x86_64',
 u'BridgeNfIp6tables': True,
 u'BridgeNfIptables': True,
 u'CPUSet': True,
 u'CPUShares': True,
 u'CgroupDriver': u'cgroupfs',
 u'ClusterAdvertise': u'',
 u'ClusterStore': u'',
 u'ContainerdCommit': {u'Expected': u'9048e5e50717ea4497b757314bad98ea3763c145',
                       u'ID': u'9048e5e50717ea4497b757314bad98ea3763c145'},
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- (u'scale_default')
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {u'Attachable': True,
 u'Containers': {},
 u'Created': u'2017-06-28T17:36:51.315937208+03:00',
 u'Driver': u'bridge',
 u'EnableIPv6': False,
 u'IPAM': {u'Config': [{u'Gateway': u'172.19.0.1',
                        u'Subnet': u'172.19.0.0/16'}],
           u'Driver': u'default',
           u'Options': None},
 u'Id': u'f0da86e837353be0987f32109b8dd81696adfef59add61767ea30e90769c939d',
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={u'label': [u'com.docker.compose.project=scale', u'com.docker.compose.oneoff=False']})
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=scale', u'com.docker.compose.service=bug', u'com.docker.compose.oneoff=False']})
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('hello-world')
compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {u'Architecture': u'amd64',
 u'Author': u'',
 u'Comment': u'',
 u'Config': {u'ArgsEscaped': True,
             u'AttachStderr': False,
             u'AttachStdin': False,
             u'AttachStdout': False,
             u'Cmd': [u'/hello'],
             u'Domainname': u'',
             u'Entrypoint': None,
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=scale', u'com.docker.compose.service=bug', u'com.docker.compose.oneoff=False']})
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.parallel.feed_queue: Pending: set([<Service: bug>])
compose.parallel.feed_queue: Starting producer thread for <Service: bug>
compose.parallel.parallel_execute_iter: Failed: <Service: bug>
compose.parallel.feed_queue: Pending: set([])

ERROR: for bug  expected string or buffer
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python2.7/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python2.7/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
TypeError: expected string or buffer

agladysh · 2017-06-28T15:27:15Z

NB: docker-compose run --service-ports works:

$ docker-compose run --service-ports bug /hello

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://cloud.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

agladysh · 2017-06-28T15:34:38Z

NB: Same error for docker-compose 1.13.0. 1.12.0 seems to work.

sudo pip install docker-compose==1.12.0

sono-bfio · 2017-06-28T16:20:38Z

Same issue here via python3

ERROR: for consul  expected string or bytes-like object
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python3.5/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
  File "/usr/local/lib/python3.5/dist-packages/compose/parallel.py", line 167, in producer
    result = func(obj)
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 410, in do
    rescale=rescale
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 460, in execute_convergence_plan
    self.show_scale_warnings(scale)
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 205, in show_scale_warnings
    if self.specifies_host_port() and desired_num > 1:
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 983, in specifies_host_port
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 983, in <genexpr>
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 963, in has_host_port
    _, external_bindings = split_port(binding)
  File "/usr/local/lib/python3.5/dist-packages/docker/utils/ports.py", line 57, in split_port
    match = PORT_SPEC.match(port)
TypeError: expected string or bytes-like object

Worked yesterday morning, started failing this morning, here is a consul snippet with the port defined (This produces the error above).

  consul:
    image: consul:0.8.3
    network_mode: "host"
    restart: always
    command: "agent -server -ui -bootstrap-expect=1 -client=0.0.0.0 -recursor=8.8.8.8 -recursor=8.8.4.4 -dns-port=53 
    environment:
      CONSUL_BIND_INTERFACE: eth0
      CONSUL_LOCAL_CONFIG: '{"dns_config":{"allow_stale":false} }'
      CONSUL_ALLOW_PRIVILEGED_PORTS: "true"
    volumes:
      - "/path/to/vol:/consul/data"
    ports:
      - "8300:8300/tcp"

If we remove the ports section consul starts.

Python 3 Modules

docker==2.4.0
docker-compose==1.14.0
docker-pycreds==0.2.1
dockerpty==0.4.1

Work around:
docker was updated to 2.4.0 15 hours ago. I reverted back to docker 2.3.0 and it worked.

pip uninstall docker
pip uninstall docker-compose

pip install docker==2.3.0
pip install docker-compose==1.14.0

Works^^

thesquelched · 2017-06-28T16:37:10Z

Changing all of the ports in my docker-compose.yml to strings did not appear to fix anything, FYI.

digitaldavenyc · 2017-06-28T16:57:47Z

Can confirm the downgrade of docker and docker-compose resolves the problem. This issue caused all our docker deployments on circle to fail.

rikvanderkemp · 2017-06-28T17:24:00Z

My Two cents, I am having similar issues where it fails with TypeError: expected string or bytes-like object

It seems like it passes the function split_port twice, once with the "correct" values like 8080:80 and once with a ServicePort object.

rikvanderkemp · 2017-06-28T17:32:47Z

It seems the culprit on my system was python-docker 2.4, I downgraded to 2.3.0 and Docker / Docker-compose both work with the latest versions!

So if you have this issue try downgrading to python-docker 2.3 first

inhumantsar · 2017-06-28T17:53:12Z

Downgrading the docker python library worked for me as well:

pip install --force docker==2.3.0

shin- · 2017-06-28T19:10:49Z

Thanks for the reports everyone, I'm currently working on a 2.4.2 release of the Python SDK that should fix this particular issue. It should be available in the next few hours.

shin- · 2017-06-28T20:57:38Z

For anyone affected, please pip install -U docker==2.4.2. Let me know if there are any issues still.

lforg37 · 2017-06-28T21:04:33Z

Hello
After upgrading I still have the issue with the following docker-compose excerpt:

services:
    nginx:
        build:
            context: ./mdb_httpproxy
        networks:
            dmz:
        ports:
            - "80"
        volumes:
            - corel-10k:/data/corel_images/:ro

shin- · 2017-06-28T21:39:28Z

@lforg37 What's the output of pip freeze ?

WhyNotHugo · 2017-06-28T21:51:11Z

Created a virtualenv, and installed docker-compose with latest deps (my distro hasn't updated to 2.4.2 yet). Working now.

lforg37 · 2017-06-28T22:08:05Z

The output of pip freeze is

cached-property==1.3.0
certifi==2017.4.17
chardet==3.0.4
colorama==0.3.9
docker==2.4.2
docker-compose==1.14.0
docker-pycreds==0.2.1
dockerpty==0.4.1
docopt==0.6.2
idna==2.5
jsonschema==2.6.0
PyYAML==3.12
requests==2.11.1
six==1.10.0
texttable==0.8.8
urllib3==1.21.1
websocket-client==0.44.0

lforg37 · 2017-06-28T22:11:22Z

I've tried to install a new virtualenv and now it is working. Sorry for the false alert.

hernandev · 2017-06-29T00:29:40Z

@lforg37 I get this error using official arch linux package

wtwde · 2017-06-29T02:44:16Z

I get the same issue

lukasojd · 2017-06-29T06:46:31Z

+1 on clear debian with use pip

nogo · 2017-06-29T11:25:25Z

Issue still exists for version 1.14 and 1.13
Arch linux

cnu · 2017-06-29T12:40:19Z

Issue exists in Arch Linux for v1.14, but sudo pip install -U docker==2.4.2 fixes the problem.

shin- · 2017-06-29T19:30:17Z

Distros will need to update their system packages, there's not much I can do about that. As always, our advice is to install using pip inside a virtualenv to avoid these issues.

nogo · 2017-07-03T08:53:22Z

@shin- you are right, was distro related. everything work now after "python-docker" package upgrade

hernandev · 2017-07-06T03:53:15Z

Not seeing the problem after a new upgrade on arch, have been working for a couple of days now.

Just a follow up.

kenji-getpowered · 2017-11-26T12:25:49Z

Using hypriot

Version 1.5.0 | hypriotos-rpi-v1.5.0.img.zip Checksum | Docker 17.05.0-ce, kernel 4.4.50 | 29.06.2017

Downgrading

pip install docker==2.3.0

Worked for me

FDiskas · 2017-12-30T18:57:45Z

in my case I changed ports to integers 🍩 and this fixed my issue

ports:
      - 8000:8000

agladysh changed the title ~~docker-compose error 'expected string or buffer' when specifying ports~~ docker-compose up fails with 'expected string or buffer' when specifying ports Jun 28, 2017

sono-bfio mentioned this issue Jun 28, 2017

Issue with port option in 2.4.0 version docker/docker-py#1668

Closed

rjruizes mentioned this issue Jun 28, 2017

Getting "TypeError: expected string or buffer" when using port mapping #4971

Closed

shin- mentioned this issue Jun 28, 2017

Docker-compose up fails in Gitlab CI with Exit code: 1 #4968

Closed

shin- added area/up kind/bug labels Jun 28, 2017

shin- closed this as completed Jun 29, 2017

So-Cool mentioned this issue Jun 29, 2017

docker-compose error IRC-SPHERE/HyperStream-Dockerfiles#6

Closed

This was referenced Jul 12, 2017

docker-compose fails to parse ports from the compose file #4967

Closed

pip installed docker-compose cannot start up the docker containers #4966

Closed

LongLiveCHIEF mentioned this issue Nov 25, 2020

[Feature] Make plugins persistent OctoPrint/octoprint-docker#103

Closed

lerlands mentioned this issue Aug 7, 2022

ERROR: for bitnami_wordpress_1 a bytes-like object is required, not 'str' bitnami/containers#1462

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker-compose up fails with 'expected string or buffer' when specifying ports #4972

docker-compose up fails with 'expected string or buffer' when specifying ports #4972

agladysh commented Jun 28, 2017 •

edited

Loading

agladysh commented Jun 28, 2017

agladysh commented Jun 28, 2017 •

edited

Loading

agladysh commented Jun 28, 2017 •

edited

Loading

agladysh commented Jun 28, 2017 •

edited

Loading

sono-bfio commented Jun 28, 2017 •

edited

Loading

thesquelched commented Jun 28, 2017

digitaldavenyc commented Jun 28, 2017

rikvanderkemp commented Jun 28, 2017

rikvanderkemp commented Jun 28, 2017

inhumantsar commented Jun 28, 2017

shin- commented Jun 28, 2017 •

edited

Loading

shin- commented Jun 28, 2017

lforg37 commented Jun 28, 2017 •

edited

Loading

shin- commented Jun 28, 2017

WhyNotHugo commented Jun 28, 2017

lforg37 commented Jun 28, 2017

lforg37 commented Jun 28, 2017 •

edited

Loading

hernandev commented Jun 29, 2017

wtwde commented Jun 29, 2017

lukasojd commented Jun 29, 2017

nogo commented Jun 29, 2017

cnu commented Jun 29, 2017 •

edited

Loading

shin- commented Jun 29, 2017

nogo commented Jul 3, 2017

hernandev commented Jul 6, 2017

kenji-getpowered commented Nov 26, 2017

FDiskas commented Dec 30, 2017

docker-compose up fails with 'expected string or buffer' when specifying ports #4972

docker-compose up fails with 'expected string or buffer' when specifying ports #4972

Comments

agladysh commented Jun 28, 2017 • edited Loading

agladysh commented Jun 28, 2017

agladysh commented Jun 28, 2017 • edited Loading

agladysh commented Jun 28, 2017 • edited Loading

agladysh commented Jun 28, 2017 • edited Loading

sono-bfio commented Jun 28, 2017 • edited Loading

thesquelched commented Jun 28, 2017

digitaldavenyc commented Jun 28, 2017

rikvanderkemp commented Jun 28, 2017

rikvanderkemp commented Jun 28, 2017

inhumantsar commented Jun 28, 2017

shin- commented Jun 28, 2017 • edited Loading

shin- commented Jun 28, 2017

lforg37 commented Jun 28, 2017 • edited Loading

shin- commented Jun 28, 2017

WhyNotHugo commented Jun 28, 2017

lforg37 commented Jun 28, 2017

lforg37 commented Jun 28, 2017 • edited Loading

hernandev commented Jun 29, 2017

wtwde commented Jun 29, 2017

lukasojd commented Jun 29, 2017

nogo commented Jun 29, 2017

cnu commented Jun 29, 2017 • edited Loading

shin- commented Jun 29, 2017

nogo commented Jul 3, 2017

hernandev commented Jul 6, 2017

kenji-getpowered commented Nov 26, 2017

FDiskas commented Dec 30, 2017

agladysh commented Jun 28, 2017 •

edited

Loading

agladysh commented Jun 28, 2017 •

edited

Loading

agladysh commented Jun 28, 2017 •

edited

Loading

agladysh commented Jun 28, 2017 •

edited

Loading

sono-bfio commented Jun 28, 2017 •

edited

Loading

shin- commented Jun 28, 2017 •

edited

Loading

lforg37 commented Jun 28, 2017 •

edited

Loading

lforg37 commented Jun 28, 2017 •

edited

Loading

cnu commented Jun 29, 2017 •

edited

Loading