Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible bug (or by-design behavior) of using IPv6 addresses to register TCP health check to Consul? #691

Open
yaobinwen-mvs opened this issue Oct 16, 2020 · 0 comments

Comments

@yaobinwen-mvs
Copy link

yaobinwen-mvs commented Oct 16, 2020

Overview

  • What version of operating system are you using?
    I'm using Ubuntu 18.04.

  • What version of docker are you running?
    I'm using Docker version 19.03.12, build 48a66213fe.

  • What version of registrator are you running?
    I'm using the Docker image gliderlabs/registrator:master.

  • Did you build a custom version of registrator? If so, what is that image?
    No. I'm not building a custom version.

  • What is the exact command you are running registrator with?
    See the Steps to Reproduce section for my script that reproduces the issue (not sure if this is a bug or a by-design behavior).

  • What is the exact command you are running your container with?
    See the Steps to Reproduce section.

  • A log capture of all the docker events before, during, and after the issue.
    N/A

  • If relevant, Dockerfile for application that is having issues.
    N/A

Description of the problem

I'm using registrator to register my Docker containers to Consul. The Docker containers have TCP health checks. When an IPv6 address is used to register the containers, the health checks always fail with the error (Note that 32789 is the published port):

dial tcp: address 2601:547:980:b3e0::172f:32789: too many colons in address

How reproducible

Always reproducible.

Steps to Reproduce

  1. Run the Shell script I put in Reproduce script section in a terminal.
  2. In the browser, open http://localhost:8500/.
  3. Open the redis-service page to show the "Health Checks".

Actual Results

  1. The "Serf Health Status" is shown as normal (in green).
  2. The "Service 'redis-service' check" fails with output dial tcp: address 2601:547:980:b3e0::172f:32789: too many colons in address.
  • Note: When the service is just started up, the Output area may be still empty. Need to wait about a few minutes to have the error message show up.

Expected Results

The IPv6 address should be properly wrapped before the port number is appended.

Additional info

  1. This issue can be worked around by wrapping the IPv6 address with [] in the -ip argument.
  2. Initially, I thought this is an issue in Consul so I posted a question here. After discussing with Consul's product manager, we thought it looks more like a bug in Registrator.
  3. In that Consul post, the product manager pointed out the lines that possibly cause the bug:
      } else if tcp := service.Attrs["check_tcp"]; tcp != "" {
		check.TCP = fmt.Sprintf("%s:%d", service.IP, service.Port)
  1. He also pointed out a possible fix:

I think this is something that Registrator could easily fix by using net.JoinHostPort instead of the current IP and port formatting function.

  1. If the user is supposed to properly wrap the IPv6 address, maybe it would be better to explicitly mention this in the IP and Port documentation.

Reproduce script

The script that reproduces the bug is listed below. Please update the variable IPv6_Addr to use your IPv6 address.

#!/bin/sh

# This script reproduces the issue that `registrator` doesn't seem to form the
# TCP health check link appropriately for IPv6 addresses.
#
# Pre-conditions:
# 1). `docker` is installed.

# The IPv6 address that's used to register services.
IPv6_Addr="2601:547:980:b3e0::172f"

# Echo the commands and exit immediately on failure.
set -ex

# Start Consul.
# `boot2docker` is officially deprecated as of 2020-06-22.
# See: https://github.com/boot2docker/boot2docker
NUM=$(docker ps --filter "name=consul" --filter "status=running" --quiet | wc -l)
if [ $NUM -eq 0 ];
then
  docker run -d --name=consul --net=host consul
fi

# Make sure Consul is accessible.
curl http://localhost:8500/v1/catalog/services

# Pull in `registrator:master`.
docker pull gliderlabs/registrator:master

# Start `registrator`.
NUM=$(docker ps --filter "name=registrator" --filter "status=running" --quiet | wc -l)
if [ $NUM -eq 0 ];
then
  docker run -d \
    --name=registrator \
    --net=host \
    --volume=/var/run/docker.sock:/tmp/docker.sock \
    gliderlabs/registrator:master \
      -ip "${IPv6_Addr}" \
      consul://localhost:8500
fi

# Start a `redis` container for testing.
docker run -d \
  --env SERVICE_CHECK_TCP="true" \
  --env SERVICE_CHECK_INTERVAL="10s" \
  --env SERVICE_CHECK_TIMEOUT="1s" \
  --env SERVICE_ID="redis-service-id" \
  --env SERVICE_NAME="redis-service" \
  --name=redis \
  --publish-all \
  redis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant