Skip to content

incorrect advertisement w/ host_network #10001

@mr-karan

Description

@mr-karan

Nomad version

Nomad v1.0.3 (08741d9f2003ec26e44c72a2c0e27cdf0eadb6ee)

Operating system and Environment details

  • Ubuntu 20.04
  • DigitalOcean Droplet

Issue

I've configured host_network on my client's config like:

  host_network "tailscale" {
    cidr = "100.119.138.27/32"
    reserved_ports = "22"
  }

After deploying a task, when I viewed nomad alloc status <id> I noticed a strange thing:

Allocation Addresses
Label   Dynamic  Address
*http   yes      68.x.y.4:20640 -> 80
*https  yes      68.x.y.4:25547 -> 443
*dns    yes      68.x.y.4:53 -> 53

Here 68.x.y.4 is the public IPv4 of my server. But when I the same port mapping using docker ps I observed a completely different (and correct) output:

100.119.138.27:53->53/tcp, 100.119.138.27:53->53/udp, 67/udp, 100.119.138.27:20640->80/tcp, 100.119.138.27:20640->80/udp, 100.119.138.27:25547->443/tcp, 100.119.138.27:25547->443/udp

Reproduction steps

  • Changed the service stanza to add http port instead of https.
  • Noticed the Docker port forwarding was still correct but Consul and Nomad both showed wrong ports.
  • Tried to change the service stanza again and the problem vanished.

It might be hard to reproduce but definitely the wrong host address was mapped inside Nomad. I tried to change the Service multiple times but couldn't reproduce this.

Job file (if appropriate)

job "pihole" {
  datacenters = ["hydra"]
  type        = "service"
  group "web" {
    count = 1
    network {
      port "dns" {
        static       = 53
        to           = 53
        host_network = "tailscale"
      }
      port "http" {
        to           = 80
        host_network = "tailscale"
      }
      port "https" {
        to           = 443
        host_network = "tailscale"
      }
    }
    service {
      name = "pihole-admin"
      tags = ["pihole", "admin"]
      port = "http" # Terminate SSL at Caddy.
    }
    restart {
      attempts = 2
      interval = "2m"
      delay    = "30s"
      mode     = "fail"
    }
    task "app" {
      driver = "docker"
      config {
        image = "pihole/pihole:v5.6"
        # Bind the data directory to preserve config.
        mount {
          type     = "bind"
          target   = "/etc/dnsmasq.d"
          source   = "/data/pihole/dnsmasq.d/"
          readonly = false
        }
        mount {
          type     = "bind"
          target   = "/etc/pihole"
          source   = "/data/pihole/conf/"
          readonly = false
        }
        ports = ["http", "https", "dns"]
      }
      env {
        TZ = "Asia/Kolkata"
      }
      resources {
        cpu    = 200
        memory = 100
      }
    }
  }
}

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions