unplaceable allocation using exec driver with bridge network

[rahadiangg]

Nomad version

Nomad v1.8.2
BuildDate 2024-07-16T08:50:09Z
Revision 7f0822c

Operating system and Environment details

PRETTY_NAME="Ubuntu 24.04 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"

AWS EC2 t3.micro

Issue

task with exec driver can't scheduled. In all Client node already installed CNI, but can't find nomad bridge interface in

plugins.cni.version.bandwidth            = v1.5.0
plugins.cni.version.bridge               = v1.5.0
plugins.cni.version.dhcp                 = v1.5.0
plugins.cni.version.dummy                = v1.5.0
plugins.cni.version.firewall             = v1.5.0
plugins.cni.version.host-device          = v1.5.0
plugins.cni.version.host-local           = v1.5.0
plugins.cni.version.ipvlan               = v1.5.0
plugins.cni.version.loopback             = v1.5.0
plugins.cni.version.macvlan              = v1.5.0
plugins.cni.version.portmap              = v1.5.0
plugins.cni.version.ptp                  = v1.5.0
plugins.cni.version.sbr                  = v1.5.0
plugins.cni.version.static               = v1.5.0
plugins.cni.version.tap                  = v1.5.0
plugins.cni.version.tuning               = v1.5.0
plugins.cni.version.vlan                 = v1.5.0
plugins.cni.version.vrf                  = v1.5.0

Reproduction steps

run job with exec driver and use bridge network

Expected Result

job should run using bridge network

Actual Result

Scheduler dry-run:
- WARNING: Failed to place all allocations.
  Task Group "hello-world-group" (failed to place 4 allocations):
    * Constraint "missing network": 2 nodes excluded by filter

Job file (if appropriate)

job "13.bridge-networing-four-alloc" {

  type = "service"
  constraint {
    attribute = "${attr.kernel.name}"
    value     = "linux"
  }

  group "hello-world-group" {

    count = 4

    network {
      mode = "bridge"

      port "app-port" {
        to = 3000
      }
    }

    task "hello-world-app" {
      driver = "exec"

      config {
        command = "app_${attr.kernel.name}_${attr.cpu.arch}"
      }

      artifact {
        source = "https://<url>/app_${attr.kernel.name}_${attr.cpu.arch}"
      }

      resources {
        cpu = 100 # MHz
        memory = 100 # MB
      }
    }
  }
}

Nomad Server logs (if appropriate)

Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:     2024-07-28T03:03:31.863Z [WARN]  client.fingerprint_mgr: failed to detect bridge kernel module, bridge network mode disabled:
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   error=
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   | 4 errors occurred:
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   | \t* failed to find /sys/module/bridge: stat /sys/module/bridge: no such file or directory
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   | \t* module bridge not in /proc/modules
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   | \t* module bridge not in /lib/modules/6.8.0-1009-aws/modules.builtin
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   | \t* module bridge not in /lib/modules/6.8.0-1009-aws/modules.dep
Jul 28 03:03:31 ip-172-31-36-203 nomad[12932]:   |

Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:     2024-07-28T02:52:10.909Z [DEBUG] worker: dequeued evaluation: worker_id=6a2aa68f-40ad-cb7f-47a8-25fcf1f1d963 eval_id=d9acc6a1-7da0-11a5-cbc3-4e4334ca3ccf type=service namespace=default job_id=13.bridge-networing-four-alloc node_id="" triggered_by=job-register
Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:     2024-07-28T02:52:10.910Z [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=d9acc6a1-7da0-11a5-cbc3-4e4334ca3ccf job_id=13.bridge-networing-four-alloc namespace=default worker_id=6a2aa68f-40ad-cb7f-47a8-25fcf1f1d963
Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:   results=
Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:   | Total changes: (place 4) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:   | Created Deployment: "96f04aaf-ff6a-136a-6277-49ee1be3961f"
Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:   | Desired Changes for "hello-world-group": (place 4) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)
Jul 28 02:52:10 ip-172-31-36-203 nomad[12763]:

Nomad Client logs (if appropriate)

[rahadiangg]

Solved, I need to enable bridge module on linux first.

sudo modprobe bridge
sudo sudo modprobe br_netfilter 

[tgross]

Similar to #23583. Looks like this changed in the upstream distro, so we'll need to make sure it's documented.