Standardise to DUT in integration tests by snuffy22 · Pull Request #3536 · ipspace/netlab

snuffy22 · 2026-06-28T23:59:45Z

Make it slightly less guess work sometimes to identify which are the DUT by consistently naming them.

snuffy22 · 2026-06-29T00:32:32Z

Please note I am still yet to run all tests to confirm I have not broken anything.

ipspace · 2026-06-29T14:33:37Z

Nice project, thank you! Please tell me when you're done, and I'll run FRR integration tests.

Copilot

Pull request overview

This PR standardizes “device under test” naming across integration topologies by renaming key nodes/groups/links to dut (and variants like dut_s1, dut_a, dut_c1) to make it easier to identify DUT roles consistently in test definitions.

Changes:

Renamed DUT nodes across many integration YAML topologies (VXLAN, VLAN, LAG/MLAG, EVPN, EIGRP, DHCP, initial scenarios).
Updated link endpoints and validation plugin references to match the new DUT node names.
Added a new VLAN integration plugin (31-dut_s2-native.py) to adjust native VLAN handling for a renamed probe/DUT scenario.

Reviewed changes

Copilot reviewed 44 out of 45 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/integration/vxlan/02-vxlan-bridging-multinode.yml	Rename DUT switches to `dut_s1`/`dut_s2` and update VLAN links and OSPF validation references
tests/integration/vlan/70-vlan-1-trunk.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link references
tests/integration/vlan/63-vlan-mixed-native.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link and OSPF validation references
tests/integration/vlan/62-vlan-mixed-trunk.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link and OSPF validation references
tests/integration/vlan/61-vlan-routed-native.yml	Rename router-on-stick node from `ros` to `dut` and update validation references
tests/integration/vlan/52-vlan-vrf-lite.yml	Rename DUT routers to `dut_a`/`dut_b` and update VLAN/link and validation references
tests/integration/vlan/51-vlan-routed-trunk.yml	Rename router-on-stick node from `ros` to `dut` and update validation references
tests/integration/vlan/42-vlan-irb-native.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link and validation references
tests/integration/vlan/41-vlan-bridge-native.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link references
tests/integration/vlan/33-vlan-irb-trunk.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link and validation references
tests/integration/vlan/32-vlan-bridge-trunk-router.yml	Rename DUT switches to `dut_s1`/`dut_s2` and update VLAN/link references
tests/integration/vlan/31-vlan-bridge-trunk.yml	Rename DUT switches to `dut_s1`/`dut_s2`, update probe membership, and update VLAN/link references
tests/integration/vlan/31-dut_s2-native.py	New integration plugin to modify trunk/native VLAN behavior for the renamed DUT/probe setup
tests/integration/vlan/23-vlan-mixed-multiple.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link references
tests/integration/vlan/22-vlan-irb-multiple.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link references
tests/integration/vlan/21-vlan-irb-single.yml	Rename DUT switch from `s1` to `dut` and update IDs, links, and validation references
tests/integration/vlan/02-vlan-bridge-multiple.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link references
tests/integration/vlan/01-vlan-bridge-single.yml	Rename DUT switch from `s1` to `dut` and update VLAN/link references
tests/integration/lag/x-12-mlag-dual-cross.yml	Rename MLAG DUT switches to `dut_s1`/`dut_s2` and update LAG member links
tests/integration/lag/11-mlag-anycast.yml	Rename MLAG DUT switches to `dut_s1`/`dut_s2` and update LAG member links
tests/integration/lag/10-mlag.yml	Rename MLAG DUT switches to `dut_s1`/`dut_s2` and update LAG member links
tests/integration/initial/07-router.yml	Rename single router node from `r` to `dut` and update links
tests/integration/initial/03-unnumbered.yml	Rename router node from `r` to `dut` and update links
tests/integration/initial/02-loopback.yml	Rename router node from `r` to `dut` and update routing static next-hop and links
tests/integration/initial/01-interfaces.yml	Rename router node from `r` to `dut` and update routing static next-hop and links
tests/integration/evpn/x-03-vxlan-symmetric-irb-same-vendor.yml	Rename EVPN/VXLAN DUT switches to `dut_s1`/`dut_s2` and update validation references
tests/integration/evpn/41-vxlan-ipv6-bridging.yml	Rename EVPN/VXLAN DUT switches to `dut_s1`/`dut_s2` and update links and validation references
tests/integration/evpn/20-vxlan-irb-ospf.yml	Rename EVPN/VXLAN DUT switches to `dut_s1`/`dut_s2` and update VLAN links and nodes list
tests/integration/evpn/15-vxlan-ebgp-unnumbered.yml	Rename DUT nodes to `dut_l1`/`dut_spine` and update EBGP validation references
tests/integration/evpn/14-vxlan-ebgp-ebgp.yml	Rename DUT nodes to `dut_l1`/`dut_spine` and update multihop sessions and validation references
tests/integration/evpn/12-vxlan-ibgp-ebgp.yml	Rename DUT nodes to `dut_l1`/`dut_spine` and update links and validation references
tests/integration/evpn/04-vxlan-central-routing.yml	Rename DUT leaf/spine nodes to `dut_l1`/`dut_l2`/`dut_spine` and update validation references
tests/integration/evpn.multihoming/02-esi-lag-auto-id.yml	Rename DUT switches to `dut_s1`/`dut_s2` and update ESI-LAG links
tests/integration/evpn.multihoming/01-esi-lag.yml	Rename DUT switches to `dut_s1`/`dut_s2` and update ESI-LAG links
tests/integration/eigrp/02-eigrp-ipv6.yml	Rename one router DUT from `r1` to `dut` and update links/message
tests/integration/eigrp/01-eigrp-ipv4.yml	Rename one router DUT from `r1` to `dut` and update links/message
tests/integration/dhcp/32-ipv6-client-ospf.yml	Rename DHCP client DUT nodes to `dut_c1`/`dut_c2` and update link endpoints
tests/integration/dhcp/31-ipv4-client-ospf.yml	Rename DHCP client DUT nodes to `dut_c1`/`dut_c2` and update link endpoints
tests/integration/dhcp/21-ipv4-relay-vrf-global.yml	Rename DHCP relay DUT node from `relay` to `dut` and update links
tests/integration/dhcp/12-ipv6-relay.yml	Rename DHCP relay DUT node from `relay` to `dut` and update links
tests/integration/dhcp/11-ipv4-relay.yml	Rename DHCP relay DUT node from `relay` to `dut` and update links
tests/integration/dhcp/04-ipv6-server.yml	Rename DHCP server DUT node from `server` to `dut` and update link endpoints
tests/integration/dhcp/03-ipv4-server.yml	Rename DHCP server DUT node from `server` to `dut` and update link endpoints
tests/integration/dhcp/02-ipv6-client.yml	Rename DHCP client DUT nodes to `dut_c1`/`dut_c2` and update node/link endpoints
tests/integration/dhcp/01-ipv4-client.yml	Rename DHCP client DUT nodes to `dut_c1`/`dut_c2` and update node/link endpoints

snuffy22 · 2026-06-30T09:40:10Z

@ipspace is there a way to run these 6 integration tests locally ?

ipspace · 2026-06-30T10:00:37Z

@ipspace is there a way to run these 6 integration tests locally ?

I have an empty directory with symlinks to various integration test directories. Running a single integration test is thus as easy as:

$ export NETLAB_DEVICE=device
$ export NETLAB_PROVIDER=provider
$ netlab up path/to/test --validate

Obviously, you can also use -p and -d flags.

To run a series of tests for a platform, see https://netlab.tools/dev/integration-tests/#running-an-integration-test-suite

snuffy22 · 2026-06-30T10:48:26Z

yep , i'm aware of that.. I have made use of the device_module_test as well
I found what I was after.. directly under tests is the 'run-*` scripts to check for yamllint etc
Really need to check my eyes...

snuffy22 · 2026-06-30T14:20:47Z

I have run all the following tests with FRR as the DUT.

(py3-snuff) netlab@netlab:~/snuffy-netlab/tests/integration$ ./device-module-test initial

Pre-test cleanup: cleanup(ok)
09:03:22 initial/01-interfaces.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:03:32 initial/01a-scripts.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:03:48 initial/01b-startup-config.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:03:48 initial/02-loopback.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:03:55 initial/03-unnumbered.yml: create(ok) up(ok) config(ok) validate(SKIPPED) cleanup(ok) OK
09:03:59 initial/04-mtu.yml: create(ok) up(FAIL) cleanup(ok)
09:04:00 initial/05-host.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:04:38 initial/06-bridge.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:04:49 initial/07-router.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:04:56 initial/08-ra.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:05:37 initial/10-reload.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK

(py3-snuff) netlab@netlab:~/snuffy-netlab/tests/integration$ ./device-module-test vlan

Pre-test cleanup:
09:09:45 vlan/01-vlan-bridge-single.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:09:50 vlan/02-vlan-bridge-multiple.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:10:00 vlan/21-vlan-irb-single.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:10:09 vlan/22-vlan-irb-multiple.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:10:16 vlan/23-vlan-mixed-multiple.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:10:27 vlan/31-vlan-bridge-trunk.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:11:39 vlan/32-vlan-bridge-trunk-router.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:11:44 vlan/33-vlan-irb-trunk.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:12:47 vlan/41-vlan-bridge-native.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:14:02 vlan/42-vlan-irb-native.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:15:05 vlan/51-vlan-routed-trunk.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:16:01 vlan/52-vlan-vrf-lite.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:16:45 vlan/61-vlan-routed-native.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:17:41 vlan/62-vlan-mixed-trunk.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:17:58 vlan/63-vlan-mixed-native.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:18:15 vlan/70-vlan-1-trunk.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK

(py3-snuff) netlab@netlab:~/snuffy-netlab/tests/integration$ ./device-module-test vxlan

Pre-test cleanup:
09:19:05 vxlan/01-vxlan-bridging.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:19:21 vxlan/02-vxlan-bridging-multinode.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:19:37 vxlan/03-vxlan-irb.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:19:51 vxlan/04-vxlan-irb-ospf.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:20:48 vxlan/05-vxlan-router-stick.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:21:10 vxlan/07-vxlan-bridging-v6only.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:21:33 vxlan/08-vxlan-alt-vtep.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK

(py3-snuff) netlab@netlab:~/snuffy-netlab/tests/integration$ ./device-module-test evpn

Pre-test cleanup:
09:22:39 evpn/01-vxlan-bridging.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:23:06 evpn/02-vxlan-asymmetric-irb.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:23:17 evpn/03-vxlan-symmetric-irb.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:23:40 evpn/04-vxlan-central-routing.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:24:02 evpn/05-vxlan-l3only.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:24:14 evpn/06-vxlan-bridging-vlan-bundle.yml: create(FAIL)
09:24:14 evpn/10-vxlan-rr.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:25:12 evpn/11-vxlan-ebgp.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:25:34 evpn/12-vxlan-ibgp-ebgp.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:25:42 evpn/13-vxlan-ebgp-allowas.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:25:50 evpn/14-vxlan-ebgp-ebgp.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:26:00 evpn/15-vxlan-ebgp-unnumbered.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:26:08 evpn/20-vxlan-irb-ospf.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:27:01 evpn/21-bgp-ce-router.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:27:22 evpn/22-ospf-ce-router.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:27:44 evpn/30-cs-bridging.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:28:05 evpn/41-vxlan-ipv6-bridging.yml: create(ok) up(ok) config(ok) validate(ok) cleanup(ok) OK
09:29:00 evpn/51-mpls-bridging.yml: create(FAIL)
09:29:00 evpn/52-mpls-asymmetric-irb.yml: create(FAIL)
09:29:01 evpn/53-evpn-l3only.yml: create(FAIL)
09:29:01 evpn/54-evpn-l3vpn.yml: create(FAIL)
09:29:01 evpn/61-sr-bridging.yml: create(FAIL)
(py3-snuff) netlab@netlab:~/snuffy-netlab/tests/integration$

ipspace · 2026-06-30T14:52:09Z

I have run all the following tests with FRR as the DUT.

Thank you. Just let me know when you're done with the changes, and I'll run the rest of the tests and merge it.

Copilot

Pull request overview

Copilot reviewed 45 out of 46 changed files in this pull request and generated 3 comments.

snuffy22 · 2026-07-01T10:45:09Z

+- dut_c1:
    ipv4: dhcp
-  c2:
+  dut_c2:
    ipv4: dhcp
    dhcp.client.default: False
  server:


Copilot

Pull request overview

Copilot reviewed 45 out of 46 changed files in this pull request and generated 3 comments.

ipspace

The saga continues :( But hey, we're almost there ;))

ipspace · 2026-07-01T16:43:57Z

      eos: "dhcp server leases | json"
    exec:
-      dnsmasq: "cat /var/lib/misc/dnsmasq.leases | grep --color=never -E ' c[1-9] ' | wc -l"
+      dnsmasq: "cat /var/lib/misc/dnsmasq.leases | grep --color=never -E ' dutc[1-9] ' | wc -l"


Missing underscore (dut_c) when testing with Cisco IOL. It looks like you were doing the tests with Arista EOS (or something similar) that does not include underscore in DHCP client name.

As much as it makes my eyes hurt, the only safe way to resolve this is to rename nodes to "dutc1" and "dutc2". The same applies to all other DHCP tests where we need to check the leases on the server.

ipspace · 2026-07-01T16:49:24Z

      eos: "dhcp server leases | json"
    exec:
-      dnsmasq: "cat /var/lib/misc/dnsmasq.leases | grep --color=never -E ' c[1-9] ' | wc -l"
+      dnsmasq: "cat /var/lib/misc/dnsmasq.leases | grep --color=never -E ' dutc[1-9] ' | wc -l"


As in the other DHCP test, use node names without underscores.

ipspace · 2026-07-01T16:54:00Z

    bgp.rr: True
  x_switches:
-    members: [ l2 ]
+    members: [ dut_l2 ]


Wrong. This is not device under test.

ipspace · 2026-07-01T17:04:04Z

    module: [ vlan ]
  probes:
-    members: [ s2 ]
+    members: [ dut_s2 ]


This is not a device under test, it's a probe

ipspace · 2026-07-01T17:07:32Z

-  adj_v2_r1_b:
-    description: Check OSPFv2 adjacencies (R2-R1)
-    wait_msg: Waiting for OSPFv2 adjacencies to form
+  adj_dut_a_b:


Dropped the 'v2' from test name (indicating it's OSPFv2 test)

ipspace · 2026-07-01T17:07:48Z

-    description: Check OSPFv2 adjacencies (R2-R1)
-    wait_msg: Waiting for OSPFv2 adjacencies to form
+    plugin: ospf_neighbor(nodes.dut_a.vrfs.blue.ospf.router_id,vrf='blue')
+  adj_dut_a_r:


As above -- v2 is missing in test name

ipspace · 2026-07-01T17:07:58Z

-    description: Check OSPFv2 adjacencies (R2-R3)
-    wait_msg: Waiting for OSPFv2 adjacencies to form
+    plugin: ospf_neighbor(nodes.dut_a.vrfs.red.ospf.router_id,vrf='red')
+  adj_dut_b_b:


As above...

ipspace

Ran (hopefully ;) all changed integration tests, they all pass.

Thanks a million, great job!

snuffy22 mentioned this pull request Jun 29, 2026

Standardize the 'dut' convention in all integration tests #3535

Closed

snuffy22 force-pushed the dut-standardise branch from d67f06d to 55ee68e Compare June 29, 2026 00:39

ipspace requested a review from Copilot June 29, 2026 14:33

Copilot started reviewing on behalf of ipspace June 29, 2026 14:34 View session

Copilot AI reviewed Jun 29, 2026

View reviewed changes

Fix: PE devices in EVPN integration tests must be routers

2a26577

snuffy22 added 15 commits June 30, 2026 22:21

Standardise on DUT for device being tested pt1

bc56afc

Standardize with dut or dut_ for vxlan integration tests

bb5e2be

Standardize with dut or dut_ for eigrp integration tests

40fe0b0

Standardize with dut or dut_ for lag integration tests

ac72438

Standardize with dut or dut_ for evpn integration tests

a4551ac

Standardize with dut or dut_ for dhcp integration tests

51081d2

Standardize with dut or dut_ for vlan integration tests

9bc8247

Standardize with dut or dut_ for evpn.multihoming integration tests

0124128

Standardize with dut or dut_ for lag integration tests missed one

25f5008

Standardize with dut or dut_ for vlan integration tests

bf9fc90

Another typo..

913608f

More typo fixes

23df654

Fix more issues with search/replace

e7a027d

Fix some leftover v2 to v3 items in new osfpv3 bfd test

76fa4fe

Fix typos

0b705d4

snuffy22 force-pushed the dut-standardise branch from bfdd27b to 0b705d4 Compare June 30, 2026 12:35

Merge branch 'dev' into dut-standardise

a14a135

ipspace requested a review from Copilot June 30, 2026 14:52

Copilot started reviewing on behalf of ipspace June 30, 2026 14:52 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Fix last issues found by Co-Pilot in DHCP

517b35c

ipspace requested a review from Copilot July 1, 2026 16:39

Copilot started reviewing on behalf of ipspace July 1, 2026 16:40 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread tests/integration/dhcp/01-ipv4-client.yml

Comment thread tests/integration/dhcp/31-ipv4-client-ospf.yml

Comment thread tests/integration/dhcp/31-ipv4-client-ospf.yml

ipspace requested changes Jul 1, 2026

View reviewed changes

ipspace added a commit that referenced this pull request Jul 1, 2026

Sample EIGRP, initial, LAG, OSPFv3, VXLAN tests for #3536

2577fef

snuffy22 added 2 commits July 2, 2026 08:01

Fix last few errors brought up by Ivan and file/plugin rename

ccfd697

Missed one again..

8cf7ed6

ipspace added a commit that referenced this pull request Jul 2, 2026

DHCP, VLAN, VXLAN tests for #3536

dcfd1f2

ipspace marked this pull request as ready for review July 2, 2026 17:58

ipspace approved these changes Jul 2, 2026

View reviewed changes

ipspace merged commit f71f28b into ipspace:dev Jul 2, 2026
5 checks passed

Conversation

snuffy22 commented Jun 28, 2026

Uh oh!

snuffy22 commented Jun 29, 2026

Uh oh!

ipspace commented Jun 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

snuffy22 commented Jun 30, 2026

Uh oh!

ipspace commented Jun 30, 2026

Uh oh!

snuffy22 commented Jun 30, 2026

Uh oh!

snuffy22 commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ipspace commented Jun 30, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

snuffy22 commented Jun 30, 2026 •

edited

Loading