Skip to content
This repository has been archived by the owner on Oct 16, 2020. It is now read-only.

Network interfaces renamed after upgrade to 2079.3.0 #2578

Closed
gigatexal opened this issue Apr 25, 2019 · 8 comments
Closed

Network interfaces renamed after upgrade to 2079.3.0 #2578

gigatexal opened this issue Apr 25, 2019 · 8 comments
Labels
area/usability component/udev kind/regression

Comments

@gigatexal
Copy link

Issue Report

Bug

Container Linux Version

2079.3.0

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
...
BUG_REPORT_URL="https://issues.coreos.com"

Environment

What hardware/cloud provider/hypervisor is being used to run Container Linux?

Bare Metal K8s with Cillium for networking and Rook for storage

Expected Behavior

No changes per the release notes found here https://coreos.com/releases/#2079.3.0

Actual Behavior

network interface ids changed and broke things

Reproduction Steps

  1. Run version 2023.5.0
  2. Upgrade to 2079.3.0
  3. See issues.

Other Information

coreos/fedora-coreos-tracker#173
@bgilbert bgilbert mentioned this issue Apr 25, 2019
Closed
@bgilbert
Copy link
Contributor

Those release notes are just saying that there were no changes between 2079.2.0 and 2079.3.0. We'll use clearer wording in the future. For the full set of changes between 2023.5.0 and 2079.3.0, see the releases page. In this case, the systemd 241 update was probably the culprit.

In order to fix the regression, we'll need more information from you. What were the interface names before and after the upgrade? Could you post the output of lspci -v and udevadm info -e? Could you post any files in /etc/systemd/network/*?

I'll retitle this bug for clarity.

@bgilbert bgilbert changed the title CoreOS 2079.3.0 does have breaking changes not found in release notes Network interfaces renamed after upgrade to 2079.3.0 Apr 25, 2019
@gigatexal
Copy link
Author

We’ll try to get you that output. Thanks for clarifying too. I believe we already downgraded to the previous version so I’m not sure we have a node with the version with changes. I’ll see what I can do.

@trevex
Copy link

trevex commented Apr 26, 2019

This is the requested output for a compute node, where the renaming of the interfaces atleast results in a working node. For more complex nodes like network nodes, the BGP configuration, sysctl and iptables settings completely fall apart. All nodes have Mellanox NICs and in the case of the compute nodes the name was always enp59s0f0 and enp59s0f1 and now they are enp1s0f*. This is worse for other nodes with four addtional NICs.

/etc/systemd/network/09-blacklist.network

[Match]
Name=veth* tun* lxc* cilium* kube-ipvs* kube-bridge kube-dummy-if
[Link]
Unmanaged=yes

/etc/systemd/network/10-enp59s0f0.network

[Match]
Name=enp59s0f0
[Network]
Address=10.7.0.58/30
DNS=8.8.8.8
[Route]
Gateway=10.7.0.57

/etc/systemd/network/10-enp59s0f1.network

[Match]
Name=enp59s0f1
[Network]
Address=10.7.1.58/30
DNS=8.8.8.8
[Route]
Gateway=10.7.1.57

/etc/systemd/network/10-lo.network

[Match]
Name=lo
[Network]
Address=10.7.32.41/32
[Route]

lspci -v

00:00.0 Host bridge: Intel Corporation Device 2020 (rev 04)
	Subsystem: Super Micro Computer Inc Device 0962
	Flags: fast devsel
	Capabilities: [90] Express Root Port (Slot-), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [144] Vendor Specific Information: ID=0004 Rev=1 Len=03c <?>
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>

00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 35
	Memory at 383ffff2c000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 37
	Memory at 383ffff28000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 35
	Memory at 383ffff24000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 37
	Memory at 383ffff20000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 35
	Memory at 383ffff1c000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 37
	Memory at 383ffff18000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 35
	Memory at 383ffff14000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 37
	Memory at 383ffff10000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

00:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:05.2 System peripheral: Intel Corporation Device 2025 (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:05.4 PIC: Intel Corporation Device 2026 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2026
	Flags: bus master, fast devsel, latency 0
	Memory at 9d20a000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

00:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E Ubox Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E Ubox Registers
	Flags: fast devsel

00:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E Ubox Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:11.0 Unassigned class [ff00]: Intel Corporation Device a1ec (rev 09)
	Subsystem: Super Micro Computer Inc Device 0962
	Flags: bus master, fast devsel, latency 0
	Capabilities: [80] Power Management version 3

00:11.1 Unassigned class [ff00]: Intel Corporation Device a1ed (rev 09)
	Subsystem: Super Micro Computer Inc Device 0962
	Flags: bus master, fast devsel, latency 0
	Capabilities: [80] Power Management version 3

00:11.5 SATA controller: Intel Corporation Lewisburg SSATA Controller [AHCI mode] (rev 09) (prog-if 01 [AHCI 1.0])
	Subsystem: Super Micro Computer Inc Lewisburg SSATA Controller [AHCI mode]
	Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 69
	Memory at 9d206000 (32-bit, non-prefetchable) [size=8K]
	Memory at 9d209000 (32-bit, non-prefetchable) [size=256]
	I/O ports at 0800 [size=8]
	I/O ports at 0810 [size=4]
	I/O ports at 0820 [size=32]
	Memory at 9d180000 (32-bit, non-prefetchable) [size=512K]
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [70] Power Management version 3
	Capabilities: [a8] SATA HBA v1.0
	Capabilities: [b0] PCI Advanced Features
	Kernel driver in use: ahci
	Kernel modules: ahci

00:14.0 USB controller: Intel Corporation Lewisburg USB 3.0 xHCI Controller (rev 09) (prog-if 30 [XHCI])
	Subsystem: Super Micro Computer Inc Lewisburg USB 3.0 xHCI Controller
	Flags: bus master, medium devsel, latency 0, IRQ 68
	Memory at 383ffff00000 (64-bit, non-prefetchable) [size=64K]
	Capabilities: [70] Power Management version 2
	Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
	Kernel driver in use: xhci_hcd
	Kernel modules: xhci_pci

00:14.2 Signal processing controller: Intel Corporation Device a1b1 (rev 09)
	Subsystem: Super Micro Computer Inc Device 0962
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at 383ffff33000 (64-bit, non-prefetchable) [size=4K]
	Capabilities: [50] Power Management version 3
	Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit-

00:16.0 Communication controller: Intel Corporation Lewisburg CSME: HECI #1 (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg CSME: HECI
	Flags: bus master, fast devsel, latency 0, IRQ 218
	Memory at 383ffff32000 (64-bit, non-prefetchable) [size=4K]
	Capabilities: [50] Power Management version 3
	Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Kernel driver in use: mei_me
	Kernel modules: mei_me

00:16.1 Communication controller: Intel Corporation Lewisburg CSME: HECI #2 (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg CSME: HECI
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at 383ffff31000 (64-bit, non-prefetchable) [size=4K]
	Capabilities: [50] Power Management version 3
	Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+

00:16.4 Communication controller: Intel Corporation Lewisburg CSME: HECI #3 (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg CSME: HECI
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at 383ffff30000 (64-bit, non-prefetchable) [size=4K]
	Capabilities: [50] Power Management version 3
	Capabilities: [8c] MSI: Enable- Count=1/1 Maskable- 64bit+

00:17.0 SATA controller: Intel Corporation Lewisburg SATA Controller [AHCI mode] (rev 09) (prog-if 01 [AHCI 1.0])
	Subsystem: Super Micro Computer Inc Lewisburg SATA Controller [AHCI mode]
	Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 128
	Memory at 9d204000 (32-bit, non-prefetchable) [size=8K]
	Memory at 9d208000 (32-bit, non-prefetchable) [size=256]
	I/O ports at 0840 [size=8]
	I/O ports at 0850 [size=4]
	I/O ports at 0860 [size=32]
	Memory at 9d100000 (32-bit, non-prefetchable) [size=512K]
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [70] Power Management version 3
	Capabilities: [a8] SATA HBA v1.0
	Capabilities: [b0] PCI Advanced Features
	Kernel driver in use: ahci
	Kernel modules: ahci

00:1c.0 PCI bridge: Intel Corporation Lewisburg PCI Express Root Port #1 (rev f9) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 24
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00001000-00001fff
	Memory behind bridge: 90000000-901fffff
	Prefetchable memory behind bridge: 0000380000000000-00003800001fffff
	Capabilities: [40] Express Root Port (Slot+), MSI 00
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: Super Micro Computer Inc Lewisburg PCI Express Root Port
	Capabilities: [a0] Power Management version 3
	Kernel driver in use: pcieport

00:1c.5 PCI bridge: Intel Corporation Lewisburg PCI Express Root Port #6 (rev f9) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 25
	Bus: primary=00, secondary=02, subordinate=03, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: 9c000000-9d0fffff
	Capabilities: [40] Express Root Port (Slot+), MSI 00
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
	Capabilities: [90] Subsystem: Super Micro Computer Inc Lewisburg PCI Express Root Port
	Capabilities: [a0] Power Management version 3
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Access Control Services
	Capabilities: [220] #19
	Kernel driver in use: pcieport

00:1f.0 ISA bridge: Intel Corporation Lewisburg LPC Controller (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg LPC Controller
	Flags: bus master, fast devsel, latency 0

00:1f.2 Memory controller: Intel Corporation Lewisburg PMC (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg PMC
	Flags: bus master, fast devsel, latency 0
	Memory at 9d200000 (32-bit, non-prefetchable) [size=16K]

00:1f.4 SMBus: Intel Corporation Lewisburg SMBus (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg SMBus
	Flags: medium devsel, IRQ 16
	Memory at 380000200000 (64-bit, non-prefetchable) [size=256]
	I/O ports at 0780 [size=32]
	Kernel driver in use: i801_smbus
	Kernel modules: i2c_i801

00:1f.5 Serial bus controller [0c80]: Intel Corporation Lewisburg SPI Controller (rev 09)
	Subsystem: Super Micro Computer Inc Lewisburg SPI Controller
	Flags: fast devsel
	Memory at fe010000 (32-bit, non-prefetchable) [size=4K]

02:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 04) (prog-if 00 [Normal decode])
	Flags: bus master, fast devsel, latency 0, IRQ 10
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: 9c000000-9d0fffff
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Power Management version 3
	Capabilities: [80] Express PCI-Express to PCI/PCI-X Bridge, MSI 00
	Capabilities: [c0] Subsystem: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge
	Capabilities: [100] Virtual Channel
	Capabilities: [800] Advanced Error Reporting

03:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41) (prog-if 00 [VGA controller])
	DeviceName: ASPEED Video AST2500
	Subsystem: Super Micro Computer Inc ASPEED Graphics Family
	Flags: bus master, medium devsel, latency 0, IRQ 10
	Memory at 9c000000 (32-bit, non-prefetchable) [size=16M]
	Memory at 9d000000 (32-bit, non-prefetchable) [size=128K]
	I/O ports at 3000 [size=128]
	[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+

17:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port 1A (rev 04) (prog-if 00 [Normal decode])
	Physical Slot: 6
	Flags: bus master, fast devsel, latency 0, IRQ 27
	Bus: primary=17, secondary=18, subordinate=18, sec-latency=0
	I/O behind bridge: 00005000-00005fff
	Memory behind bridge: aae00000-aaefffff
	Prefetchable memory behind bridge: 0000384000000000-00003840000fffff
	Capabilities: [40] Subsystem: Super Micro Computer Inc Sky Lake-E PCI Express Root Port 1A
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [110] Access Control Services
	Capabilities: [148] Advanced Error Reporting
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>
	Kernel driver in use: pcieport

17:05.0 System peripheral: Intel Corporation Device 2034 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

17:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

17:05.4 PIC: Intel Corporation Device 2036 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2036
	Flags: bus master, fast devsel, latency 0
	Memory at aaf00000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

17:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

17:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

17:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

17:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

17:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

17:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

17:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

17:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

18:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
	Subsystem: Super Micro Computer Inc I350 Gigabit Network Connection
	Flags: bus master, fast devsel, latency 0, IRQ 57
	Memory at aae20000 (32-bit, non-prefetchable) [size=128K]
	I/O ports at 5020 [size=32]
	Memory at aae44000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
	Capabilities: [a0] Express Endpoint, MSI 00
	Capabilities: [e0] Vital Product Data
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number ac-1f-6b-ff-ff-b1-73-ce
	Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
	Capabilities: [1a0] Transaction Processing Hints
	Capabilities: [1c0] Latency Tolerance Reporting
	Capabilities: [1d0] Access Control Services
	Kernel driver in use: igb
	Kernel modules: igb

18:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
	Subsystem: Super Micro Computer Inc I350 Gigabit Network Connection
	Flags: bus master, fast devsel, latency 0, IRQ 70
	Memory at aae00000 (32-bit, non-prefetchable) [size=128K]
	I/O ports at 5000 [size=32]
	Memory at aae40000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
	Capabilities: [a0] Express Endpoint, MSI 00
	Capabilities: [e0] Vital Product Data
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [140] Device Serial Number ac-1f-6b-ff-ff-b1-73-ce
	Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
	Capabilities: [1a0] Transaction Processing Hints
	Capabilities: [1d0] Access Control Services
	Kernel driver in use: igb
	Kernel modules: igb

3a:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port 1A (rev 04) (prog-if 00 [Normal decode])
	Physical Slot: 1
	Flags: bus master, fast devsel, latency 0, IRQ 29
	Bus: primary=3a, secondary=3b, subordinate=3b, sec-latency=0
	Memory behind bridge: b8500000-b86fffff
	Prefetchable memory behind bridge: 000038bffc000000-000038bfffffffff
	Capabilities: [40] Subsystem: Super Micro Computer Inc Sky Lake-E PCI Express Root Port 1A
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [110] Access Control Services
	Capabilities: [148] Advanced Error Reporting
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>
	Kernel driver in use: pcieport

3a:05.0 System peripheral: Intel Corporation Device 2034 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:05.4 PIC: Intel Corporation Device 2036 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2036
	Flags: bus master, fast devsel, latency 0
	Memory at b8700000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

3a:08.0 System peripheral: Intel Corporation Device 2066 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:09.0 System peripheral: Intel Corporation Device 2066 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0a.0 System peripheral: Intel Corporation Device 2040 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0a.1 System peripheral: Intel Corporation Device 2041 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0a.2 System peripheral: Intel Corporation Device 2042 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0a.3 System peripheral: Intel Corporation Device 2043 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0a.4 System peripheral: Intel Corporation Device 2044 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0a.5 System peripheral: Intel Corporation Device 2045 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0a.6 System peripheral: Intel Corporation Device 2046 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0a.7 System peripheral: Intel Corporation Device 2047 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0b.0 System peripheral: Intel Corporation Device 2048 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0b.1 System peripheral: Intel Corporation Device 2049 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0b.2 System peripheral: Intel Corporation Device 204a (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0b.3 System peripheral: Intel Corporation Device 204b (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0c.0 System peripheral: Intel Corporation Device 2040 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0c.1 System peripheral: Intel Corporation Device 2041 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0c.2 System peripheral: Intel Corporation Device 2042 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0c.3 System peripheral: Intel Corporation Device 2043 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0c.4 System peripheral: Intel Corporation Device 2044 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0c.5 System peripheral: Intel Corporation Device 2045 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0c.6 System peripheral: Intel Corporation Device 2046 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0c.7 System peripheral: Intel Corporation Device 2047 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0d.0 System peripheral: Intel Corporation Device 2048 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0d.1 System peripheral: Intel Corporation Device 2049 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3a:0d.2 System peripheral: Intel Corporation Device 204a (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

3a:0d.3 System peripheral: Intel Corporation Device 204b (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

3b:00.0 Ethernet controller: Mellanox Technologies MT27640 Family
	Subsystem: Mellanox Technologies MT27640 Family
	Flags: bus master, fast devsel, latency 0, IRQ 67
	Memory at 38bffe000000 (64-bit, prefetchable) [size=32M]
	Expansion ROM at b8600000 [disabled] [size=1M]
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [48] Vital Product Data
	Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
	Capabilities: [c0] Vendor Specific Information: Len=18 <?>
	Capabilities: [40] Power Management version 3
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [180] Single Root I/O Virtualization (SR-IOV)
	Capabilities: [1c0] #19
	Capabilities: [230] Access Control Services
	Kernel driver in use: mlx5_core
	Kernel modules: mlx5_core

3b:00.1 Ethernet controller: Mellanox Technologies MT27640 Family
	Subsystem: Mellanox Technologies MT27640 Family
	Flags: bus master, fast devsel, latency 0, IRQ 173
	Memory at 38bffc000000 (64-bit, prefetchable) [size=32M]
	Expansion ROM at b8500000 [disabled] [size=1M]
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [48] Vital Product Data
	Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
	Capabilities: [c0] Vendor Specific Information: Len=18 <?>
	Capabilities: [40] Power Management version 3
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [180] Single Root I/O Virtualization (SR-IOV)
	Capabilities: [230] Access Control Services
	Kernel driver in use: mlx5_core
	Kernel modules: mlx5_core

5d:05.0 System peripheral: Intel Corporation Device 2034 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

5d:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

5d:05.4 PIC: Intel Corporation Device 2036 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2036
	Flags: bus master, fast devsel, latency 0
	Memory at c5f00000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

5d:0e.0 Performance counters: Intel Corporation Device 2058 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

5d:0e.1 System peripheral: Intel Corporation Device 2059 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=0 Len=0f0 <?>

5d:0f.0 Performance counters: Intel Corporation Device 2058 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

5d:0f.1 System peripheral: Intel Corporation Device 2059 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=0 Len=0f0 <?>

5d:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M3KTI Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

5d:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M3KTI Registers
	Flags: fast devsel
	Kernel driver in use: skx_uncore

5d:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M3KTI Registers
	Flags: fast devsel
	Kernel driver in use: skx_uncore

5d:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M2PCI Registers
	Flags: fast devsel

5d:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M2PCI Registers
	Flags: fast devsel

5d:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M2PCI Registers
	Flags: fast devsel

80:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 45
	Memory at 393ffff1c000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 47
	Memory at 393ffff18000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 45
	Memory at 393ffff14000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 47
	Memory at 393ffff10000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 45
	Memory at 393ffff0c000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 47
	Memory at 393ffff08000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 45
	Memory at 393ffff04000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CBDMA Registers
	Flags: bus master, fast devsel, latency 0, IRQ 47
	Memory at 393ffff00000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: ioatdma

80:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

80:05.2 System peripheral: Intel Corporation Device 2025 (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

80:05.4 PIC: Intel Corporation Device 2026 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2026
	Flags: bus master, fast devsel, latency 0
	Memory at d3700000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

80:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E Ubox Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

80:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E Ubox Registers
	Flags: fast devsel

80:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E Ubox Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

85:05.0 System peripheral: Intel Corporation Device 2034 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

85:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

85:05.4 PIC: Intel Corporation Device 2036 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2036
	Flags: bus master, fast devsel, latency 0
	Memory at e0f00000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

85:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E CHA Registers
	Flags: fast devsel

85:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

85:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

85:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

85:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

85:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

85:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

85:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E PCU Registers
	Flags: fast devsel

ae:05.0 System peripheral: Intel Corporation Device 2034 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:05.4 PIC: Intel Corporation Device 2036 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2036
	Flags: bus master, fast devsel, latency 0
	Memory at ee700000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

ae:08.0 System peripheral: Intel Corporation Device 2066 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:09.0 System peripheral: Intel Corporation Device 2066 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0a.0 System peripheral: Intel Corporation Device 2040 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0a.1 System peripheral: Intel Corporation Device 2041 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0a.2 System peripheral: Intel Corporation Device 2042 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0a.3 System peripheral: Intel Corporation Device 2043 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0a.4 System peripheral: Intel Corporation Device 2044 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0a.5 System peripheral: Intel Corporation Device 2045 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0a.6 System peripheral: Intel Corporation Device 2046 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0a.7 System peripheral: Intel Corporation Device 2047 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0b.0 System peripheral: Intel Corporation Device 2048 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0b.1 System peripheral: Intel Corporation Device 2049 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0b.2 System peripheral: Intel Corporation Device 204a (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0b.3 System peripheral: Intel Corporation Device 204b (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0c.0 System peripheral: Intel Corporation Device 2040 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0c.1 System peripheral: Intel Corporation Device 2041 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0c.2 System peripheral: Intel Corporation Device 2042 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0c.3 System peripheral: Intel Corporation Device 2043 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0c.4 System peripheral: Intel Corporation Device 2044 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0c.5 System peripheral: Intel Corporation Device 2045 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0c.6 System peripheral: Intel Corporation Device 2046 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0c.7 System peripheral: Intel Corporation Device 2047 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0d.0 System peripheral: Intel Corporation Device 2048 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0d.1 System peripheral: Intel Corporation Device 2049 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

ae:0d.2 System peripheral: Intel Corporation Device 204a (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

ae:0d.3 System peripheral: Intel Corporation Device 204b (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

d7:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port 1A (rev 04) (prog-if 00 [Normal decode])
	Physical Slot: 14
	Flags: bus master, fast devsel, latency 0, IRQ 31
	Bus: primary=d7, secondary=d8, subordinate=d8, sec-latency=0
	Memory behind bridge: fbe00000-fbefffff
	Prefetchable memory behind bridge: 000039c000000000-000039c0001fffff
	Capabilities: [40] Subsystem: Super Micro Computer Inc Sky Lake-E PCI Express Root Port 1A
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [110] Access Control Services
	Capabilities: [148] Advanced Error Reporting
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>
	Kernel driver in use: pcieport

d7:01.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port 1B (rev 04) (prog-if 00 [Normal decode])
	Physical Slot: 15
	Flags: bus master, fast devsel, latency 0, IRQ 32
	Bus: primary=d7, secondary=d9, subordinate=d9, sec-latency=0
	Memory behind bridge: ee800000-ee9fffff
	Prefetchable memory behind bridge: 000039c000200000-000039c0003fffff
	Capabilities: [40] Subsystem: Super Micro Computer Inc Sky Lake-E PCI Express Root Port 1B
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [110] Access Control Services
	Capabilities: [148] Advanced Error Reporting
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>
	Kernel driver in use: pcieport

d7:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port 1C (rev 04) (prog-if 00 [Normal decode])
	Physical Slot: 16
	Flags: bus master, fast devsel, latency 0, IRQ 33
	Bus: primary=d7, secondary=da, subordinate=da, sec-latency=0
	Memory behind bridge: eea00000-eebfffff
	Prefetchable memory behind bridge: 000039c000400000-000039c0005fffff
	Capabilities: [40] Subsystem: Super Micro Computer Inc Sky Lake-E PCI Express Root Port 1C
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [110] Access Control Services
	Capabilities: [148] Advanced Error Reporting
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>
	Kernel driver in use: pcieport

d7:03.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port 1D (rev 04) (prog-if 00 [Normal decode])
	Physical Slot: 17
	Flags: bus master, fast devsel, latency 0, IRQ 34
	Bus: primary=d7, secondary=db, subordinate=db, sec-latency=0
	I/O behind bridge: 0000f000-0000ffff
	Memory behind bridge: eec00000-eedfffff
	Prefetchable memory behind bridge: 000039c000600000-000039c0007fffff
	Capabilities: [40] Subsystem: Super Micro Computer Inc Sky Lake-E PCI Express Root Port 1D
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
	Capabilities: [90] Express Root Port (Slot+), MSI 00
	Capabilities: [e0] Power Management version 3
	Capabilities: [100] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Capabilities: [110] Access Control Services
	Capabilities: [148] Advanced Error Reporting
	Capabilities: [1d0] Vendor Specific Information: ID=0003 Rev=1 Len=00a <?>
	Capabilities: [250] #19
	Capabilities: [280] Vendor Specific Information: ID=0005 Rev=3 Len=018 <?>
	Capabilities: [298] Vendor Specific Information: ID=0007 Rev=0 Len=024 <?>
	Capabilities: [300] Vendor Specific Information: ID=0008 Rev=0 Len=038 <?>
	Kernel driver in use: pcieport

d7:05.0 System peripheral: Intel Corporation Device 2034 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

d7:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 04)
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

d7:05.4 PIC: Intel Corporation Device 2036 (rev 04) (prog-if 20 [IO(X)-APIC])
	Subsystem: Intel Corporation Device 2036
	Flags: bus master, fast devsel, latency 0
	Memory at fbf00000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [44] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3

d7:0e.0 Performance counters: Intel Corporation Device 2058 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

d7:0e.1 System peripheral: Intel Corporation Device 2059 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=0 Len=0f0 <?>

d7:0f.0 Performance counters: Intel Corporation Device 2058 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Kernel driver in use: skx_uncore

d7:0f.1 System peripheral: Intel Corporation Device 2059 (rev 04)
	Subsystem: Intel Corporation Device 0000
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=0 Len=0f0 <?>

d7:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M3KTI Registers
	Flags: fast devsel
	Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

d7:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M3KTI Registers
	Flags: fast devsel
	Kernel driver in use: skx_uncore

d7:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M3KTI Registers
	Flags: fast devsel
	Kernel driver in use: skx_uncore

d7:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M2PCI Registers
	Flags: fast devsel

d7:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M2PCI Registers
	Flags: fast devsel

d7:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 04)
	Subsystem: Intel Corporation Sky Lake-E M2PCI Registers
	Flags: fast devsel

d8:00.0 Non-Volatile memory controller: Intel Corporation Device 0a54 (prog-if 02 [NVM Express])
	Subsystem: Intel Corporation Device 4802
	Physical Slot: 14-1
	Flags: bus master, fast devsel, latency 0, IRQ 55
	Memory at fbe10000 (64-bit, non-prefetchable) [size=16K]
	Expansion ROM at fbe00000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 3
	Capabilities: [50] MSI-X: Enable+ Count=129 Masked-
	Capabilities: [60] Express Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [150] Virtual Channel
	Capabilities: [180] Power Budgeting <?>
	Capabilities: [190] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [270] Device Serial Number 55-cd-2e-41-4f-db-1f-46
	Capabilities: [2a0] #19
	Capabilities: [2d0] Latency Tolerance Reporting
	Capabilities: [310] L1 PM Substates
	Kernel driver in use: nvme
	Kernel modules: nvme

udevadm omitted for now output was ~13000 lines.

@lucab
Copy link

lucab commented Apr 26, 2019 •
edited
Loading

@gigatexal @trevex thanks for the follow up. udevadm info /sys/class/net/<$IFNAME> should produce a reasonably smaller output. The output of udevadm test /sys/class/net/<$IFNAME> would also help double-checking udev config.

The network card(s) at 3b:00.0 is indeed reasonable to be called enp59s0*, but I don't have specific knowledge about Mellanox card. (A gut feeling is that this may be related to its ARI capabilities)

While we track down what's going on, you may try experimenting and getting back the old behavior via https://www.freedesktop.org/software/systemd/man/systemd-udevd.service.html#net.naming-scheme=.

@trevex
Copy link

trevex commented Apr 26, 2019

I haven't gotten around setting net.naming-scheme yet, but in the mean time I was digging through the systemd news and there are several changes from 238 to 241 which are related:

CHANGES WITH 241:
...
        * A new network device NamePolicy "keep" is implemented for link files,
          and used by default in 99-default.link (the fallback configuration
          provided by systemd). With this policy, if the network device name
          was already set by userspace, the device will not be renamed again.
          This matches the naming scheme that was implemented before
          systemd-240. If naming-scheme < 240 is specified, the "keep" policy
          is also enabled by default, even if not specified. Effectively, this
          means that if naming-scheme >= 240 is specified, network devices will
          be renamed according to the configuration, even if they have been
          renamed already, if "keep" is not specified as the naming policy in
          the .link file. The 99-default.link file provided by systemd includes
          "keep" for backwards compatibility, but it is recommended for user
          installed .link files to *not* include it.

          The "kernel" policy, which keeps kernel names declared to be
          "persistent", now works again as documented.
...
CHANGES WITH 240:
...

        * Predictable interface naming has been extended to cover InfiniBand
NICs. They will be exposed with an "ib" prefix.
...
        * systemd-udevd's network naming logic now understands a new
          net.naming-scheme= kernel command line switch, which may be used to
          pick a specific version of the naming scheme. This helps stabilizing
          interface names even as systemd/udev are updated and the naming logic
is improved.
...
        * The naming scheme for network devices was changed to always rename
          devices, even if they were already renamed by userspace. The "kernel"
          policy was changed to only apply as a fallback, if no other naming
policy took effect.
...
CHANGES WITH 239:

        * NETWORK INTERFACE DEVICE NAMING CHANGES: systemd-udevd's "net_id"
          builtin will name network interfaces differently than in previous
          versions for virtual network interfaces created with SR-IOV and NPAR
          and for devices where the PCI network controller device does not have
a slot number associated.
...

Source: https://github.com/systemd/systemd/blob/master/NEWS

This would indeed lead me to believe that systemd is the "culprit".

@dm0- dm0- mentioned this issue Apr 30, 2019
Merged
@dm0-
Copy link

dm0- commented May 7, 2019

The naming scheme has been reverted in the alpha and beta channels, if you want to test it.

@trevex
Copy link

trevex commented May 8, 2019 •
edited
Loading

We will, thank you. I will come back with more information once we tested it.

@bgilbert
Copy link
Contributor

bgilbert commented Jun 5, 2019

This should be fixed in stable 2079.5.0, due shortly. Thanks for reporting.

@bgilbert bgilbert closed this as completed Jun 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area/usability component/udev kind/regression
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@lucab @bgilbert @gigatexal @trevex @dm0-