Skip to content

task 4 updated

Aditya-Sarna edited this page Oct 1, 2026 · 1 revision

4. Verify the BlueField-3 host and DPU setup

I checked the lab repository first, then verified the live Dell host (dh5) and the BlueField-3. I did not flash a BF-Bundle. Recaptured 2026-10-01. Setup is verified. NVIDIA’s doca_dma_copy sample was built on the DPU, not run. That extra copy test is not required to close this task.

OOB SSH host key for 172.22.222.3 changed (ED25519 SHA256:XTzDfcnjS/3Md+81CuFC5rRJdFKimaWwp8xJWBklooE). After ssh-keygen -R 172.22.222.3 the login is the same DPU: bf-bundle-3.5.0-89_26.07_ubuntu-24.04_64k_prod.

Check Status
Host DOCA and NVIDIA drivers Done 2026-10-01. doca_caps DOCA 3.5.0098; ofed_info -s OFED-internal-26.07-0.7.7. doca-all / doca-ofed 3.5.0-082000. Host Ubuntu 24.04.5, kernel 6.8.0-142-generic
Host NIC firmware Done 2026-10-01. ethtool -i enp13s0f0np0: mlx5_core 2607.0.100, firmware 32.50.1002 (MT_0000000884), bus 0000:0d:00.0. That is the host ConnectX-7 view, not DPU bfver
DPU DOCA / BF-Bundle Done 2026-10-01 over OOB. /etc/mlnx-release: bf-bundle-3.5.0-89_26.07_ubuntu-24.04_64k_prod. DPU doca-samples / doca-sdk-dma 3.5.0098-1
DPU bfver Done 2026-10-01 (sudo bfver). ATF 4.16.0-8, UEFI 4.16.0-48, BSP 4.16.0.14075
DMA sample path Present. Built 2026-10-01 on the DPU: /tmp/doca-apps-build/dma_copy/doca_dma_copy (-h prints DOCA 3.5.0098). Not run
Which versions belong together Live host 3.5 (doca-all) + BF-Bundle 3.5.0 is in policy (matched release, Level 2). Old 3.4 host + 2.6 DPU is gone
lspci / mst / mlx5 Done 2026-10-01. BF3 at 0d:00.0 / .1 / .2. mst: /dev/mst/mt41692_pciconf0 → enp13s0f0np0 / mlx5_0; .1 → enp13s0f1np1 / mlx5_1
Host-to-DPU management OOB ubuntu@172.22.222.3 works. rshim ping 2/2; rshim SSH ubuntu@192.168.100.2 → localhost.localdomain. Planned 172.22.3.5 unused
doca_dma_copy Built on DPU. Not run. Not required for this task

Host DOCA and NVIDIA driver versions

On root@dh5 (2026-10-01):

hostname
cat /etc/os-release
uname -r
ofed_info -s
doca_caps --version
dpkg -l | grep -iE 'doca-all|doca-ofed|doca-sdk-dma|doca-samples|rshim|mft|mlnx-ofed-kernel'
ethtool -i enp13s0f0np0
Item Live
Login ssh root@172.22.1.5, hostname dh5
Host OS Ubuntu 24.04.5, 6.8.0-142-generic x86_64
Management eno12399np0 172.22.1.5/16
doca_caps --version DOCA 3.5.0098
ofed_info -s OFED-internal-26.07-0.7.7
doca-all / doca-ofed 3.5.0-082000
doca-sdk-dma / doca-samples 3.5.0098-1
mlnx-ofed-kernel-dkms 2607.0.100-1
MFT 4.37.0-154
rshim package 2.8.5-1 (service active)
mlx5_core 2607.0.100
NIC firmware 32.50.1002 (MT_0000000884), 0000:0d:00.0

Earlier captures (stale): kernel 6.8.0-139, rshim 2.7.3, MFT 4.36, firmware 32.40.1000, mlx5_core 26.04-0.8.6.

DPU DOCA / BF-Bundle and firmware

Working login: ssh ubuntu@172.22.222.3. Password in 1Password. bfver must be sudo bfver.

hostname
cat /etc/os-release
uname -a
cat /etc/mlnx-release
sudo bfver
ip -br addr show oob_net0 tmfifo_net0
ip -br addr show enp3s0f0s0
ip -br addr show enp3s0f1s0
ls /opt/mellanox/doca/applications/dma_copy
Item Live (2026-10-01, OOB)
Hostname localhost.localdomain
OS Ubuntu 24.04.5, 6.8.0-1030-bluefield-64k #35-Ubuntu, aarch64
/etc/mlnx-release bf-bundle-3.5.0-89_26.07_ubuntu-24.04_64k_prod
ATF v2.2(release):4.16.0-8-gda930bca9
UEFI 4.16.0-48-g3e1cb60aa0
BSP 4.16.0.14075
oob_net0 172.22.222.3/16
tmfifo_net0 192.168.100.2/30
enp3s0f0s0 172.16.3.67/24
enp3s0f1s0 172.16.3.143/24
DMA sample Source present. Binary /tmp/doca-apps-build/dma_copy/doca_dma_copy. DPU PCI 03:00.0 / 03:00.1. Host PCI 0d:00.0

172.16.3.26 / 172.16.3.139 are stale. They do not ping from the VPN or from dh5. Live DPU data is 67 / 143.

Host-side data (same day on dh5): enp13s0f0np0 172.16.3.65/24, enp13s0f1np1 172.16.3.66/24.

Which DOCA-Host and BF-Bundle versions belong together

NVIDIA ships both packages per release. Install DOCA on the host before BF-Bundle on the card. This host has doca-all, so host vs BF-Bundle is Level 2.

Pair Policy
Live: DOCA-Host 3.5.0098 + BF-Bundle 3.5.0-89 Yes. Matched 3.5 / 26.07
Old: DOCA-Host 3.4 + DPU 2.6 No. No longer installed
Lab BF2 file bf-bundle-2.7.0-33_… No. Wrong generation and card

Host PCIe detection (lspci, mst, mlx5)

lspci | grep -i bluefield
mst start
mst status -v
cat /dev/rshim0/misc
PCI Function
0d:00.0 BlueField-3 integrated ConnectX-7
0d:00.1 BlueField-3 integrated ConnectX-7
0d:00.2 BlueField-3 SoC Management Interface

/dev/rshim0/misc: DEV_INFO BlueField-3(Rev 1), BF_MODE DPU mode, DEV_NAME pcie-0000:0d:00.2.

MST device PCI NET RDMA
/dev/mst/mt41692_pciconf0 0d:00.0 enp13s0f0np0 mlx5_0
/dev/mst/mt41692_pciconf0.1 0d:00.1 enp13s0f1np1 mlx5_1

mst warned mst_pciconf already loaded. Devices listed. 0d:00.2 is rshim, not a third mlx5 netdev.

Host-to-DPU management connectivity

# from VPN
ssh ubuntu@172.22.222.3
ssh root@172.22.222.2
# from dh5
ping -c 2 192.168.100.2
ssh ubuntu@192.168.100.2
Path 2026-10-01
OOB ICMP / SSH 172.22.222.3 up / login (ubuntu@localhost)
rshim ICMP 192.168.100.2 2/2 from dh5
rshim SSH works (hostname → localhost.localdomain)
BF3 BMC 172.22.222.2 ICMP 2/2; ssh root@172.22.222.2 → root@dpu-bmc
Planned 172.22.3.5 / 172.22.4.5 unused

doca_dma_copy

Not part of the written task title. I compiled the sample on the DPU (meson setup + ninja under /tmp/doca-apps-build). doca_dma_copy -h works (DOCA 3.5.0098). -p / -r / -f are the run flags. I did not start the two-sided copy.

Host meson of the full apps tree failed earlier on missing json-c until libjson-c-dev is installed. I did not need a host binary to close Task 4.

Status

Task 4 is done. Host DOCA 3.5.0098, DPU bf-bundle-3.5.0-89_26.07_ubuntu-24.04_64k_prod, bfver, PCIe / mst, OOB and rshim login are verified. Live pair is in policy. doca_dma_copy was built on the DPU .

Clone this wiki locally