Skip to content

Commit

Permalink
[dash]: Add DPU kvm topology guidance (#1676)
Browse files Browse the repository at this point in the history
  • Loading branch information
Pterosaur committed May 8, 2024
1 parent 389816a commit ada7fd0
Showing 1 changed file with 93 additions and 0 deletions.
93 changes: 93 additions & 0 deletions doc/dash/dash-sonic-kvm.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@
- [4 Dataflow](#4-dataflow)
- [4.1 Data plane](#41-data-plane)
- [4.2 Control plane](#42-control-plane)
- [5 KVM topology](#5-kvm-topology)
- [5.1 Single device mode testbed](#51-single-device-mode-testbed)
- [5.1.1 Create DPU testbed](#511-create-dpu-testbed)
- [5.1.2 Connect to the DPU](#512-connect-to-the-dpu)
- [5.1.3 gnmi client access](#513-gnmi-client-access)
- [5.2 DPU with VPP NPU testbed](#52-dpu-with-vpp-npu-testbed)

# 1 Motivation

Expand Down Expand Up @@ -87,3 +93,90 @@ graph TD

In the physical SmartSwitch, configuration is forwarded via GNMI in the NPU. So, in the virtual SONiC environment, the SWSS module is capable of receiving configuration from an external GNMI service through the management port, eth-midplane. However, in the single device mode, the GNMI service can also be run within the KVM and directly forward the configuration to the local SWSS module.

# 5 KVM topology

This section describes the steps to setup a virtual DPU testbed, please check a [pre-learning document](https://github.com/sonic-net/sonic-mgmt/blob/master/docs/testbed/README.testbed.VsSetup.md) to setup your environment and download the EOS container. Beginning the following steps, I assume you have got the cEOS image with the specific version(https://github.com/sonic-net/sonic-mgmt/blob/master/ansible/group_vars/all/ceos.yml) and sonic-mgmt containers.


- The normal SONiC KVM image without DPU dataplane can be downloaded from: https://sonic-build.azurewebsites.net/api/sonic/artifacts?branchName=master&platform=vs&target=target/sonic-vs.img.gz
- The DPU SONiC KVM image with dataplane will be released at the next stage

## 5.1 Single device mode testbed

## 5.1.1 Create DPU testbed

```shell

# Login to your sonic-mgmt container
ssh -i ~/.ssh/id_rsa_docker_sonic_mgmt zegan@172.17.0.2

# CD to the ansible directory
cd /data/sonic-mgmt/ansible

# Build the DPU testbed
./testbed-cli.sh -t vtestbed.yaml -m veos_vtb add-topo vms-kvm-dpu password.txt
./testbed-cli.sh -t vtestbed.yaml -m veos_vtb deploy-mg vms-kvm-dpu veos_vtb password.txt

```

If you are lucky, your testbed should pass all testcases via following commands

``` shell
# Login to your sonic-mgmt container
ssh -i ~/.ssh/id_rsa_docker_sonic_mgmt zegan@172.17.0.2

# CD to the tests directory
cd /data/sonic-mgmt/tests

# Run testcases
./run_tests.sh -u -n vms-kvm-dpu -d vlab-01 -m individual -e --skip_sanity -e --disable_loganalyzer -c dash/test_dash_vnet.py -f vtestbed.yaml -i ../ansible/veos_vtb -e "--skip_dataplane_checking"
```

## 5.1.2 Connect to the DPU

``` shell
# Login to the DPU, the default password is `password` and the default mgmt IP address is `10.250.0.101`
sshpass -p 'password' ssh -o TCPKeepAlive=yes -o ServerAliveInterval=30 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no admin@10.250.0.101

```

## 5.1.3 gnmi client access

Once dash test cases are completed, we can find required certificates in sonic-mgmt container.

```
# The certificates are in /tmp directory
user@mgmt:~$ ls -l /tmp/e55648be-5a9c-49da-bcf7-645b1a926fb6
total 40
-rw-r--r-- 1 user user 99 Apr 12 05:52 extfile.cnf
-rw------- 1 user user 1675 Apr 12 05:52 gnmiCA.key
-rw-r--r-- 1 user user 1131 Apr 12 05:52 gnmiCA.pem
-rw-r--r-- 1 user user 41 Apr 12 05:52 gnmiCA.srl
-rw-r--r-- 1 user user 1017 Apr 12 05:52 gnmiclient.crt
-rw-r--r-- 1 user user 907 Apr 12 05:52 gnmiclient.csr
-rw------- 1 user user 1679 Apr 12 05:52 gnmiclient.key
-rw-r--r-- 1 user user 1070 Apr 12 05:52 gnmiserver.crt
-rw-r--r-- 1 user user 907 Apr 12 05:52 gnmiserver.csr
-rw------- 1 user user 1675 Apr 12 05:52 gnmiserver.key
```

Please install gnmi_cli_py from https://github.com/lguohan/gnxi/tree/master/gnmi_cli_py.

Restart GNMI server with server certificates:

```
# Run below commands on DUT
docker exec gnmi supervisorctl stop gnmi-native
docker exec gnmi bash -c "/usr/bin/nohup /usr/sbin/telemetry -logtostderr --port 50052 --server_crt /etc/sonic/telemetry/gnmiserver.crt --server_key /etc/sonic/telemetry/gnmiserver.key --ca_crt /etc/sonic/telemetry/gnmiCA.pem -gnmi_native_write=true -v=10 >/root/gnmi.log 2>&1 &"
```

Follow below example to run GNMI client:

```
# /root/update1 and /root/update2 are protobuf value for DASH table
python2 /root/gnxi/gnmi_cli_py/py_gnmicli.py --timeout 30 -t 10.0.0.88 -p 50052 -xo sonic-db -rcert /root/gnmiCA.pem -pkey /root/gnmiclient.key -cchain /root/gnmiclient.crt -m set-update --xpath /APPL_DB/localhost/DASH_APPLIANCE_TABLE[key=123] /APPL_DB/localhost/DASH_VNET_TABLE[key=Vnet1] --value $/root/update1 $/root/update2
```

## 5.2 DPU with VPP NPU testbed

TBD

0 comments on commit ada7fd0

Please sign in to comment.