Skip to content

Commit

Permalink
Added new test plans for VNET to VNET testing (#297)
Browse files Browse the repository at this point in the history
* Added new test plans for VNET to VNET testing

* Incorporated changes recommended

* Updated link to SONiC test plan template

The recommendation was to de-duplicate the SONiC test plan template and referenced it to its original placeholder

* Deleting the SONiC test plan template

Removing the test plan template as the link to it is provided in the README file

* Linked to the original VNET-to-VNET-service doc

Removed the diagram and referenced to the original VNET service document under DASH documentation

* Removed vm-to-vm-communication-vnet.svg

Since the original document is reference, no need for the VM to VM communication svg file here

* Fixed the VNET service link

rectified the incorrect VNET service link to reflect the correct one

* Fixed typos

Rectified some typos in the document

* Fixed the image in Test plan document
  • Loading branch information
murthyvijay committed Jan 19, 2023
1 parent aff7eae commit 9d9c229
Show file tree
Hide file tree
Showing 4 changed files with 78 additions and 0 deletions.
3 changes: 3 additions & 0 deletions test/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,6 @@
| [SAI Challenger DASH Schema](README-SAIC-DASH-config-spec.md) | High-level schema to generate scaled configurations (e.g. via [dpugen](https://pypi.org/project/dpugen/)) as well as the DASH "SAI Record" format which comprises SAI "CRUD" operations.
| [SAI Challenger Tutorials](../test-cases/functional/saic/tutorial/README.md) | Guided tour of SAI Challenger for DASH
| [SAI Challenger Test Workflows](dash-test-sai-challenger.md) | How to run scalable tests using SAI-Challenger and snappi. |
| [Testbed](testbed/README.md) | Describes the setup and configuration of a DASH testbed.|
| [snappi and SAI-Challenger based tests](dash-test-sai-challenger.md) | How to run scalable tests using SAI-Challenger and snappi. The scalability is achieved with additional DASH/SAI abstraction level in test code to simplify high scale DUT configuration. |
| [Test Plans](testplans/README.md) | Home of DASH features test plans including test plan template |
11 changes: 11 additions & 0 deletions test/docs/testplans/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# DASH Test Plans

In this document you will see a summary of the test plans available for DASH testing.

A test plan template is available [here](https://github.com/sonic-net/SONiC/blob/master/doc/SONiC%20Test%20Plan%20Template.md) for reference.


| Test Plan | Description |
|----------|-------------|
| [ACL](acl.md) | Access control list test plan|
| [VNET to VNET](vnet_to_vnet.md) | Describes test plan for testing the mechanisms of VM to VM communication in VNET|
60 changes: 60 additions & 0 deletions test/docs/testplans/vnet_to_vnet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# **VNET to VNET**

## Overview

This test plan corresponds to the phase where standardized, automated test cases
will be scripted to verify Data plane Testing Stage 3 as outlined in [dash test maturity stages](../dash-test-maturity-stages.md#data-plane-testing-stage-3-dut-configuration-via-sai-thrift).

In these tests we want to leverage the snappi API to test some of the functional aspects of DUT.
By verifying and validating the scaling aspects of the switches (hardware/BMv2)
we are able to empirically baseline the scaling values. Additionally, we also
want to check some of the relevant counters to features like IPv4, IPv6, TCP,
UDP, common traffic stats etc.

### Scope

These test cases are targeted for testing the feature functionality of DASH implementation. The purpose of these test cases are to verify inbound/outbound/bidirectional traffic streams originating and terminating at the traffic generator. We plan to use the [OTG](https://github.com/open-traffic-generator) (Open traffic generator model) by leveraging the snappi APIs.

### Testbed

The generic testbed setup may consist of two OTG connected to two ports of a switch (hardware-based or BMv2 software switch).
In this specific testbed setup we will use [Ixia-c](https://github.com/open-traffic-generator/ixia-c) which is a modern, powerful, and API-driven open traffic generator model-based traffic tool that will generate traffic to observe the behavior of the device under test (DUT).

### Testbed Topology

The testbed is setup to emulate a VNET to VNET scenario and to test the mechanisms of VM to VM communication in VNET as outlined [here](https://github.com/Azure/DASH/blob/main/documentation/vnet2vnet-service/vnet-to-vnet-service.md).



To emulate each VM in a VNET an Ixia-c port will be utilized. The BMv2 will be the DUT (This can later be replaced by a hardware-based appliance that connects to physical ports of a traffic generator)

![ixia-c.DUT](../../images/ixia-c.dut.svg)

In the above diagram:
- Ixia-C can be replaced by a hardware-based OTG
- BMv2 switch can be replaced with another software-based or a hardware-based switch

### Test Methodology

Following test methodologies will be utilized to verify and validate the behavior of the switch

- Traffic generator will be used to configure traffic streams that originate and/or terminate on traffic generator ports
- Data traffic will be sent from VNET to VNET to verify performance properties like: CPS, flow, PPS, and rule scale. These performance properties will be verified while features like: VM to VM communication in VNET, route support, LPM support, ACL support are enabled.
- Depending on the test cases, a combination of UDP traffic for IPv4, IPv6 addresses will be generated

### Test Cases
Note: No counters exist to date. Once counters are defined, the test-cases will be refined to incorporate them.

| Test Description | Expected Result |
| :----------------------------------------------------------- | :------------------------ |
| UDP IPv4 inbound | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv4 outbound | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv4 bidirectional | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv6 inbound | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv6 outbound direction | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv6 bidirectional | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| ACL inbound | Verify data traffic is allowed and unwanted traffic based on ACL is dropped. <br><br>Verify ACL inbound counters on DUT are incremented
| ACL outbound | Verify data traffic is allowed and unwanted traffic based on ACL is dropped. <br><br>Verify ACL outbound counters on DUT are incremented
| BGP underlay | Verify BGP adjacency is established. <br><br>Verify BGP counters on DUT are incremented


0 comments on commit 9d9c229

Please sign in to comment.