Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new test plans for VNET to VNET testing #297

Merged
merged 12 commits into from
Jan 19, 2023
1 change: 1 addition & 0 deletions test/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
| [DASH P4 SAI-Thrift Test Workflow](dash-test-workflow-p4-saithrift.md) | Use of P4-based simulators or SW data planes to verify DASH behavior, using saithrift API. |
| [Testbed](testbed/README.md) | Describes the setup and configuration of a DASH testbed.|
| [snappi and SAI-Challenger based tests](dash-test-sai-challenger.md) | How to run scalable tests using SAI-Challenger and snappi. The scalability is achieved with additional DASH/SAI abstraction level in test code to simplify high scale DUT configuration. |
| [Testplans](testplans/README.md) | Home of DASH features test plans including test plan template |


You can start with the [High-Level Description (HLD) Test Specification](dash-test-HLD.md).
11 changes: 11 additions & 0 deletions test/docs/testplans/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# DASH Test Plans

In this document you will see a summary of the test plans available for DASH testing.

A test plan template is available [here](https://github.com/sonic-net/SONiC/blob/master/doc/SONiC%20Test%20Plan%20Template.md) for reference.


| Test Plan | Description |
|----------|-------------|
| [ACL](acl.md) | Access control list test plan|
| [VNET to VNET](vnet_to_vnet.md) | Describes test plan for testing the mechanisms of VM to VM communication in VNET|
58 changes: 58 additions & 0 deletions test/docs/testplans/vnet_to_vnet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# **VNET to VNET**

## Overview

This test plan corresponds to the phase where standardized, automated test cases
will be scripted to verify Data plane Testing Stage 3 as outlined in [dash test maturity stages](../dash-test-maturity-stages.md#data-plane-testing-stage-3-dut-configuration-via-sai-thrift).

In these tests we want to leverage the snappi API to test some of the functional aspects of DUT.
By verifying and validating the scaling aspects of the switches (hardware/BMv2)
we are able to empirically baseline the scaling values. Additionally, we also
want to check some of the relevant counters to features like IPv4, IPv6, TCP,
UDP, common traffic stats etc.

### Scope

These test cases are targeted for testing the feature functionality of DASH implementation. The purpose of these test cases are to verify inbound/outbound/bidirectional traffic streams originating and terminating at the traffic generator. We plan to use the [OTG](https://github.com/open-traffic-generator) (Open traffic generator model) by leveraging the snappi APIs.

### Testbed

The generic testbed setup may consist of two OTG connected to two ports of a switch (hardware-based or BMv2 software switch).
In this specific testbed setup we will use [Ixia-c](https://github.com/open-traffic-generator/ixia-c) which is a modern, powerful, and API-driven open traffic generator model-based traffic tool that will generate traffic to observe the behavior of the device under test (DUT).

### Testbed Topology

The testbed is setup to emulate a VNET to VNET scenario and to test the mechanisms of VM to VM communication in VNET as outlined [here](https://github.com/Azure/DASH/blob/main/documentation/vnet2vnet-service/vnet-to-vnet-service.md).



To emulate each VM in a VNET an Ixia-c port will be utilized. The BMv2 will be the DUT (This can later be replaced by a hardware-based appliance that connects to physical ports of a traffic generator)

![ixia-c.drawio](../../images/ixia-c.drawio.svg)
murthyvijay marked this conversation as resolved.
Show resolved Hide resolved



### Test Methodology

Following test methodologies will be utilized to verify and validate the behavior of the switch

- Traffic generator will be used to configure traffic streams that originate and/or terminate on traffic generator ports
- Data traffic will be sent from VNET to VNET to verify performance properties like: CPS, flow, PPS, and rule scale. These performance properties will be verified while features like: VM to VM communication in VNET, route support, LPM support, ACL support are enabled.
- Depending on the test cases, a combination of UDP traffic for IPv4, IPv6 addresses will be generated

### Test Cases
Note: No counters exist to date. Once counters are defined, the test-cases will be refined to incorporate them.

| Test Description | Expected Result |
| :----------------------------------------------------------- | :------------------------ |
| UDP IPv4 inbound | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv4 outbound | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv4 bidirectional | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv6 inbound | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv6 outbound direction | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| UDP IPv6 bidirectional | Verify traffic originating from one VNET simulated terminates on another simulated VNET. <br><br> Verify counters in DUT in the inbound direction are accurately incremented. <br><br> Verify latency measurements <br> Verify packet loss is 0 (Tx frames - Rx frames = 0) <br><br> Verify configured throughput rate is achieved in traffic statistics |
| ACL inbound | Verify data traffic is allowed and unwanted traffic based on ACL is dropped. <br><br>Verify ACL inbound counters on DUT are incremented
murthyvijay marked this conversation as resolved.
Show resolved Hide resolved
| ACL outbound | Verify data traffic is allowed and unwanted traffic based on ACL is dropped. <br><br>Verify ACL outbound counters on DUT are incremented
| BGP underlay | Verify BGP neighborship is established. <br><br>Verify BGP counters on DUT are incremented