New e2e tests #343

LionelJouin · 2023-01-09T08:27:32Z

Description

The configuration of the new tests has been done only for the operator environment. The tests for the Helm environment have been disabled.

E2E new max targets test and error handling improved
- Error handler for the e2e has been improved to catch all errors
- New e2e test for the max-targets field in the stream (Disabled for now due to TAPA bugs)
Add second conduit as e2e test init config
- Initial e2e configuration now requires 2 conduit in the same trench.
- Add new TAPA test to open a stream in a second conduit
Add new new-stream test
- new-stream tests the stream creation/deletion
Add second stream e2e test
- Configuration e2e test renamed to Vip
- New second stream e2e test for TAPA
Flow e2e tests
- new-flow test adding a flow
- flow-priority
nsm-vlan attractor e2e test
- new-attractor-nsm-vlan e2e test added.
Fix e2e destination port vars and fix helm env
- All destination port variables for the e2e tests are now renamed with simplier name
- Helm chart is now running with only supported tests
Dest port range and port nat e2e tests
- Flow test of the destination port range
- Conduit port nat test
e2e flow byte-match test
- Add e2e test for the byte-match flow property
Fix e2e test checking nfqlb in LBs
- It could happen error was checked when verifying nfqlb in the LBs while the stream was not even configured.

https://github.com/Nordix/Meridio/blob/e2e-extended/docs/test.md#list-of-tests

List of new tests

open-second-stream

STEP: Selecting the first target from the deployment with label app=target-a in namespace red @ 01/02/23 23:35:14.308
STEP: Opening stream stream-a-ii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-5kf52 in namespace red @ 01/02/23 23:35:14.311
STEP: Waiting the stream to be opened in pod target-a-7fd479c8f5-5kf52 using ./target-client watch @ 01/02/23 23:35:14.362
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-bspbz (red) to have 1 targets configured @ 01/02/23 23:35:14.906
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-jhzv7 (red) to have 1 targets configured @ 01/02/23 23:35:14.951
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4000 @ 01/02/23 23:35:15.041
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4000 @ 01/02/23 23:35:15.142
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4001 @ 01/02/23 23:35:15.274
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4001 @ 01/02/23 23:35:15.437
STEP: Closing stream stream-a-ii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-5kf52 in namespace red @ 01/02/23 23:35:15.618
STEP: Waiting the stream to be closed in pod target-a-7fd479c8f5-5kf52 using ./target-client watch @ 01/02/23 23:35:15.678
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-bspbz (red) to have 4 targets configured @ 01/02/23 23:35:16.203

new-stream

STEP: Selecting the first target from the deployment with label app=target-a in namespace red @ 01/02/23 23:31:19.777 
STEP: Configuring the new stream @ 01/02/23 23:31:19.782
STEP: Opening stream stream-a-iii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-5kf52 in namespace red @ 01/02/23 23:31:19.961 
STEP: Waiting the stream to be opened in pod target-a-7fd479c8f5-5kf52 using ./target-client watch @ 01/02/23 23:31:20.023
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-bspbz (red) to have 1 targets configured @ 01/02/23 23:31:20.575
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-jhzv7 (red) to have 1 targets configured @ 01/02/23 23:31:20.625
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4002 @ 01/02/23 23:31:20.652 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4002 @ 01/02/23 23:31:20.857
STEP: Reverting the configuration of the new stream @ 01/02/23 23:31:21 
STEP: Closing stream stream-a-iii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-5kf52 in namespace red @ 01/02/23 23:31:21.121 
STEP: Waiting the stream to be closed in pod target-a-7fd479c8f5-5kf52 using ./target-client watch @ 01/02/23 23:31:21.187
STEP: Waiting for nfqlb to have removed the stream configuration @ 01/02/23 23:31:21.713
STEP: Waiting for nfqlb to have removed the stream configuration @ 01/02/23 23:31:21.781

open-second-stream-second-conduit

STEP: Selecting the first target from the deployment with label app=target-a in namespace red @ 01/03/23 23:30:07.463 
STEP: Opening stream stream-a-iv (conduit: conduit-a-2, trench: trench-a) in target target-a-7fd479c8f5-7cf9g in namespace red @ 01/03/23 23:30:07.463
STEP: Waiting the stream to be opened in pod target-a-7fd479c8f5-7cf9g using ./target-client watch @ 01/03/23 23:30:07.488
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-2-55f5f4db88-dsn4w (red) to have 1 targets configured @ 01/03/23 23:30:10.59 
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-2-55f5f4db88-qhmdf (red) to have 1 targets configured @ 01/03/23 23:30:10.654
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4000 @ 01/03/23 23:30:10.694 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4000 @ 01/03/23 23:30:10.825
STEP: Sending tcp traffic from the TG trench-a (red) to 40.0.0.1:4000 @ 01/03/23 23:30:10.998
STEP: Sending tcp traffic from the TG trench-a (red) to [4000::1]:4000 @ 01/03/23 23:30:11.144
STEP: Closing stream stream-a-iv (conduit: conduit-a-2, trench: trench-a) in target target-a-7fd479c8f5-7cf9g in namespace red @ 01/03/23 23:30:12.273
STEP: Waiting the stream to be closed in pod target-a-7fd479c8f5-7cf9g using ./target-client watch @ 01/03/23 23:30:12.337
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-2-55f5f4db88-dsn4w (red) to have 4 targets configured @ 01/03/23 23:30:12.886
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-2-55f5f4db88-qhmdf (red) to have 4 targets configured @ 01/03/23 23:30:12.963

stream-max-targets (Disabled for now)

STEP: Selecting the first target from the deployment with label app=target-a in namespace red @ 01/11/23 13:21:16.796
STEP: Selecting the second target from the deployment with label app=target-a in namespace red @ 01/11/23 13:21:16.799 
STEP: Configuring the new stream with max-targets set to 1 @ 01/11/23 13:21:16.799 
STEP: Opening stream stream-a-iii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-5gjl7 in namespace red @ 01/11/23 13:21:16.91 
STEP: Waiting the stream to be opened in pod target-a-7fd479c8f5-5gjl7 using ./target-client watch @ 01/11/23 13:21:16.994
STEP: Opening stream stream-a-iii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-8mzz5 in namespace red @ 01/11/23 13:21:17.527
STEP: Waiting the stream to be unavailable in pod target-a-7fd479c8f5-8mzz5 using ./target-client watch @ 01/11/23 13:21:17.569
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-blbzn (red) to have 1 targets configured @ 01/11/23 13:21:18.094 
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-1-7c86c8db64-wgbtx (red) to have 1 targets configured @ 01/11/23 13:21:18.116 
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4002 @ 01/11/23 13:21:18.155
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4002 @ 01/11/23 13:21:18.262 
STEP: Reverting the configuration of the new stream @ 01/11/23 13:21:18.442
STEP: Closing stream stream-a-iii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-8mzz5 in namespace red @ 01/11/23 13:21:18.526
STEP: Waiting the stream to be closed in pod target-a-7fd479c8f5-8mzz5 using ./target-client watch @ 01/11/23 13:21:18.618
STEP: Closing stream stream-a-iii (conduit: conduit-a-1, trench: trench-a) in target target-a-7fd479c8f5-5gjl7 in namespace red @ 01/11/23 13:21:19.142
STEP: Waiting the stream to be closed in pod target-a-7fd479c8f5-5gjl7 using ./target-client watch @ 01/11/23 13:21:19.218 
STEP: Waiting for nfqlb to have removed the stream configuration @ 01/11/23 13:21:19.741 
STEP: Waiting for nfqlb to have removed the stream configuration @ 01/11/23 13:21:19.79

new-flow

STEP: Configuring the new flow @ 01/04/23 22:41:45.652
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4002 @ 01/04/23 22:41:45.895 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4002 @ 01/04/23 22:41:47.154
STEP: Reverting the configuration of the new flow @ 01/04/23 22:41:54.534

flow-priority

STEP: Configuring the new flow @ 01/04/23 22:41:54.588
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4001 @ 01/04/23 22:41:54.8 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4001 @ 01/04/23 22:41:57.928
STEP: Reverting the configuration of the new flow @ 01/04/23 22:41:58.214

new-attractor-nsm-vlan (Disabled for now)

STEP: Selecting the first target from the deployment with label app=target-a in namespace red @ 01/17/23 16:55:22.821
STEP: Configuring the new attractor @ 01/17/23 16:55:22.824
STEP: Opening stream stream-a-iii (conduit: conduit-a-3, trench: trench-a) in target target-a-7fd479c8f5-d8bhs in namespace red @ 01/17/23 16:55:45.317
STEP: Waiting the stream to be opened in pod target-a-7fd479c8f5-d8bhs using ./target-client watch @ 01/17/23 16:55:45.405
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-3-8569bd95c4-49qtf (red) to have 1 targets configured @ 01/17/23 16:55:48.512
STEP: Waiting for nfqlb in the stateless-lb-frontend-attractor-a-3-8569bd95c4-jg8td (red) to have 1 targets configured @ 01/17/23 16:55:48.575
STEP: Sending tcp traffic from the TG trench-a (red) to 60.0.0.150:4000 @ 01/17/23 16:55:48.596
STEP: Sending tcp traffic from the TG trench-a (red) to [6000::150]:4000 @ 01/17/23 16:55:48.836
STEP: Closing stream stream-a-iii (conduit: conduit-a-3, trench: trench-a) in target target-a-7fd479c8f5-d8bhs in namespace red @ 01/17/23 16:55:49.935
STEP: Waiting the stream to be closed in pod target-a-7fd479c8f5-d8bhs using ./target-client watch @ 01/17/23 16:55:49.984
STEP: Reverting the configuration of the new attractor @ 01/17/23 16:55:50.50

flow-destination-ports-range

STEP: Configuring the flow @ 01/05/23 20:54:39.689
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4000 @ 01/05/23 20:54:39.804 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4000 @ 01/05/23 20:54:41.005
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4001 @ 01/05/23 20:55:12.641 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4001 @ 01/05/23 20:55:12.849
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4002 @ 01/05/23 20:55:12.964 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4002 @ 01/05/23 20:55:13.104
STEP: Reverting the configuration of the flow @ 01/05/23 20:55:13.253

conduit-destination-port-nats

STEP: Configuring the Conduit @ 01/05/23 21:00:12.811 
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:80 @ 01/05/23 21:00:12.935
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:80 @ 01/05/23 21:00:14.233
STEP: Reverting the configuration of the Conduit @ 01/05/23 21:01:18.438

flow-byte-matches

STEP: Configuring the flow @ 01/05/23 22:09:42.274
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4000 @ 01/05/23 22:09:42.386 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4000 @ 01/05/23 22:09:43.457
STEP: Sending tcp traffic from the TG trench-a (red) to 20.0.0.1:4002 @ 01/05/23 22:09:44.528 
STEP: Sending tcp traffic from the TG trench-a (red) to [2000::1]:4002 @ 01/05/23 22:09:44.683
STEP: Reverting the configuration of the flow @ 01/05/23 22:09:44.793

Issue link

#318

Checklist

Purpose
- Bug fix
- New functionality
- Documentation
- Refactoring
- CI
Test
- Unit test
- E2E Test
- Tested manually
Introduce a breaking change
- Yes (description required)
- No
Introduce changes in the Operator
- Yes (description required)
- No

LionelJouin · 2023-01-10T16:00:26Z

Here is a job run with the new tests: https://jenkins.nordix.org/blue/organizations/jenkins/meridio-e2e-test-kind/detail/meridio-e2e-test-kind/9670/pipeline

LionelJouin · 2023-01-19T19:15:40Z

All tests are now passing with these 2 PRs (tested with seed 1402278817 and 2108022354): #346 and #348
I created a new branch with them merged: https://github.com/Nordix/Meridio/tree/e2e-extended-tapa-nsp-merged

LionelJouin · 2023-02-10T17:20:33Z

I rebase and solve 2 things (mconnect failed connection and delete-create-trench).
I tried with seed 1360673835, everything passed, but there was probably IPv6 issues causing failed connection

LionelJouin · 2023-02-20T15:22:36Z

I restarted the CI and started to track the failing runs here: #316 (comment)

uablrek · 2023-02-22T12:30:50Z

Sprouts warnings like

Warning: meridio.nordix.org/v1alpha1 Trench is deprecated; use meridio.nordix.org/v1 Trench

LionelJouin · 2023-02-22T12:34:42Z

Right, the tests that doesn't require new feature are still deploying v1alpha1. I am not sure if I should update them or not because at some point we should tests old Meridio + new TAPA.

- Configuration e2e test renamed to Vip - New second stream e2e test for TAPA

new-stream tests the stream creation/deletion

- Initial e2e configuration now requires 2 conduit in the same trench. - Add new TAPA test to open a stream in a second conduit

- Error handler for the e2e has been improved to catch all errors - New e2e test for the max-targets field in the stream (Disabled for now due to TAPA bugs)

- new-flow test adding a flow - flow-priority

new-attractor-nsm-vlan e2e test added.

- All destination port variables for the e2e tests are now renamed with simplier name - Helm chart is now running with only supported tests

- Flow test of the destination port range - Conduit port nat test

Add e2e test for the byte-match flow property

It could happen error was checked when verifying nfqlb in the LBs while the stream was not even configured.

- Fix flow name - Fix execution order of the reverse step

Add missing delete-create-trench script for ipv4 and ipv6 Add missing checks in delete-create-trench Add ignore-lost-connections parameter Increase the memory limit of the nsm registry Improve log collector

LionelJouin added kind/enhancement New feature or request size/L area/testing labels Jan 9, 2023

LionelJouin marked this pull request as ready for review January 10, 2023 13:54

LionelJouin requested review from uablrek and zolug January 10, 2023 13:54

This was referenced Jan 13, 2023

Fix TAPA concurrency issue #346

Merged

Hanging NSM interfaces inside target pods #344

Open

Fix NSP Target Registry on stream conduit switch #348

Merged

LionelJouin force-pushed the e2e-extended branch from 89fa44d to e9a10d5 Compare February 10, 2023 17:04

LionelJouin force-pushed the e2e-extended branch 5 times, most recently from 6436bca to 61e1ca4 Compare February 20, 2023 14:08

LionelJouin force-pushed the e2e-extended branch 2 times, most recently from 49cf1a3 to 6b318d6 Compare February 21, 2023 17:22

LionelJouin force-pushed the e2e-extended branch from 1ceffe6 to 850be4b Compare February 22, 2023 13:50

LionelJouin force-pushed the e2e-extended branch from 850be4b to 271c81d Compare March 6, 2023 08:25

LionelJouin added 5 commits March 7, 2023 11:43

Add second stream e2e test

4173f28

- Configuration e2e test renamed to Vip - New second stream e2e test for TAPA

Add new new-stream test

be9fffc

new-stream tests the stream creation/deletion

Add second conduit as e2e test init config

bcb1489

- Initial e2e configuration now requires 2 conduit in the same trench. - Add new TAPA test to open a stream in a second conduit

E2E new maxt targets test and error handling improved

35b534f

- Error handler for the e2e has been improved to catch all errors - New e2e test for the max-targets field in the stream (Disabled for now due to TAPA bugs)

Flow e2e tests

c0d3167

- new-flow test adding a flow - flow-priority

LionelJouin added 8 commits March 7, 2023 11:43

nsm-vlan attractor e2e test

31fdc5c

new-attractor-nsm-vlan e2e test added.

Fix e2e destination port vars and fix helm env

b1d0342

- All destination port variables for the e2e tests are now renamed with simplier name - Helm chart is now running with only supported tests

Dest port range and port nat e2e tests

6c60773

- Flow test of the destination port range - Conduit port nat test

e2e flow byte-match test

e49df88

Add e2e test for the byte-match flow property

Fix e2e test checking nfqlb in LBs

1dd8149

It could happen error was checked when verifying nfqlb in the LBs while the stream was not even configured.

Fix new-attractor-nsm-vlan e2e test

853cfb4

- Fix flow name - Fix execution order of the reverse step

Add e2e instructions for Kind with Operator

ccb7575

Fix delete-create-trench and check failed connection with mconnect

59da4bf

Add missing delete-create-trench script for ipv4 and ipv6 Add missing checks in delete-create-trench Add ignore-lost-connections parameter Increase the memory limit of the nsm registry Improve log collector

LionelJouin force-pushed the e2e-extended branch from 271c81d to 52fa4e6 Compare March 7, 2023 10:44

e2e tests: add delay for the configuration to be applied

59f1acd

LionelJouin force-pushed the e2e-extended branch from 52fa4e6 to 59f1acd Compare March 10, 2023 17:46

LionelJouin merged commit 1355f1d into master Mar 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New e2e tests #343

New e2e tests #343

LionelJouin commented Jan 9, 2023 •

edited

Loading

LionelJouin commented Jan 10, 2023 •

edited

Loading

LionelJouin commented Jan 19, 2023 •

edited

Loading

LionelJouin commented Feb 10, 2023

LionelJouin commented Feb 20, 2023

uablrek commented Feb 22, 2023

LionelJouin commented Feb 22, 2023

New e2e tests #343

New e2e tests #343

Conversation

LionelJouin commented Jan 9, 2023 • edited Loading

Description

List of new tests

open-second-stream

new-stream

open-second-stream-second-conduit

stream-max-targets (Disabled for now)

new-flow

flow-priority

new-attractor-nsm-vlan (Disabled for now)

flow-destination-ports-range

conduit-destination-port-nats

flow-byte-matches

Issue link

Checklist

LionelJouin commented Jan 10, 2023 • edited Loading

LionelJouin commented Jan 19, 2023 • edited Loading

LionelJouin commented Feb 10, 2023

LionelJouin commented Feb 20, 2023

uablrek commented Feb 22, 2023

LionelJouin commented Feb 22, 2023

LionelJouin commented Jan 9, 2023 •

edited

Loading

LionelJouin commented Jan 10, 2023 •

edited

Loading

LionelJouin commented Jan 19, 2023 •

edited

Loading