Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tests/mclag] Submit mclag testcases according to mclag test plan #1239

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

[tests/mclag] Submit mclag testcases according to mclag test plan #1239

wants to merge 3 commits into from

Conversation

freeman386
Copy link

@freeman386 freeman386 commented Nov 27, 2019 •
edited
Loading

Description of PR

Summary:
Submit MCLAG testcases according to mclag test plan [Azure/SONiC/pull/422].

Dependencies:

  1. MCLAG feature for SONIC [Azure/sonic-buildimage/pull/2514]
  2. MCLAG topo deployment [Azure/sonic-mgmt/pull/1251]
  3. Need install teamd on PTF

Type of change

  • [] Bug fix
  • [] Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Approach

How did you do it?

  1. mclag test on test_scenario L2
root@sonic-mgmt:/var/root/sonic-mgmt-main/tests# pytest -vvv --disable_loganalyzer --inventory veos --host-pattern all --user admin --testbed vms-t0-mclag-16 --testbed_file testbed.csv --show-capture=stdout test_mclag.py                                    
====================================================================== test session starts =======================================================================
platform linux2 -- Python 2.7.12, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python
cachedir: .pytest_cache
ansible: 2.0.0.2
rootdir: /var/root/sonic-mgmt-main/tests, inifile: pytest.ini
plugins: repeat-0.8.0, ansible-2.0.2
collected 56 items                                                                                                                                               

test_mclag.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link PASSED                                                                               [  1%]
test_mclag.py::TestCase1_VerifyMclagStatus::test_check_teamd_system_id PASSED                                                                              [  3%]
test_mclag.py::TestCase1_VerifyMclagStatus::test_traffic_between_servers PASSED                                                                            [  5%]
test_mclag.py::TestCase1_VerifyMclagStatus::test_syncheck[syncheck0] PASSED                                                                                [  7%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_traffic_between_servers PASSED                                                                  [  8%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_down PASSED                                                                   [ 10%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_up PASSED                                                                     [ 12%]
test_mclag.py::TestCase2_MclagMemberPortStatusChange::test_syncheck[syncheck0] PASSED                                                                      [ 14%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_traffic_between_servers PASSED                                                                         [ 16%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_peer_link_interface_down PASSED                                                                        [ 17%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_syncheck_on_mclag_interface[syncheck0] PASSED                                                          [ 19%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_peer_link_interface_up PASSED                                                                          [ 21%]
test_mclag.py::TestCase3_PeerLinkStatusChange::test_syncheck[syncheck0] PASSED                                                                             [ 23%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_traffic_between_servers PASSED                                                                    [ 25%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_keepalive_link_down PASSED                                                                        [ 26%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_keepalive_link_up PASSED                                                                          [ 28%]
test_mclag.py::TestCase4_KeepaliveLinkStatusChange::test_syncheck[syncheck0] PASSED                                                                        [ 30%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_traffic_between_servers PASSED                                                                [ 32%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_peer_keepalive_link_down PASSED                                                               [ 33%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_peer_keepalive_link_up PASSED                                                                 [ 35%]
test_mclag.py::TestCase5_PeerKeepaliveBothStatusChange::test_syncheck[syncheck0] PASSED                                                                    [ 37%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_traffic_between_servers PASSED                                                                        [ 39%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_active_down PASSED                                                                                    [ 41%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_active_up PASSED                                                                                      [ 42%]
test_mclag.py::TestCase6_ActiveDevStatusChange::test_syncheck[syncheck0] PASSED                                                                            [ 44%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_traffic_between_servers PASSED                                                                       [ 46%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_standby_down PASSED                                                                                  [ 48%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_standby_up PASSED                                                                                    [ 50%]
test_mclag.py::TestCase7_StandbyDevStatusChange::test_syncheck[syncheck0] PASSED                                                                           [ 51%]
test_mclag.py::TestCase8_ActiveDevWarmreboot::test_traffic_between_servers PASSED                                                                          [ 53%]
test_mclag.py::TestCase8_ActiveDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                        [ 55%]
test_mclag.py::TestCase8_ActiveDevWarmreboot::test_syncheck_after_warmreboot[syncheck0] PASSED                                                             [ 57%]
test_mclag.py::TestCase9_StandbyDevWarmreboot::test_traffic_between_servers PASSED                                                                         [ 58%]
test_mclag.py::TestCase9_StandbyDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                       [ 60%]
test_mclag.py::TestCase9_StandbyDevWarmreboot::test_syncheck_after_warmreboot[syncheck0] PASSED                                                            [ 62%]
test_mclag.py::TestCase10_MacFlapping::test_traffic_before_mac_flapping PASSED                                                                             [ 64%]
test_mclag.py::TestCase10_MacFlapping::test_syncheck_before_mac_flapping[syncheck0] PASSED                                                                 [ 66%]
test_mclag.py::TestCase10_MacFlapping::test_mac_flapping PASSED                                                                                            [ 67%]
test_mclag.py::TestCase11_MacSyncAndAge::test_traffic_between_servers PASSED                                                                               [ 69%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_mclag_member_port_down PASSED                                                           [ 71%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_mclag_member_port_up PASSED                                                             [ 73%]
test_mclag.py::TestCase11_MacSyncAndAge::test_traffic_between_servers_after_mclag_member_port_up PASSED                                                    [ 75%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_traffic_between_servers PASSED                                                          [ 76%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_before_age PASSED                                                                             [ 78%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mac_aging_on_peer PASSED                                                                                     [ 80%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_aging PASSED                                                                            [ 82%]
test_mclag.py::TestCase11_MacSyncAndAge::test_relearn_after_mac_age PASSED                                                                                 [ 83%]
test_mclag.py::TestCase11_MacSyncAndAge::test_mclag_age_flag_after_relearn PASSED                                                                          [ 85%]
test_mclag.py::TestCase12_ICCP_CSM::test_active_restart PASSED                                                                                             [ 87%]
test_mclag.py::TestCase12_ICCP_CSM::test_standby_restart PASSED                                                                                            [ 89%]
test_mclag.py::TestCase13_Scaling::test_traffic_between_servers PASSED                                                                                     [ 91%]
test_mclag.py::TestCase13_Scaling::test_mac_arp_sync PASSED                                                                                                [ 92%]
test_mclag.py::TestCase14_CornerTest::test_stop_teamd_on_active PASSED                                                                                     [ 94%]
test_mclag.py::TestCase14_CornerTest::test_start_teamd_on_active PASSED                                                                                    [ 96%]
test_mclag.py::TestCase14_CornerTest::test_stop_teamd_on_standby PASSED                                                                                    [ 98%]
test_mclag.py::TestCase14_CornerTest::test_start_teamd_on_standby PASSED                                                                                   [100%]

======================================================================== warnings summary ========================================================================
test_mclag.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link
  /usr/local/lib/python2.7/dist-packages/pytest_ansible/module_dispatcher/v2.py:70: UserWarning: provided hosts list is empty, only localhost is available
    warnings.warn("provided hosts list is empty, only localhost is available")

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================ 56 passed, 1 warnings in 5482.20 seconds ============================================================
  1. mclag test on test_scenario L3
root@sonic-mgmt:/var/root/sonic-mgmt-main/tests# pytest -vvv --disable_loganalyzer --inventory veos --host-pattern all --user admin --testbed vms-t0-mclag-16 --testbed_file testbed.csv --show-capture=stdout test_mclag_l3.py 
====================================================================== test session starts =======================================================================
platform linux2 -- Python 2.7.12, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python
cachedir: .pytest_cache
ansible: 2.0.0.2
rootdir: /var/root/sonic-mgmt-main/tests, inifile: pytest.ini
plugins: repeat-0.8.0, ansible-2.0.2
collected 24 items                                                                                                                                               

test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link PASSED                                                                            [  4%]
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_check_teamd_system_id PASSED                                                                           [  8%]
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_traffic_between_servers PASSED                                                                         [ 12%]
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_syncheck PASSED                                                                                        [ 16%]
test_mclag_l3.py::TestCase2_MclagMemberPortStatusChange::test_traffic_between_servers PASSED                                                               [ 20%]
test_mclag_l3.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_down PASSED                                                                [ 25%]
test_mclag_l3.py::TestCase2_MclagMemberPortStatusChange::test_mclag_member_port_up PASSED                                                                  [ 29%]
test_mclag_l3.py::TestCase3_KeepaliveLinkStatusChange::test_traffic_between_servers PASSED                                                                 [ 33%]
test_mclag_l3.py::TestCase3_KeepaliveLinkStatusChange::test_keepalive_link_down PASSED                                                                     [ 37%]
test_mclag_l3.py::TestCase3_KeepaliveLinkStatusChange::test_keepalive_link_up PASSED                                                                       [ 41%]
test_mclag_l3.py::TestCase4_ActiveDevStatusChange::test_traffic_between_servers PASSED                                                                     [ 45%]
test_mclag_l3.py::TestCase4_ActiveDevStatusChange::test_active_down PASSED                                                                                 [ 50%]
test_mclag_l3.py::TestCase4_ActiveDevStatusChange::test_active_up PASSED                                                                                   [ 54%]
test_mclag_l3.py::TestCase5_StandbyDevStatusChange::test_traffic_between_servers PASSED                                                                    [ 58%]
test_mclag_l3.py::TestCase5_StandbyDevStatusChange::test_standby_down PASSED                                                                               [ 62%]
test_mclag_l3.py::TestCase5_StandbyDevStatusChange::test_standby_up PASSED                                                                                 [ 66%]
test_mclag_l3.py::TestCase6_ActiveDevWarmreboot::test_traffic_between_servers PASSED                                                                       [ 70%]
test_mclag_l3.py::TestCase6_ActiveDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                     [ 75%]
test_mclag_l3.py::TestCase6_ActiveDevWarmreboot::test_syncheck PASSED                                                                                      [ 79%]
test_mclag_l3.py::TestCase7_StandbyDevWarmreboot::test_traffic_between_servers PASSED                                                                      [ 83%]
test_mclag_l3.py::TestCase7_StandbyDevWarmreboot::test_traffic_during_warmreboot PASSED                                                                    [ 87%]
test_mclag_l3.py::TestCase7_StandbyDevWarmreboot::test_syncheck PASSED                                                                                     [ 91%]
test_mclag_l3.py::TestCase8_Scaling::test_traffic_between_servers PASSED                                                                                   [ 95%]
test_mclag_l3.py::TestCase8_Scaling::test_syncheck PASSED                                                                                                  [100%]

======================================================================== warnings summary ========================================================================
test_mclag_l3.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link
  /usr/local/lib/python2.7/dist-packages/pytest_ansible/module_dispatcher/v2.py:70: UserWarning: provided hosts list is empty, only localhost is available
    warnings.warn("provided hosts list is empty, only localhost is available")

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================ 24 passed, 1 warnings in 2234.17 seconds ============================================================

Supported testbed topology if it's a new test case?

According test plan, we build mclag topo based on topo-t0, supported for topo_t0-mclag(which has 32 dut ports) and topo_t0-mclag-16(which has 16 dut ports).

Documentation

https://github.com/shine4chen/SONiC/blob/mclag-test-case/doc/mclag_hld/MCLAG-ansible-test-plan.md

@freeman386 freeman386 changed the title [tests/mclag] Submit mclag testcases according to mclagtest plan [tests/mclag] Submit mclag testcases according to mclag test plan Nov 27, 2019
@William-zx William-zx mentioned this pull request Dec 2, 2019
Open
1 task
@freeman386 freeman386 marked this pull request as ready for review December 2, 2019 08:32
@chenkelly
Copy link
Contributor

chenkelly commented Jul 1, 2020 •
edited
Loading

Hi @freeman386 and @William-zx
Could you guide us how to install teamd on PTF and run it successfully?
We try to install it (https://github.com/jpirko/libteam), but get error when we execute.
root@9a5d2e8482f3:/home# teamd This program is not intended to be run as root. Failed to create team device. teamd_init() failed. Failed: Operation not permitted
The source code of teamd does not allow the execution by root.
We also try remove code checking for root limitation, but get other issues.

Has you experienced this?
Please advise us your suggestion.
Thanks very much.

@William-zx
Copy link
Contributor

William-zx commented Jul 1, 2020 •
edited
Loading

@chenkelly
"teamd This program is not intended to be run as root." This prompt will also appear in our environment, but the teamd works normally.
Two ways to install teamd on PTF:

  1. Copy all teamd-related DEBs to the PTF and install them. Then use "docker commit" command to generate a new image.
  2. Modify the makefile of ptf in sonic-buildimage. Then bulid new ptf image. For example:
dockers/docker-ptf/Dockerfile.j2 
+        libdbus-1-3         \
+        libdaemon0          \
+        libjansson4         \

platform/nephos/docker-ptf-nephos.mk 
-$(DOCKER_PTF_NEPHOS)_DEPENDS += $(PYTHON_SAITHRIFT_NEPHOS)
+$(DOCKER_PTF_NEPHOS)_DEPENDS += $(PYTHON_SAITHRIFT) $(LIBTEAM) $(LIBTEAMDCT) $(LIBTEAM_UTILS) $(LIBNL3) $(LIBNL_GENL3) $(LIBNL_ROUTE3) $(LIBNL_NF3) $(LIBNL_CLI)

make configure PLATFORM=nephos
make target/docker-ptf-nephos.gz

@chenkelly
Copy link
Contributor

Hi @William-zx
Thanks. We use method 1 to generate PTF image which supports temad.
After we add t0-mclag topology on sonic-mgmt docker, we using "dmesg" command to check server status, but find that some error messages as follows.
[84977.213102] eth56: renamed from ens2f0.2457 [84977.247906] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247928] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247948] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247957] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.602181] eth57: renamed from ens2f0.2458 [84980.628172] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628190] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628211] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628230] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28
Has you experienced this? Our testbed server OS is ubuntu 16.04. Is it the appropriate version? Thanks.

@William-zx
Copy link
Contributor

Hi @William-zx
Thanks. We use method 1 to generate PTF image which supports temad.
After we add t0-mclag topology on sonic-mgmt docker, we using "dmesg" command to check server status, but find that some error messages as follows.
[84977.213102] eth56: renamed from ens2f0.2457 [84977.247906] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247928] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247948] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84977.247957] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.602181] eth57: renamed from ens2f0.2458 [84980.628172] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628190] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628211] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28 [84980.628230] __ib_cache_gid_add: unable to add gid fe80:0000:0000:0000:9a03:9bff:fe2a:ec28 error=-28
Has you experienced this? Our testbed server OS is ubuntu 16.04. Is it the appropriate version? Thanks.

@chenkelly
We do not experienced this, my testbed server OS is ubuntu 16.04.2.
You mean that you can add-topo T0/T1/T1-lag/ PTF32 successfully, but add-topo t0-mclag failed?
We just only added a few ports, not modified the common process of add-topo step.

@chenkelly
Copy link
Contributor

chenkelly commented Jul 6, 2020 •
edited
Loading

Hi @William-zx
We can add t0-mclag topology and deploy successfully, but see these message on testbed server.
We are not sure whether test_mclag.py is failed due to these message or not.
After running test_mclag.py, it hang on test case test_mclag.py::TestCase1_VerifyMclagStatus::test_check_keepalive_link and then we cannot remove topology successfully. We are checking configuration during testing.
If you have any suggestions, please tell us. Thanks very much
`[77845.001485] ens2f0.2456: renamed from eth55

[77846.329985] PortChannel24: Port device ens2f0.2456 removed

[77847.495120] ens2f0.2457: renamed from eth56

[77850.126644] ens2f0.2458: renamed from eth57

[77854.348950] device ens2f0.2427 left promiscuous mode

[77856.021577] device ens2f0.2459 left promiscuous mode

[77856.865984] device br-vms2-4-ic-1 left promiscuous mode

[77858.645834] device ens2f0.2428 left promiscuous mode

[77860.421388] device ens2f0.2460 left promiscuous mode

[77861.432696] device br-vms2-4-ic-2 left promiscuous mode

[77873.724953] unregister_netdevice: waiting for ens2f0.2401 to become free. Usage count = 2

[77883.901366] unregister_netdevice: waiting for ens2f0.2401 to become free. Usage count = 2`

@chenkelly
Copy link
Contributor

Hi @William-zx
We found root case. On our DUT, Ethernet index is not continuous, and will select incorrect Ethernet port for VLAN member assignment.
https://github.com/freeman386/sonic-mgmt/blob/master/tests/mclag/mclag_configuration.j2#L78
We will consider to use port_alias_map instead of Ethernet index.
Thanks very much.

vadymhlushko-mlnx pushed a commit to vadymhlushko-mlnx/sonic-mgmt that referenced this pull request Mar 15, 2021
@qiluo-msft
Add more log message, fix test code (sonic-net#1239)
8df99da
@freeman386 freeman386 requested a review from a team as a code owner March 2, 2022 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Design Review Needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@freeman386 @chenkelly @William-zx @lguohan