VXLAN config and show utilities #870

srj102 · 2020-04-08T10:33:30Z

- What I did

Added support for VXLAN config and commands as described in the PR sonic-net/SONiC#437

- How I did it

Modified the config and show main.py scripts for vxlan support.

- How to verify it

config vxlan add/del and config vxlan evpn_nvo add/del can be verified by using the
"show vxlan interface" command.
config vxlan map/map_range add can be verified by using the "show vxlan vlanvnimap"
command.
show vxlan remote vni/show vxlan remote mac can be verified by populating the
VXLAN_REMOTE_VNI and VXLAN_FDB tables in APP DB.
show vxlan tunnel can be verified by populating the VXLAN_TUNNEL_TABLE in STATE DB.

- Previous command output (if the output of a command-line utility has changed)

- New command output (if the output of a command-line utility has changed)
Please refer to the HLD for outputs.

lgtm-com · 2020-04-08T10:40:05Z

This pull request introduces 2 alerts when merging 9165184b710f39d5da4beae4854d991f66a88ddd into a0f3b93 - view on LGTM.com

new alerts:

1 for Unnecessary pass
1 for Unused local variable

lguohan · 2020-04-08T10:47:56Z

@Sj102, can you fix the lgtm warnings

lgtm-com · 2020-04-17T15:03:38Z

This pull request introduces 3 alerts when merging 831931bddefdb2fa6d712dc5c6efa3ce6d113ec0 into eccefa4 - view on LGTM.com

new alerts:

1 for Unnecessary pass
1 for Unused local variable
1 for Variable defined multiple times

lgtm-com · 2020-05-29T10:46:18Z

This pull request introduces 1 alert when merging 819b757481fb2692e3259c9be1385f438f57d50f into fab196e - view on LGTM.com

new alerts:

1 for Unnecessary pass

gord1306 · 2020-06-02T09:15:34Z

config/main.py

+    """Del VXLAN"""
+    db = ctx.obj['db']
+
+    vxlan_keys = db.keys('CONFIG_DB', "EVPN_NVO|*")


In sonic-swss-commin PR339, this table is named as VXLAN_EVPN_NVO

Changed to be in sync with PR339

gord1306 · 2020-06-02T09:15:47Z

config/main.py

+def add_vxlan_evpn_nvo(ctx, nvo_name, vxlan_name):
+    """Add NVO"""
+    db = ctx.obj['db']
+    vxlan_keys = db.keys('CONFIG_DB', "EVPN_NVO|*")


In sonic-swss-commin PR339, this table is named as VXLAN_EVPN_NVO

Changed to be in sync with PR 339

lgtm-com · 2020-06-18T11:08:25Z

This pull request introduces 2 alerts and fixes 5 when merging 0541c729f6c5d4b312b4b7c6199b315aecebf217 into fd7781b - view on LGTM.com

new alerts:

1 for Unnecessary pass
1 for Syntax error

fixed alerts:

4 for Module-level cyclic import
1 for Unused local variable

lgtm-com · 2020-06-18T12:43:33Z

This pull request introduces 1 alert when merging 316171402fcf3d136471f56d885bff48a2654493 into fd7781b - view on LGTM.com

new alerts:

1 for Unnecessary pass

config/main.py

dev-aviz · 2020-09-19T17:17:18Z

config/main.py

+           break
+
+    if (found == 1):
+        print "VNI {} mapped to Vrf {}, Please remove VRF VNI mapping".format(vni, vrf_key)


should it throw error or is it a warning? If it is error use ctx.fail().

This is a utility function which is used by other functions and hence returning True or False.

Seems that this line is the cause of errors during test phase. Probably because of Python version
See an error log after build, i.e. make target/python-wheels/sonic_utilities-1.2-py3-none-any.whl:

============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0 rootdir: /sonic/src/sonic-utilities, inifile: pytest.ini plugins: cov-2.6.0 collected 142 items / 22 errors ==================================== ERRORS ==================================== ________________ ERROR collecting tests/chassis_modules_test.py ________________ /usr/lib/python3/dist-packages/_pytest/python.py:450: in _importtestmodule mod = self.fspath.pyimport(ensuresyspath=importmode) /usr/lib/python3/dist-packages/py/_path/local.py:668: in pyimport __import__(modname) <frozen importlib._bootstrap>:983: in _find_and_load ??? <frozen importlib._bootstrap>:967: in _find_and_load_unlocked ??? <frozen importlib._bootstrap>:668: in _load_unlocked ??? <frozen importlib._bootstrap>:638: in _load_backward_compatible ??? /usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py:294: in load_module six.exec_(co, mod.__dict__) tests/chassis_modules_test.py:9: in <module> import show.main as show show/main.py:9: in <module> import utilities_common.cli as clicommon E File "/sonic/src/sonic-utilities/utilities_common/cli.py", line 359 E print "VNI {} mapped to Vrf {}, Please remove VRF VNI mapping".format(vni, vrf_key) E ^ E SyntaxError: invalid syntax ____________________ ERROR collecting tests/console_test.py ____________________ /usr/lib/python3/dist-packages/_pytest/python.py:450: in _importtestmodule mod = self.fspath.pyimport(ensuresyspath=importmode) /usr/lib/python3/dist-packages/py/_path/local.py:668: in pyimport __import__(modname) <frozen importlib._bootstrap>:983: in _find_and_load ??? <frozen importlib._bootstrap>:967: in _find_and_load_unlocked ??? <frozen importlib._bootstrap>:668: in _load_unlocked ??? <frozen importlib._bootstrap>:638: in _load_backward_compatible ??? /usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py:294: in load_module six.exec_(co, mod.__dict__) tests/console_test.py:9: in <module> import config.main as config config/main.py:23: in <module> import utilities_common.cli as clicommon E File "/sonic/src/sonic-utilities/utilities_common/cli.py", line 359 E print "VNI {} mapped to Vrf {}, Please remove VRF VNI mapping".format(vni, vrf_key) E ^ E SyntaxError: invalid syntax

... totally 22 times the same.

Probably the reason is #1128

Thanks for the comment. resolved now.

config/main.py

dev-aviz

review comments provided

prsunny

Suggest moving this functionality to separate file, like "vxlan.py" as there is lots of code being added.
Also can you provide some sample config/show output in the description so that can be used as reference?

config/main.py

show/main.py

lgtm-com · 2020-11-11T17:37:01Z

This pull request introduces 2 alerts when merging 9095051bd556093ff36d76390e6eced9861b781b into d5eb2f8 - view on LGTM.com

new alerts:

2 for Unused import

prsunny · 2020-11-13T16:16:49Z

Unit tests required

Config and show commands added to click infra for VXLAN objects. Please refer to sonic-net/SONiC#437 for the commands. The fast-reboot script is changed to not clear VXLAN tunnel state table during a warm reboot.

Added Neighbor Suppression CLI.

2. Tested on vs except for L3VXLAN+NeighSuppress+remote mac

1. Moved out vxlan related config and show utilities to a separate file. 2. Moved some utility functions to utilitie_common

changes to vxlan config utils

prsunny · 2020-12-11T16:47:43Z

retest this please

srj102 · 2020-12-14T04:15:36Z

retest this please

show vxlan tunnel_cfg (which reads from cfg db) which was originally show vxlan tunnel has been reverted back to show vxlan tunnel.

srj102 · 2020-12-16T19:43:06Z

retest vs please

srj102 · 2020-12-17T02:32:28Z

retest vs please

prsunny · 2020-12-17T03:12:17Z

utilities_common/cli.py

+#
+# Use this method to validate unicast IPv4 address
+#
+def is_ip4_addr_valid(addr, display):


Suggest use existing is_ipaddress function in this file

This API is not used currently. Removing it.

prsunny · 2020-12-17T03:13:31Z

utilities_common/cli.py

+
+    return True
+
+def is_vni_vrf_mapped(db, vni):


i think this is specific to vxlan case right? Suggest moving to vxlan.py?

This is a common API will be needed in both vxlan and vrf. So better to keep it in common.

prsunny · 2020-12-17T03:15:42Z

config/main.py

+@click.argument('vrfname', metavar='<vrf-name>', required=True, type=str)
+@click.argument('vni', metavar='<vni>', required=True)
+@click.pass_context
+def add_vrf_vni_map(ctx, vrfname, vni):


Could you have some test coverage for this case as well?

ok will do.

prsunny · 2020-12-17T03:17:22Z

retest this please

gord1306 · 2020-12-17T06:07:43Z

show/vxlan.py

+             output = '\tNVO Name  : ' + vtepname + ',  VTEP : ' + vxlan_table[key]['source_vtep']
+             click.echo(output)
+
+       vxlan_keys = config_db.keys('CONFIG_DB', "LOOPBACK_INTERFACE|*")


Is there any description to show that only loopback interface can be set to vtep

https://github.com/Azure/SONiC/blob/8645a6a3dd1467153b45ee4f9a08cbf7fd9183d0/doc/vxlan/EVPN/EVPN_VXLAN_HLD.md#512-show-commands

Apart from the show command output there is no other assumption in SONiC components. The VTEP IP should be reachable. Loopback interface is typically used.

Str1ker17 · 2020-12-18T05:39:33Z

config/main.py

+                break
+
+    if (found == 0):
+        ctx.fail(" VLAN VNI not mapped. Please create VLAN VNI map entry first ")


Do we need this leading space?

No. Updated code.

Str1ker17 · 2020-12-18T05:41:57Z

show/main.py

@@ -1522,6 +1522,5 @@ def ztp(status, verbose):
       cmd = cmd + " --verbose"
    run_command(cmd, display_cmd=verbose)

-


This change doesn't look obligatory.

Will handle this in the next PR.

Str1ker17 · 2020-12-18T05:43:01Z

show/vxlan.py

 from tabulate import tabulate

-


This change doesn't look obligatory.

Str1ker17 · 2020-12-18T05:46:37Z

show/vxlan.py

@@ -13,7 +13,6 @@ def vxlan():
    """Show vxlan related information"""
    pass

-


Same with this

Str1ker17 · 2020-12-18T05:46:42Z

show/vxlan.py

@@ -41,7 +40,6 @@ def name(vxlan_name):

    click.echo(tabulate(table, header))

-


Same with this

Removed space as per review comment.

Added support for VXLAN config and commands as described in the PR sonic-net/SONiC#437 config vxlan add/del and config vxlan evpn_nvo add/del config vxlan map/map_range add show vxlan remote vni/show vxlan remote mac show vxlan tunnel Co-authored-by: Tapash Das <tapash.das@broadcom.com> Co-authored-by: Karthikeyan Ananthakrishnan <karthikeyan.ananthakrishnan@broadcom.com> Co-authored-by: Tapash Das <48195098+tapashdas@users.noreply.github.com>

srj102 changed the title ~~Evpn vxlan~~ VXLAN config and show utilities Apr 8, 2020

prsunny self-assigned this Apr 8, 2020

gord1306 reviewed Jun 2, 2020

View reviewed changes

srj102 force-pushed the evpn_vxlan branch from 819b757 to 0541c72 Compare June 18, 2020 11:02

srj102 marked this pull request as ready for review June 19, 2020 04:43

dev-aviz reviewed Sep 19, 2020

View reviewed changes

config/main.py Outdated Show resolved Hide resolved

dev-aviz reviewed Sep 19, 2020

View reviewed changes

config/main.py Outdated Show resolved Hide resolved

dev-aviz reviewed Sep 19, 2020

View reviewed changes

prsunny reviewed Oct 7, 2020

View reviewed changes

config/main.py Outdated Show resolved Hide resolved

config/main.py Outdated Show resolved Hide resolved

show/main.py Outdated Show resolved Hide resolved

srj102 force-pushed the evpn_vxlan branch from 287396c to 3ef7b1c Compare November 11, 2020 16:03

srj102 and others added 10 commits December 11, 2020 00:50

Changes to support EVPN VXLAN.

42b055d

Config and show commands added to click infra for VXLAN objects. Please refer to sonic-net/SONiC#437 for the commands. The fast-reboot script is changed to not clear VXLAN tunnel state table during a warm reboot.

Changed the same for REMOTE_VNI Table.

2c609e2

Fixed LGTM Warnings

7e036df

Added VRF VNI Map CLI.

be0278f

Added Neighbor Suppression CLI.

LTGM Warning Fix.

fa13994

NeighSuppression Changes

68521b6

1. Updated EVPN DB names.

39561f0

2. Tested on vs except for L3VXLAN+NeighSuppress+remote mac

NeighSuppressConfigCliFix

c410175

Added UT Fix for VRF VNI MAP CLI.

818009b

Incorporated Review comment regarding hardcoding of values.

d507145

Review comments incorporated.

432e7c1

1. Moved out vxlan related config and show utilities to a separate file. 2. Moved some utility functions to utilitie_common

srj102 force-pushed the evpn_vxlan branch from 3112736 to 432e7c1 Compare December 11, 2020 09:48

srj102 added 2 commits December 11, 2020 02:21

Added tests for VXLAN CLI

7221a17

changes to vxlan config utils

Changes to test script to add newline in output.

c696f69

srj102 added 2 commits December 13, 2020 06:07

test scripts for vxlan config and show utilities added.

403d07e

Removing ARP/ND suppression CLIs on behalf of karthikeyan

b95be1b

srj102 and others added 3 commits December 16, 2020 09:48

Merge branch 'master' into evpn_vxlan

86779ec

Update main.py

95dc9b9

Renamed show vxlan tunnel to show vxlan remotevtep.

d9e9df6

show vxlan tunnel_cfg (which reads from cfg db) which was originally show vxlan tunnel has been reverted back to show vxlan tunnel.

prsunny reviewed Dec 17, 2020

View reviewed changes

gord1306 reviewed Dec 17, 2020

View reviewed changes

Updated review comments.

24958ff

msreddypaluru previously approved these changes Dec 18, 2020

View reviewed changes

Str1ker17 reviewed Dec 18, 2020

View reviewed changes

show/vxlan.py

from tabulate import tabulate

Copy link

Str1ker17 Dec 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change doesn't look obligatory.

Str1ker17 reviewed Dec 18, 2020

View reviewed changes

show/vxlan.py

@@ -13,7 +13,6 @@ def vxlan():

"""Show vxlan related information"""

pass

Copy link

Str1ker17 Dec 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with this

Str1ker17 reviewed Dec 18, 2020

View reviewed changes

show/vxlan.py

@@ -41,7 +40,6 @@ def name(vxlan_name):

click.echo(tabulate(table, header))

Copy link

Str1ker17 Dec 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with this

Update main.py

074ca0d

Removed space as per review comment.

tapashdas dismissed msreddypaluru’s stale review via 074ca0d December 18, 2020 06:21

prsunny approved these changes Dec 19, 2020

View reviewed changes

prsunny merged commit aad2c38 into sonic-net:master Dec 19, 2020

srj102 mentioned this pull request Mar 16, 2021

Fix the EVPN orch may access invalid address sonic-net/sonic-swss#1661

Open

		@@ -1522,6 +1522,5 @@ def ztp(status, verbose):
		cmd = cmd + " --verbose"
		run_command(cmd, display_cmd=verbose)

		@@ -13,7 +13,6 @@ def vxlan():
		"""Show vxlan related information"""
		pass

		@@ -41,7 +40,6 @@ def name(vxlan_name):

		click.echo(tabulate(table, header))

VXLAN config and show utilities #870

VXLAN config and show utilities #870

Conversation

srj102 commented Apr 8, 2020

lgtm-com bot commented Apr 8, 2020

lguohan commented Apr 8, 2020

lgtm-com bot commented Apr 17, 2020

lgtm-com bot commented May 29, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgtm-com bot commented Jun 18, 2020

lgtm-com bot commented Jun 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dev-aviz left a comment

Choose a reason for hiding this comment

prsunny left a comment

Choose a reason for hiding this comment

lgtm-com bot commented Nov 11, 2020

prsunny commented Nov 13, 2020

prsunny commented Dec 11, 2020

srj102 commented Dec 14, 2020

srj102 commented Dec 16, 2020

srj102 commented Dec 17, 2020

Choose a reason for hiding this comment

tapashdas Dec 17, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tapashdas Dec 17, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prsunny commented Dec 17, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tapashdas Dec 17, 2020 •

edited

tapashdas Dec 17, 2020 •

edited