Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SONiC Yang model support for LLDP #7191

Closed
wants to merge 4 commits into from
Closed

SONiC Yang model support for LLDP #7191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
13b4b93
SONiC Yang model support for LLDP
sandeep-kulambi Mar 30, 2021
ad3ad06
SONiC Yang model support for LLDP
sandeep-kulambi Apr 19, 2021
262b916
SONiC Yang model support for LLDP
sandeep-kulambi Apr 19, 2021
654c6b0
SONiC Yang model support for LLDP
sandeep-kulambi May 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions src/sonic-yang-models/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
'./yang-models/sonic-versions.yang',
'./yang-models/sonic-vlan.yang',
'./yang-models/sonic-vrf.yang',
'./yang-models/sonic-lldp.yang',
'./yang-models/sonic_yang_tree']),
],
zip_safe=False,
Expand Down
18 changes: 18 additions & 0 deletions src/sonic-yang-models/tests/files/sample_config_db.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -695,6 +695,24 @@
"nexthop_group_threshold_type": "percentage",
"polling_interval": "0"
}
},
"LLDP": {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added tests

"GLOBAL": {
"mode": "TRANSMIT",
"enabled": "true",
"hello_time": "12",
"multiplier": "5",
"supp_mgmt_address_tlv": "true",
"supp_system_capabilities_tlv": "false",
"system_name": "sonic",
"system_description": "sonic-system"
}
},
"LLDP_PORT": {
"Ethernet0": {
"mode": "TRANSMIT",
"enabled": "true"
}
}
},

Expand Down
28 changes: 28 additions & 0 deletions src/sonic-yang-models/tests/yang_model_tests/tests/lldp.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"LLDP_GLOBAL_TEST": {
"desc": "LLDP test global LLDP table"
},
"LLDP_GLOBAL_TEST_INVALID_HELLO": {
"desc": "LLDP test global table with invalid hello value",
"eStrKey" : "InvalidValue",
"eStr": ["hello_time"]
},
"LLDP_GLOBAL_TEST_INVALID_MULTIPLIER": {
"desc": "LLDP test global table with invalid multiplier value",
"eStrKey" : "InvalidValue",
"eStr": ["multiplier"]
},
"LLDP_PORT_TEST": {
"desc": "LLDP test LLDP port table"
},
"LLDP_PORT_TEST_INVALID_IFNAME": {
"desc": "LLDP test port table with invalid ifname",
"eStrKey" : "LeafRef",
"eStr": ["Eth"]
},
"LLDP_PORT_TEST_INVALID_MODE": {
"desc": "LLDP test port table with invalid mode",
"eStrKey" : "InvalidValue",
"eStr": ["mode"]
}
}
166 changes: 166 additions & 0 deletions src/sonic-yang-models/tests/yang_model_tests/tests_config/lldp.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
{
"LLDP_GLOBAL_TEST": {
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP": {
"sonic-lldp:GLOBAL": {
"mode": "TRANSMIT",
"enabled": "true",
"hello_time": "20",
"multiplier": "5",
"supp_mgmt_address_tlv": "true",
"supp_system_capabilities_tlv": "false",
"system_name": "sonic",
"system_description": "sonic-system"
}
}
}
},
"LLDP_GLOBAL_TEST_INVALID_ID": {
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP": {
"sonic-lldp:GLOBAL": {
"mode": "TRANSMIT",
"enabled": "true",
"hello_time": "10",
"multiplier": "5",
"supp_mgmt_address_tlv": "true",
"supp_system_capabilities_tlv": "false",
"system_name": "sonic",
"system_description": "sonic-system"
}
}
}
},
"LLDP_GLOBAL_TEST_INVALID_HELLO": {
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP": {
"sonic-lldp:GLOBAL": {
"mode": "TRANSMIT",
"enabled": "true",
"hello_time": "test",
"multiplier": "5",
"supp_mgmt_address_tlv": "true",
"supp_system_capabilities_tlv": "false",
"system_name": "sonic",
"system_description": "sonic-system"
}
}
}
},
"LLDP_GLOBAL_TEST_INVALID_MULTIPLIER": {
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP": {
"sonic-lldp:GLOBAL": {
"mode": "TRANSMIT",
"enabled": "true",
"hello_time": "10",
"multiplier": "xyz",
"supp_mgmt_address_tlv": "true",
"supp_system_capabilities_tlv": "false",
"system_name": "sonic",
"system_description": "sonic-system"
}
}
}
},
"LLDP_GLOBAL_TEST_DEFAULT_SUPP_MGMT_ADDR_TLV": {
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP": {
"sonic-lldp:GLOBAL": {
"mode": "TRANSMIT",
"enabled": "true",
"hello_time": "10",
"multiplier": "5",
"supp_mgmt_address_tlv": "false",
"supp_system_capabilities_tlv": "false",
"system_name": "sonic",
"system_description": "sonic-system"
}
}
}
},
"LLDP_PORT_TEST": {
"sonic-port:sonic-port": {
"sonic-port:PORT": {
"PORT_LIST": [
{
"admin_status": "up",
"alias": "eth0",
"description": "Ethernet0",
"lanes": "65",
"mtu": 9000,
"name": "Ethernet0",
"speed": 25000
}
]
}
},
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP_PORT": {
"LLDP_PORT_LIST": [
{
"ifname": "Ethernet0",
"mode": "TRANSMIT",
"enabled": "true"
}
]
}
}
},
"LLDP_PORT_TEST_INVALID_IFNAME": {
"sonic-port:sonic-port": {
"sonic-port:PORT": {
"PORT_LIST": [
{
"admin_status": "up",
"alias": "eth0",
"description": "Ethernet0",
"lanes": "65",
"mtu": 9000,
"name": "Ethernet0",
"speed": 25000
}
]
}
},
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP_PORT": {
"LLDP_PORT_LIST": [
{
"ifname": "Eth",
"mode": "TRANSMIT",
"enabled": "true"
}
]
}
}
},
"LLDP_PORT_TEST_INVALID_MODE": {
"sonic-port:sonic-port": {
"sonic-port:PORT": {
"PORT_LIST": [
{
"admin_status": "up",
"alias": "eth0",
"description": "Ethernet0",
"lanes": "65",
"mtu": 9000,
"name": "Ethernet0",
"speed": 25000
}
]
}
},
"sonic-lldp:sonic-lldp": {
"sonic-lldp:LLDP_PORT": {
"LLDP_PORT_LIST": [
{
"ifname": "Ethernet0",
"mode": "TRANSMITTING",
"enabled": "true"
}
]
}
}
}
}
154 changes: 154 additions & 0 deletions src/sonic-yang-models/yang-models/sonic-lldp.yang
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
module sonic-lldp {
namespace "http://github.com/Azure/sonic-lldp";
prefix slldp;
yang-version 1.1;

import sonic-port {
prefix prt;
}

import sonic-extension {
prefix sonic-ext;
}

organization
"SONiC";

contact
"SONiC";

description
"SONiC LLDP yang model";

revision 2020-04-16 {
description
"Initial revision.";
}

grouping lldp_mode_config {
leaf enabled {
type boolean;
description
"Enable/Disable LLDP";
}

leaf mode {
type enumeration {
enum RECEIVE;
enum TRANSMIT;
}

description
"RX/TX mode for LLDP frames";
}
}

container sonic-lldp {
container LLDP {
container GLOBAL {

leaf hello_time {
type uint8 {
range "5..254" {
error-message "Invalid hello timer value.";
}
}
default 30;
units seconds;
description
"It is the time interval at which periodic hellos are
exchanged. Default is 30 seconds";
}

leaf multiplier {
type uint8 {
range "1..10" {
error-message "Invalid LLDP multiplier value.";
}
}
default 4;
description
"This multiplier value is used to determine the timeout
interval (i.e. hello-time x multiplier value) after
which LLDP neighbor entry is deleted.";
}

leaf system_name {
type string;
description
"System administratively assigned name";
}

leaf system_description {
type string;
description
"System description";
}

leaf supp_mgmt_address_tlv {
type boolean;
description
"Suppress sending of Management Address TLV in LLDP frames";
}

leaf supp_system_capabilities_tlv {
type boolean;
default false;
description
"Suppress sending of System Capabilities TLV in LLDP frames";
}

leaf enabled {
type boolean;
default true;
description
"Enable/Disable LLDP";
}

leaf mode {
type enumeration {
enum RECEIVE;
enum TRANSMIT;
}

description
"RX/TX mode for LLDP frames";
}

//uses lldp_mode_config;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this from RFC 7950, can you plz point to the section. Thx

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RFC7950 section -
7.13. The "uses" Statement

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for commenting "//uses lldp_mode_config;"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compilation fails with the usage of grouping (not related to this PR) hence its commented out, will uncomment once that issue is fixed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We dont see any outstanding issue on this, please mention the exact issue here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see below error when grouping is enabled -

Using /sonic/src/sonic-yang-models/.eggs/ijson-2.6.1-py3.7-linux-x86_64.egg
running egg_info
writing sonic_yang_models.egg-info/PKG-INFO
writing dependency_links to sonic_yang_models.egg-info/dependency_links.txt
writing top-level names to sonic_yang_models.egg-info/top_level.txt
reading manifest file 'sonic_yang_models.egg-info/SOURCES.txt'
writing manifest file 'sonic_yang_models.egg-info/SOURCES.txt'
running build_ext
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /sonic/src/sonic-yang-models, inifile:
plugins: cov-2.6.0
collected 3 items

tests/test_sonic_yang_models.py .F [ 66%]
tests/yang_model_tests/test_yang_model.py . [100%]

=================================== FAILURES ===================================
___________________________ test_generate_yang_tree ____________________________

def test_generate_yang_tree():

    # Generate YANG Tree, see no error in it.
    pyang_tree_cmd = "pyang -f tree ./yang-models/*.yang > ./yang-models/sonic_yang_tree"
    if (system(pyang_tree_cmd)):
        print("Failed: {}".format(pyang_tree_cmd))

      exit(1)

E SystemExit: 1

tests/test_sonic_yang_models.py:30: SystemExit
----------------------------- Captured stdout call -----------------------------
Failed: pyang -f tree ./yang-models/*.yang > ./yang-models/sonic_yang_tree
----------------------------- Captured stderr call -----------------------------
./yang-models/sonic-acl.yang:8: error: module "ietf-inet-types" not found in search path
./yang-models/sonic-acl.yang:17: warning: imported module "sonic-extension" not used
./yang-models/sonic-breakout_cfg.yang:8: warning: imported module "sonic-extension" not used
./yang-models/sonic-device_metadata.yang:8: error: module "ietf-yang-types" not found in search path
./yang-models/sonic-device_neighbor.yang:12: warning: imported module "sonic-extension" not used
./yang-models/sonic-interface.yang:13: warning: imported module "sonic-extension" not used
./yang-models/sonic-lldp.yang:10: warning: imported module "sonic-extension" not used
./yang-models/sonic-loopback-interface.yang:13: warning: imported module "sonic-extension" not used
./yang-models/sonic-port.yang:13: warning: imported module "sonic-extension" not used
./yang-models/sonic-portchannel.yang:13: warning: imported module "sonic-extension" not used
./yang-models/sonic-vlan.yang:17: warning: imported module "sonic-extension" not used
./yang-models/sonic-vrf.yang:5: warning: imported module "sonic-extension" not used
====================== 1 failed, 2 passed in 1.11 seconds ======================
[ FAIL LOG END ] [ target/python-wheels/sonic_yang_models-1.0-py3-none-any.whl ]
make: *** [slave.mk:602: target/python-wheels/sonic_yang_models-1.0-py3-none-any.whl] Error 1
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Makefile.work:287: recipe for target 'target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl' failed
make[1]: *** [target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl] Error 2
make[1]: Leaving directory '/projects/csg_sonic2/sk408012/SONiC/upstream/yang/sonic-buildimage'
Makefile:7: recipe for target 'target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl' failed
make: *** [target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl] Error 2

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems you need to re-base, this doesn't have the latest fix in this PYANG test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post rebase seeing below issue -

running build_ext
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
rootdir: /sonic/src/sonic-yang-mgmt, inifile:
plugins: cov-2.6.0
collected 26 items

tests/test_sonic_yang_mgmt.py . [ 3%]
tests/libyang-python-tests/test_sonic_yang.py ......................FF. [100%]

=================================== FAILURES ===================================
___________________ Test_SonicYang.test_validate_yang_models ___________________

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
configdbJson = {'ACL_RULE': {'V4-ACL-TABLE|DEFAULT_DENY': {'IP_TYPE': 'IPv4ANY', 'PACKET_ACTION': 'DROP', 'PRIORITY': '0'}, 'V4-ACL-T...ter_low_threshold': '70', 'acl_counter_threshold_type': 'percentage', 'ipv6_neighbor_high_threshold': '67', ...}}, ...}
debug = False

def loadData(self, configdbJson, debug=False):

   try:
      # write Translated config in file if debug enabled
      xlateFile = None
      if debug:
          xlateFile = "xlateConfig.json"
      self.jIn = configdbJson
      # reset xlate and tablesWithOutYang
      self.xlateJson = dict()
      self.tablesWithOutYang = dict()
      # self.jIn will be cropped
      self._cropConfigDB()
      # xlated result will be in self.xlateJson

    self._xlateConfigDB(xlateFile=xlateFile)

sonic_yang_ext.py:715:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>, xlateFile = None

def _xlateConfigDB(self, xlateFile=None):

    jIn= self.jIn
    yangJ = self.xlateJson
    # xlation is written in self.xlateJson

  self._xlateConfigDBtoYang(jIn, yangJ)

sonic_yang_ext.py:432:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
jIn = {'ACL_RULE': {'V4-ACL-TABLE|DEFAULT_DENY': {'IP_TYPE': 'IPv4ANY', 'PACKET_ACTION': 'DROP', 'PRIORITY': '0'}, 'V4-ACL-T...ter_low_threshold': '70', 'acl_counter_threshold_type': 'percentage', 'ipv6_neighbor_high_threshold': '67', ...}}, ...}
yangJ = {'sonic-acl:sonic-acl': {'sonic-acl:ACL_RULE': {'ACL_RULE_LIST': [{'ACL_TABLE_NAME': 'V4-ACL-TABLE', 'IP_TYPE': 'IPv4A...TA': {'localhost': {'bgp_asn': '64850', 'buffer_model': 'dynamic', 'hostname': 'asw.dc', 'hwsku': 'Stone', ...}}}, ...}

def _xlateConfigDBtoYang(self, jIn, yangJ):

    # find top level container for each table, and run the xlate_container.
    for table in jIn.keys():
        cmap = self.confDbYangMap[table]
        # create top level containers
        key = cmap['module']+":"+cmap['topLevelContainer']
        subkey = cmap['topLevelContainer']+":"+cmap['container']['@name']
        # Add new top level container for first table in this container
        yangJ[key] = dict() if yangJ.get(key) is None else yangJ[key]
        yangJ[key][subkey] = dict()
        self.sysLog(msg="xlateConfigDBtoYang {}:{}".format(key, subkey))
        self._xlateContainer(cmap['container'], yangJ[key][subkey], \

                          jIn[table], table)

sonic_yang_ext.py:420:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
model = OrderedDict([('@name', 'LLDP'), ('container', OrderedDict([('@name', 'GLOBAL'), ('leaf', [OrderedDict([('@name', 'hell...ities TLV in LLDP frames')])), ('__isleafList', False)])]), ('uses', OrderedDict([('@name', 'lldp_mode_config')]))]))])
yang = {'GLOBAL': {'hello_time': 12, 'multiplier': 5, 'supp_mgmt_address_tlv': 'true', 'supp_system_capabilities_tlv': 'false', ...}}
config = {'GLOBAL': {'enabled': 'true', 'hello_time': '12', 'mode': 'TRANSMIT', 'multiplier': '5', ...}}
table = 'LLDP'

def _xlateContainer(self, model, yang, config, table):

    # To Handle multiple Lists, Make a copy of config, because we delete keys
    # from config after each match. This is done to match one pkey with one list.
    configC = config.copy()
    exceptionList = list()
    clist = model.get('list')
    # If single list exists in container,
    if clist and isinstance(clist, dict) and \
       clist['@name'] == model['@name']+"_LIST" and bool(configC):
            self._xlateListInContainer(clist, yang, configC, table, \
                exceptionList)
    # If multi-list exists in container,
    elif clist and isinstance(clist, list) and bool(configC):
        for modelList in clist:
            self._xlateListInContainer(modelList, yang, configC, table, \
                exceptionList)

    # Handle container(s) in container
    ccontainer = model.get('container')
    # If single list exists in container,
    if ccontainer and isinstance(ccontainer, dict) and bool(configC):

      self._xlateContainerInContainer(ccontainer, yang, configC, table)

sonic_yang_ext.py:376:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
model = OrderedDict([('@name', 'GLOBAL'), ('leaf', [OrderedDict([('@name', 'hello_time'), ('type', OrderedDict([('@name', 'uin...bilities TLV in LLDP frames')])), ('__isleafList', False)])]), ('uses', OrderedDict([('@name', 'lldp_mode_config')]))])
yang = {'GLOBAL': {'hello_time': 12, 'multiplier': 5, 'supp_mgmt_address_tlv': 'true', 'supp_system_capabilities_tlv': 'false', ...}}
configC = {'GLOBAL': {'enabled': 'true', 'hello_time': '12', 'mode': 'TRANSMIT', 'multiplier': '5', ...}}
table = 'LLDP'

def _xlateContainerInContainer(self, model, yang, configC, table):
    ccontainer = model
    #print(ccontainer['@name'])
    yang[ccontainer['@name']] = dict()
    if not configC.get(ccontainer['@name']):
        return
    self.sysLog(msg="xlateProcessListOfContainer: {}".format(ccontainer['@name']))
    self._xlateContainer(ccontainer, yang[ccontainer['@name']], \

  configC[ccontainer['@name']], table)

sonic_yang_ext.py:340:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
model = OrderedDict([('@name', 'GLOBAL'), ('leaf', [OrderedDict([('@name', 'hello_time'), ('type', OrderedDict([('@name', 'uin...bilities TLV in LLDP frames')])), ('__isleafList', False)])]), ('uses', OrderedDict([('@name', 'lldp_mode_config')]))])
yang = {'hello_time': 12, 'multiplier': 5, 'supp_mgmt_address_tlv': 'true', 'supp_system_capabilities_tlv': 'false', ...}
config = {'enabled': 'true', 'hello_time': '12', 'mode': 'TRANSMIT', 'multiplier': '5', ...}
table = 'LLDP'

def _xlateContainer(self, model, yang, config, table):

    # To Handle multiple Lists, Make a copy of config, because we delete keys
    # from config after each match. This is done to match one pkey with one list.
    configC = config.copy()
    exceptionList = list()
    clist = model.get('list')
    # If single list exists in container,
    if clist and isinstance(clist, dict) and \
       clist['@name'] == model['@name']+"_LIST" and bool(configC):
            self._xlateListInContainer(clist, yang, configC, table, \
                exceptionList)
    # If multi-list exists in container,
    elif clist and isinstance(clist, list) and bool(configC):
        for modelList in clist:
            self._xlateListInContainer(modelList, yang, configC, table, \
                exceptionList)

    # Handle container(s) in container
    ccontainer = model.get('container')
    # If single list exists in container,
    if ccontainer and isinstance(ccontainer, dict) and bool(configC):
        self._xlateContainerInContainer(ccontainer, yang, configC, table)
    # If multi-list exists in container,
    elif ccontainer and isinstance(ccontainer, list) and bool(configC):
        for modelContainer in ccontainer:
            self._xlateContainerInContainer(modelContainer, yang, configC, table)

    ## Handle other leaves in container,
    leafDict = self._createLeafDict(model)
    vKeys = list(configC.keys())
    for vKey in vKeys:
        #vkey must be a leaf\leaf-list\choice in container
        if leafDict.get(vKey):
            self.sysLog(syslog.LOG_DEBUG, "xlateContainer vkey {}".format(vKey))
            yang[vKey] = self._findYangTypedValue(vKey, configC[vKey], leafDict)
            # delete entry from copy of config
            del configC[vKey]

    # All entries in copy of config must have been parsed.
    if len(configC):
        self.sysLog(msg="All Keys are not parsed in {}\n{}".format(table, \
            configC.keys()), debug=syslog.LOG_ERR, doPrint=True)
        self.sysLog(msg="exceptionList:{}".format(exceptionList), \
            debug=syslog.LOG_ERR, doPrint=True)
        raise(Exception("All Keys are not parsed in {}\n{}".format(table, \

          configC.keys())))

E Exception: All Keys are not parsed in LLDP
E dict_keys(['mode', 'enabled'])

sonic_yang_ext.py:400: Exception

During handling of the above exception, another exception occurred:

self = <test_sonic_yang.Test_SonicYang object at 0x7fc63b710128>
sonic_yang_data = {'syc': <sonic_yang.SonicYang object at 0x7fc63bb549b0>, 'test_file': '../sonic-yang-models/tests/files/sample_config_db.json', 'yang_dir': '../sonic-yang-models/yang-models/'}

def test_validate_yang_models(self, sonic_yang_data):
    '''
    In this test, we validate yang models
    a.) by converting the config as per RFC 7951 using YANG Models,
    b.) by creating data tree using new YANG models and
    c.) by validating config against YANG models.
    Successful execution of these steps can be treated as
    validation of new Yang models.
    '''
    test_file = sonic_yang_data['test_file']
    syc = sonic_yang_data['syc']
    # Currently only 2 YANG files are not directly related to config
    # which are: sonic-extension.yang and sonic-types.yang. Hard coding
    # it right now.
    # If any more such helper yang files are added, we need to update here.
    NON_CONFIG_YANG_FILES = 2
    # read config
    jIn = self.readIjsonInput(test_file, 'SAMPLE_CONFIG_DB_JSON')
    jIn = json.loads(jIn)
    numTables = len(jIn)
    # load config and create Data tree

  syc.loadData(jIn)

tests/libyang-python-tests/test_sonic_yang.py:305:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
configdbJson = {'ACL_RULE': {'V4-ACL-TABLE|DEFAULT_DENY': {'IP_TYPE': 'IPv4ANY', 'PACKET_ACTION': 'DROP', 'PRIORITY': '0'}, 'V4-ACL-T...ter_low_threshold': '70', 'acl_counter_threshold_type': 'percentage', 'ipv6_neighbor_high_threshold': '67', ...}}, ...}
debug = False

def loadData(self, configdbJson, debug=False):

   try:
      # write Translated config in file if debug enabled
      xlateFile = None
      if debug:
          xlateFile = "xlateConfig.json"
      self.jIn = configdbJson
      # reset xlate and tablesWithOutYang
      self.xlateJson = dict()
      self.tablesWithOutYang = dict()
      # self.jIn will be cropped
      self._cropConfigDB()
      # xlated result will be in self.xlateJson
      self._xlateConfigDB(xlateFile=xlateFile)
      #print("load data - {}".format(self.xlateJson))
      self.sysLog(msg="Try to load Data in the tree")
      self.root = self.ctx.parse_data_mem(dumps(self.xlateJson), \
                    ly.LYD_JSON, ly.LYD_OPT_CONFIG|ly.LYD_OPT_STRICT)

   except Exception as e:
       self.root = None
       self.sysLog(msg="Data Loading Failed:{}".format(str(e)), \
        debug=syslog.LOG_ERR, doPrint=True)

     raise SonicYangException("Data Loading Failed\n{}".format(str(e)))

E sonic_yang_ext.SonicYangException: Data Loading Failed
E All Keys are not parsed in LLDP
E dict_keys(['mode', 'enabled'])

sonic_yang_ext.py:725: SonicYangException
----------------------------- Captured stdout call -----------------------------
Read JSON Section: SAMPLE_CONFIG_DB_JSON
sonic_yang(3):All Keys are not parsed in LLDP
dict_keys(['mode', 'enabled'])
sonic_yang(3):exceptionList:[]
sonic_yang(3):Data Loading Failed:All Keys are not parsed in LLDP
dict_keys(['mode', 'enabled'])
_____________________ Test_SonicYang.test_xlate_rev_xlate ______________________

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
configdbJson = {'ACL_RULE': {'V4-ACL-TABLE|DEFAULT_DENY': {'IP_TYPE': 'IPv4ANY', 'PACKET_ACTION': 'DROP', 'PRIORITY': '0'}, 'V4-ACL-T...ter_low_threshold': '70', 'acl_counter_threshold_type': 'percentage', 'ipv6_neighbor_high_threshold': '67', ...}}, ...}
debug = False

def loadData(self, configdbJson, debug=False):

   try:
      # write Translated config in file if debug enabled
      xlateFile = None
      if debug:
          xlateFile = "xlateConfig.json"
      self.jIn = configdbJson
      # reset xlate and tablesWithOutYang
      self.xlateJson = dict()
      self.tablesWithOutYang = dict()
      # self.jIn will be cropped
      self._cropConfigDB()
      # xlated result will be in self.xlateJson

    self._xlateConfigDB(xlateFile=xlateFile)

sonic_yang_ext.py:715:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>, xlateFile = None

def _xlateConfigDB(self, xlateFile=None):

    jIn= self.jIn
    yangJ = self.xlateJson
    # xlation is written in self.xlateJson

  self._xlateConfigDBtoYang(jIn, yangJ)

sonic_yang_ext.py:432:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
jIn = {'ACL_RULE': {'V4-ACL-TABLE|DEFAULT_DENY': {'IP_TYPE': 'IPv4ANY', 'PACKET_ACTION': 'DROP', 'PRIORITY': '0'}, 'V4-ACL-T...ter_low_threshold': '70', 'acl_counter_threshold_type': 'percentage', 'ipv6_neighbor_high_threshold': '67', ...}}, ...}
yangJ = {'sonic-acl:sonic-acl': {'sonic-acl:ACL_RULE': {'ACL_RULE_LIST': [{'ACL_TABLE_NAME': 'V4-ACL-TABLE', 'IP_TYPE': 'IPv4A...TA': {'localhost': {'bgp_asn': '64850', 'buffer_model': 'dynamic', 'hostname': 'asw.dc', 'hwsku': 'Stone', ...}}}, ...}

def _xlateConfigDBtoYang(self, jIn, yangJ):

    # find top level container for each table, and run the xlate_container.
    for table in jIn.keys():
        cmap = self.confDbYangMap[table]
        # create top level containers
        key = cmap['module']+":"+cmap['topLevelContainer']
        subkey = cmap['topLevelContainer']+":"+cmap['container']['@name']
        # Add new top level container for first table in this container
        yangJ[key] = dict() if yangJ.get(key) is None else yangJ[key]
        yangJ[key][subkey] = dict()
        self.sysLog(msg="xlateConfigDBtoYang {}:{}".format(key, subkey))
        self._xlateContainer(cmap['container'], yangJ[key][subkey], \

                          jIn[table], table)

sonic_yang_ext.py:420:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
model = OrderedDict([('@name', 'LLDP'), ('container', OrderedDict([('@name', 'GLOBAL'), ('leaf', [OrderedDict([('@name', 'hell...ities TLV in LLDP frames')])), ('__isleafList', False)])]), ('uses', OrderedDict([('@name', 'lldp_mode_config')]))]))])
yang = {'GLOBAL': {'hello_time': 12, 'multiplier': 5, 'supp_mgmt_address_tlv': 'true', 'supp_system_capabilities_tlv': 'false', ...}}
config = {'GLOBAL': {'enabled': 'true', 'hello_time': '12', 'mode': 'TRANSMIT', 'multiplier': '5', ...}}
table = 'LLDP'

def _xlateContainer(self, model, yang, config, table):

    # To Handle multiple Lists, Make a copy of config, because we delete keys
    # from config after each match. This is done to match one pkey with one list.
    configC = config.copy()
    exceptionList = list()
    clist = model.get('list')
    # If single list exists in container,
    if clist and isinstance(clist, dict) and \
       clist['@name'] == model['@name']+"_LIST" and bool(configC):
            self._xlateListInContainer(clist, yang, configC, table, \
                exceptionList)
    # If multi-list exists in container,
    elif clist and isinstance(clist, list) and bool(configC):
        for modelList in clist:
            self._xlateListInContainer(modelList, yang, configC, table, \
                exceptionList)

    # Handle container(s) in container
    ccontainer = model.get('container')
    # If single list exists in container,
    if ccontainer and isinstance(ccontainer, dict) and bool(configC):

      self._xlateContainerInContainer(ccontainer, yang, configC, table)

sonic_yang_ext.py:376:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
model = OrderedDict([('@name', 'GLOBAL'), ('leaf', [OrderedDict([('@name', 'hello_time'), ('type', OrderedDict([('@name', 'uin...bilities TLV in LLDP frames')])), ('__isleafList', False)])]), ('uses', OrderedDict([('@name', 'lldp_mode_config')]))])
yang = {'GLOBAL': {'hello_time': 12, 'multiplier': 5, 'supp_mgmt_address_tlv': 'true', 'supp_system_capabilities_tlv': 'false', ...}}
configC = {'GLOBAL': {'enabled': 'true', 'hello_time': '12', 'mode': 'TRANSMIT', 'multiplier': '5', ...}}
table = 'LLDP'

def _xlateContainerInContainer(self, model, yang, configC, table):
    ccontainer = model
    #print(ccontainer['@name'])
    yang[ccontainer['@name']] = dict()
    if not configC.get(ccontainer['@name']):
        return
    self.sysLog(msg="xlateProcessListOfContainer: {}".format(ccontainer['@name']))
    self._xlateContainer(ccontainer, yang[ccontainer['@name']], \

  configC[ccontainer['@name']], table)

sonic_yang_ext.py:340:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
model = OrderedDict([('@name', 'GLOBAL'), ('leaf', [OrderedDict([('@name', 'hello_time'), ('type', OrderedDict([('@name', 'uin...bilities TLV in LLDP frames')])), ('__isleafList', False)])]), ('uses', OrderedDict([('@name', 'lldp_mode_config')]))])
yang = {'hello_time': 12, 'multiplier': 5, 'supp_mgmt_address_tlv': 'true', 'supp_system_capabilities_tlv': 'false', ...}
config = {'enabled': 'true', 'hello_time': '12', 'mode': 'TRANSMIT', 'multiplier': '5', ...}
table = 'LLDP'

def _xlateContainer(self, model, yang, config, table):

    # To Handle multiple Lists, Make a copy of config, because we delete keys
    # from config after each match. This is done to match one pkey with one list.
    configC = config.copy()
    exceptionList = list()
    clist = model.get('list')
    # If single list exists in container,
    if clist and isinstance(clist, dict) and \
       clist['@name'] == model['@name']+"_LIST" and bool(configC):
            self._xlateListInContainer(clist, yang, configC, table, \
                exceptionList)
    # If multi-list exists in container,
    elif clist and isinstance(clist, list) and bool(configC):
        for modelList in clist:
            self._xlateListInContainer(modelList, yang, configC, table, \
                exceptionList)

    # Handle container(s) in container
    ccontainer = model.get('container')
    # If single list exists in container,
    if ccontainer and isinstance(ccontainer, dict) and bool(configC):
        self._xlateContainerInContainer(ccontainer, yang, configC, table)
    # If multi-list exists in container,
    elif ccontainer and isinstance(ccontainer, list) and bool(configC):
        for modelContainer in ccontainer:
            self._xlateContainerInContainer(modelContainer, yang, configC, table)

    ## Handle other leaves in container,
    leafDict = self._createLeafDict(model)
    vKeys = list(configC.keys())
    for vKey in vKeys:
        #vkey must be a leaf\leaf-list\choice in container
        if leafDict.get(vKey):
            self.sysLog(syslog.LOG_DEBUG, "xlateContainer vkey {}".format(vKey))
            yang[vKey] = self._findYangTypedValue(vKey, configC[vKey], leafDict)
            # delete entry from copy of config
            del configC[vKey]

    # All entries in copy of config must have been parsed.
    if len(configC):
        self.sysLog(msg="All Keys are not parsed in {}\n{}".format(table, \
            configC.keys()), debug=syslog.LOG_ERR, doPrint=True)
        self.sysLog(msg="exceptionList:{}".format(exceptionList), \
            debug=syslog.LOG_ERR, doPrint=True)
        raise(Exception("All Keys are not parsed in {}\n{}".format(table, \

          configC.keys())))

E Exception: All Keys are not parsed in LLDP
E dict_keys(['mode', 'enabled'])

sonic_yang_ext.py:400: Exception

During handling of the above exception, another exception occurred:

self = <test_sonic_yang.Test_SonicYang object at 0x7fc63b694da0>
sonic_yang_data = {'syc': <sonic_yang.SonicYang object at 0x7fc63bb549b0>, 'test_file': '../sonic-yang-models/tests/files/sample_config_db.json', 'yang_dir': '../sonic-yang-models/yang-models/'}

def test_xlate_rev_xlate(self, sonic_yang_data):
    # In this test, xlation and revXlation is tested with latest Sonic
    # YANG model.
    test_file = sonic_yang_data['test_file']
    syc = sonic_yang_data['syc']

    jIn = self.readIjsonInput(test_file, 'SAMPLE_CONFIG_DB_JSON')
    jIn = json.loads(jIn)
    numTables = len(jIn)

  syc.loadData(jIn)

tests/libyang-python-tests/test_sonic_yang.py:332:

self = <sonic_yang.SonicYang object at 0x7fc63bb549b0>
configdbJson = {'ACL_RULE': {'V4-ACL-TABLE|DEFAULT_DENY': {'IP_TYPE': 'IPv4ANY', 'PACKET_ACTION': 'DROP', 'PRIORITY': '0'}, 'V4-ACL-T...ter_low_threshold': '70', 'acl_counter_threshold_type': 'percentage', 'ipv6_neighbor_high_threshold': '67', ...}}, ...}
debug = False

def loadData(self, configdbJson, debug=False):

   try:
      # write Translated config in file if debug enabled
      xlateFile = None
      if debug:
          xlateFile = "xlateConfig.json"
      self.jIn = configdbJson
      # reset xlate and tablesWithOutYang
      self.xlateJson = dict()
      self.tablesWithOutYang = dict()
      # self.jIn will be cropped
      self._cropConfigDB()
      # xlated result will be in self.xlateJson
      self._xlateConfigDB(xlateFile=xlateFile)
      #print("load data - {}".format(self.xlateJson))
      self.sysLog(msg="Try to load Data in the tree")
      self.root = self.ctx.parse_data_mem(dumps(self.xlateJson), \
                    ly.LYD_JSON, ly.LYD_OPT_CONFIG|ly.LYD_OPT_STRICT)

   except Exception as e:
       self.root = None
       self.sysLog(msg="Data Loading Failed:{}".format(str(e)), \
        debug=syslog.LOG_ERR, doPrint=True)

     raise SonicYangException("Data Loading Failed\n{}".format(str(e)))

E sonic_yang_ext.SonicYangException: Data Loading Failed
E All Keys are not parsed in LLDP
E dict_keys(['mode', 'enabled'])

sonic_yang_ext.py:725: SonicYangException
----------------------------- Captured stdout call -----------------------------
Read JSON Section: SAMPLE_CONFIG_DB_JSON
sonic_yang(3):All Keys are not parsed in LLDP
dict_keys(['mode', 'enabled'])
sonic_yang(3):exceptionList:[]
sonic_yang(3):Data Loading Failed:All Keys are not parsed in LLDP
dict_keys(['mode', 'enabled'])
===================== 2 failed, 24 passed in 0.50 seconds ======================
[ FAIL LOG END ] [ target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl ]
make: *** [slave.mk:602: target/python-wheels/sonic_yang_mgmt-1.0-py3-none-any.whl] Error 1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@praveen-li @venkatmahalingam - can you please comment on the above failure?

Also there was another issue (see below) which is mentioned in PR description, this was also discussed in yang subgroup, is this addressed or still pending -

Compile sonic_yang_mgmt-1.0-py3-none-any.whl
tests/libyang-python-tests/test_sonic_yang.py - test_xlate_rev_xlate fails as the output generated in revXlateConfig.json has one of the field values "True" instead of "true" which causes failure in comparison.

yang subgroup reference mail dated Apr 9, 2021, subject - sonic_yang_mgmt-1.0-py3-none-any.whl build error when use leaf with type boolean in sample_config_db.json

}
}

container LLDP_PORT {
list LLDP_PORT_LIST {
key "ifname";

leaf ifname {
type leafref {
path "/prt:sonic-port/prt:PORT/prt:PORT_LIST/prt:name";
}
description
"Reference of port on which LLDP to be configured.";
}

leaf enabled {
type boolean;
default true;
description
"Enable/Disable LLDP";
}

leaf mode {
type enumeration {
enum RECEIVE;
enum TRANSMIT;
}

description
"RX/TX mode for LLDP frames";
}
//uses lldp_mode_config;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use the grouping "lldp_mode_config", any reason for this comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compilation fails with the usage of grouping (not related to this PR) hence its commented out, will uncomment once that issue is fixed. This was discussed in the PR review and the relevant information was also shared to yang sub-group over email.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please mention the exact issue.

}
}
}
}