gNMI script results in a segmentation fault if repository is missing a required model

[111pontes]

gNMI script results in a segmentation fault if repository is missing a required model.

$ ./gn-read-xr-shellutil-oper-20-ydk.py ssh://admin:admin@198.18.1.11:57400 -v
2019-01-14 19:02:40,437 - ydk - INFO - gNMIServiceProvider Connected to 198.18.1.11 via Insecure Channel
2019-01-14 19:02:40,439 - ydk - INFO - Executing CRUD read operation on [Cisco-IOS-XR-shellutil-oper:system-time]
2019-01-14 19:02:40,440 - ydk - INFO - 
=============== Get Request Sent ================
path {
  origin: "Cisco-IOS-XR-shellutil-oper"
  elem {
    name: "system-time"
  }
}
encoding: JSON_IETF


2019-01-14 19:02:40,507 - ydk - INFO - 
============= Get Response Received =============
notification {
  timestamp: 1547470569905132305
  update {
    path {
      origin: "Cisco-IOS-XR-shellutil-oper"
      elem {
        name: "system-time"
      }
    }
    val {
      json_ietf_val: "{\"clock\":{\"year\":2019,\"month\":1,\"day\":14,\"hour\":12,\"minute\":56,\"second\":9,\"millisecond\":874,\"wday\":1,\"time-zone\":\"UTC\",\"time-source\":\"calendar\"},\"uptime\":{\"host-name\":\"r1\",\"uptime\":24349}}"
    }
  }
}
error {
}


2019-01-14 19:02:40,508 - ydk - INFO - Get Operation Succeeded
Segmentation fault (core dumped)
$ 

Repository directory with missing model (Cisco-IOS-XR-shellutil-oper.yang):

$ ls -a /home/admin/.ydk/198.18.1.11
.  ..  ietf-netconf@2011-06-01.yang  ydk@2016-02-26.yang
$ 

Script:

#!/usr/bin/env python
#
# Copyright 2016 Cisco Systems, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

"""
Read all data for model Cisco-IOS-XR-shellutil-oper.

usage: gn-read-xr-shellutil-oper-20-ydk.py [-h] [-v] device

positional arguments:
  device         gNMI device (ssh://user:password@host:port)

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose  print debugging messages
"""

from argparse import ArgumentParser
from urlparse import urlparse

from ydk.path import Repository
from ydk.services import CRUDService
from ydk.gnmi.providers import gNMIServiceProvider
from ydk.models.cisco_ios_xr import Cisco_IOS_XR_shellutil_oper \
    as xr_shellutil_oper
import datetime
import textwrap
import os
import logging


def process_system_time(system_time):
    """Process data in system_time object."""
    # format string for system time
    show_system_time = textwrap.dedent("""
        Host: {host}
        System time: {time} {tzone} {date}
        Time source: {source}
        System uptime: {uptime}
        """).strip()

    # create time object
    clock_time = datetime.time(system_time.clock.hour,
                               system_time.clock.minute,
                               system_time.clock.second,
                               system_time.clock.millisecond / 1000)

    # create date object
    clock_date = datetime.date(system_time.clock.year,
                               system_time.clock.month,
                               system_time.clock.day)

    # convert uptime from seconds
    clock_delta = datetime.timedelta(seconds=system_time.uptime.uptime)

    # return formatted string
    return(show_system_time.format(host=system_time.uptime.host_name,
                                   time=clock_time,
                                   tzone=system_time.clock.time_zone,
                                   date=clock_date,
                                   source=system_time.clock.time_source.name,
                                   uptime=clock_delta))


if __name__ == "__main__":
    """Execute main program."""
    parser = ArgumentPgNMI script results in a segmentation fault if repository is missing a required modelarser()
    parser.add_argument("-v", "--verbose", help="print debugging messages",
                        action="store_true")
    parser.add_argument("device",
                        help="gNMI device (ssh://user:password@host:port)")
    args = parser.parse_args()
    device = urlparse(args.device)

    # log debug messages if verbose argument specified
    if args.verbose:
        logger = logging.getLogger("ydk")
        logger.setLevel(logging.INFO)
        handler = logging.StreamHandler()
        formatter = logging.Formatter(("%(asctime)s - %(name)s - "
                                      "%(levelname)s - %(message)s"))
        handler.setFormatter(formatter)
        logger.addHandler(handler)

    # create gNMI session
    repository = Repository(os.path.expanduser("~/.ydk/"+device.hostname))
    provider = gNMIServiceProvider(repo=repository,
                                   address=device.hostname,
                                   port=device.port,
                                   username=device.username,
                                   password=device.password)

    # create CRUD service
    crud = CRUDService()

    system_time = xr_shellutil_oper.SystemTime()  # create object

    # read data from NETCONF device
    system_time = crud.read(provider, system_time)
    print(process_system_time(system_time))  # process object data

    exit()
# End of script

A test case should be added to cover broken repositories.

[ygorelik]

Resolved the issue to YDK-0.8.1.

Test Script

#!/usr/bin/env python
#
# test_856.py
# Testing resolution of GitHub issue 856

# import providers, services and models
from ydk.path import Repository
from ydk.services import CRUDService
from ydk.gnmi.providers import gNMIServiceProvider

from ydk.models.ydktest import openconfig_interfaces

import logging

def enable_logging(level):
    log = logging.getLogger('ydk')
    log.setLevel(level)
    handler = logging.StreamHandler()
    formatter = logging.Formatter(("%(asctime)s - %(name)s - %(levelname)s - %(message)s"))
    handler.setFormatter(formatter)
    log.addHandler(handler)

if __name__ == "__main__":
    """Main execution path"""

    enable_logging(logging.INFO)
    
    # create gNMI session with empty repository
    provider = gNMIServiceProvider(Repository(),
                                      address="127.0.0.1",
                                      port=50051,
                                      username="admin",
                                      password="admin")
    # create CRUD service
    crud = CRUDService()

    # create openconfig interfaces filter
    ifcs = openconfig_interfaces.Interfaces()

    # read interfaces
    get_ifcs = crud.read(provider, ifcs)

The script produces the following log

yan@ubuntu-yan:~/ydk-workspace/ydk-gen$ ./scripts/tests/test_856.py
2019-01-23 14:44:21,667 - ydk - INFO - Path where models are to be downloaded: /home/yan/.ydk/127.0.0.1_50051
2019-01-23 14:44:21,681 - ydk - INFO - gNMIServiceProvider Connected to 127.0.0.1 via Insecure Channel
2019-01-23 14:44:21,682 - ydk - INFO - Executing CRUD read operation on [openconfig-interfaces:interfaces]
2019-01-23 14:44:21,683 - ydk - INFO - 
=============== Get Request Sent ================
path {
  origin: "openconfig-interfaces"
  elem {
    name: "interfaces"
  }
}
encoding: JSON_IETF


2019-01-23 14:44:21,684 - ydk - INFO - 
============= Get Response Received =============
notification {
  timestamp: 1548283461
  update {
    path {
      origin: "openconfig-interfaces"
      elem {
        name: "interfaces"
      }
    }
    val {
      json_ietf_val: "{\"interface\":[{\"name\":\"Loopback10\",\"config\":{\"name\":\"Loopback10\",\"description\":\"Test\"}}]}"
    }
  }
}


2019-01-23 14:44:21,684 - ydk - INFO - Get Operation Succeeded
2019-01-23 14:44:21,685 - ydk - ERROR - Cannot find model with module name 'openconfig-interfaces'
2019-01-23 14:44:21,685 - ydk - ERROR - Data is invalid according to the yang model. Libyang error: Data model "openconfig-interfaces" not found.
2019-01-23 14:44:21,685 - ydk - ERROR - Data is invalid according to the yang model. Libyang error: Unknown element "interfaces".
2019-01-23 14:44:21,685 - ydk - ERROR - Parsing failed with message Unknown element "interfaces".
Traceback (most recent call last):
  File "./scripts/tests/test_856.py", line 41, in <module>
    get_ifcs = crud.read(provider, ifcs)
  File "/usr/local/lib/python2.7/dist-packages/ydk/services/crud_service.py", line 65, in read
    read_top_entity = self._crud.read(provider, top_filters)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/ydk/errors/error_handler.py", line 82, in handle_runtime_error
    _raise(_exc)
  File "/usr/local/lib/python2.7/dist-packages/ydk/errors/error_handler.py", line 56, in _raise
    raise exc
ydk.errors.YCoreError:  YCodecError:Unknown element "interfaces".. Path: 
2019-01-23 14:44:21,699 - ydk - INFO - Disconnected from device
yan@ubuntu-yan:~/ydk-workspace/ydk-gen$ 

Note that instead of Segfault the YDK raises ydk.errors.YCoreError exception.