Make Ansible JSNAPy output be the same as JSNAPy output

[rogerwiklund]

When I run JSNAPy the output looks like this:

root@scv-utils:~/jsnapy/interface-test# jsnapy --check pre post -f config_mop.yml
*************************** Device: 172.30.105.59 ***************************
Tests Included: show_interface_terse
*********************** Command: show interfaces terse ***********************
Test Failed!! oper-status got changed, before it was <['up']>, now it is <['down']> with name <['ge-1/0/0']> and admin status
FAIL | All "oper-status" is not same in pre and post snapshot [ 48 matched / 1 failed ]
------------------------------- Final Result!! -------------------------------
show_interface_terse : Failed
Total No of tests passed: 0
Total No of tests failed: 1
Overall Tests failed!!!

You can see that test failed because I disabled the interface, and the output prints what interface failed. Ge-1/0/0 in this case.

Using the same test-file with JSNAPy Ansible module just gives me this output:

JSNAPy Results for: mx80-r03-01

---

Value of 'oper-status' not 'no-diff' at 'physical-interface' with {"admin-status": "down", "oper-status": ["down"]}

It does not say what interfaces got affected, how can I make the Ansible JSNAPy output to be the same as JSNAPy?

[jnpr-bowen]

The JSNAPy command options include selecting whether to include the error or info and error messages in the output. To include that option in the jsnapy callback, there needs to be a parameter in the jsnapy module to set this, not just updating the callback.
An additional thing to note is that there is an error that is not just from the err field in the test definition. There is an error from when the XML rpc is able to run, but the text is not able to complete from the XPath not being able run against the data. This could from something like looking at IS-IS neighbors when ISIS is not running, the rpc works, but no data to pulled from the XPath.
I may be able to get change committed soon.

[jnpr-bowen]

So I am trying to working out how to get output of the Ansible module to duplicate the jsnapy command output, and I am having a problem with how to match up the test name to the command/xml that is used for the test. An example I have from jsnapy looks like this:

Connecting to device 192.168.1.2 ................
Taking snapshot of RPC: get_alarm_information
Taking snapshot of RPC: get_software_information
**************************** Device: 192.168.1.2 ****************************
Tests Included: no_alarm_check
RPC is get_alarm_information
PASS | All "no-active-alarms" exists at xpath "/alarm-information/alarm-summary" [ 1 matched ]
**************************** Device: 192.168.1.2 ****************************
Tests Included: version_check
RPC is get_software_information
comment is JUNOS Software Release [12.1X46-D81]. Junos version is not 15.1
FAIL | All "comment" do not contains 15.1" [ 0 matched / 1 failed ]

The output shows that the 'no_alarm_check' test uses the get_alarm_information rpc and 'version_check' test uses the get_software_information. My problem is that the module data does not have a way to match these to up. There is a 'result_dict' that has a dictionary of the key of test names and the value of true/false of whether the test passed, and there is a 'test_details' and 'test_results' dictionaries that are keyed to the command/rpc with the value being the results of the test. Test_details is a defaultdict and can be looped over to get things 'in order', but 'result_dict' is just a dictionary and so can not be looped over in a defined way. I may need the jsnapy module to change the change the 'result_dict' to a defaultdict in order to be sure that the results are matched up to the right test name, or add putting the test name in the result_dict or test_details. Here is an example from my jsnapy test from the output of my Pycharm debugger:

0 = {Operator} <jnpr.jsnapy.operator.Operator instance at 0x104b62710>
device = {str} '192.168.1.2'
log_detail = {dict} <type 'dict'>: {'hostname': '192.168.1.2'}
logger_testop = {Logger} <logging.Logger object at 0x103f73cd0>
no_failed = {int} 1
no_passed = {int} 1
result = {str} 'Failed'
result_dict = {dict} <type 'dict'>: {'version_check': False, 'no_alarm_check': True}

test_details = {defaultdict} defaultdict(<type 'list'>, {'get_alarm_information': [{'xpath': '/alarm-information/alarm-summary', 'count': {'fail': 0, 'pass': 1}, 'testoperation': 'exists', 'node_name': 'no-active-alarms', 'failed': [], 'command': 'get_alarm_information', 'result': True, 'passed': [{'pre': {'no-active-alarms': None}, 'post': {'no-active-alarms': None}, 'message': 'There are no alarms on this system', 'actual_node_value': None, 'id': {}}]}], 'get_software_information': [{'xpath': '/software-information/package-information[name = "junos"]', 'count': {'fail': 1, 'pass': 0}, 'testoperation': 'contains', 'expected_node_value': '15.1', 'node_name': 'comment', 'failed': [{'pre': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'post': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'message': 'comment is JUNOS Software Release [12.1X46-D81]. Junos version is not 15.1', 'actual_node_value': 'JUNOS Software Release [12.1X46-D81]', 'id': {}}], 'command': 'get_software_information', 'res...

  'get_alarm_information' (4373921008) = {list} <type 'list'>: [{'xpath': '/alarm-information/alarm-summary', 'count': {'fail': 0, 'pass': 1}, 'testoperation': 'exists', 'node_name': 'no-active-alarms', 'failed': [], 'command': 'get_alarm_information', 'result': True, 'passed': [{'pre': {'no-active-alarms': None}, 'post': {'no-active-alarms': None}, 'message': 'There are no alarms on this system', 'actual_node_value': None, 'id': {}}]}]

  'get_software_information' (4373918512) = {list} <type 'list'>: [{'xpath': '/software-information/package-information[name = "junos"]', 'count': {'fail': 1, 'pass': 0}, 'testoperation': 'contains', 'expected_node_value': '15.1', 'node_name': 'comment', 'failed': [{'pre': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'post': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'message': 'comment is JUNOS Software Release [12.1X46-D81]. Junos version is not 15.1', 'actual_node_value': 'JUNOS Software Release [12.1X46-D81]', 'id': {}}], 'command': 'get_software_information', 'result': False, 'passed': []}]

default_factory = {type} <type 'list'>
__len__ = {int} 2

test_results = {dict} <type 'dict'>: {'get_alarm_information': [{'xpath': '/alarm-information/alarm-summary', 'count': {'fail': 0, 'pass': 1}, 'testoperation': 'exists', 'node_name': 'no-active-alarms', 'failed': [], 'command': 'get_alarm_information', 'result': True, 'passed': [{'pre': {'no-active-alarms': None}, 'post': {'no-active-alarms': None}, 'message': 'There are no alarms on this system', 'actual_node_value': None, 'id': {}}]}], 'get_software_information': [{'xpath': '/software-information/package-information[name = "junos"]', 'count': {'fail': 1, 'pass': 0}, 'testoperation': 'contains', 'expected_node_value': '15.1', 'node_name': 'comment', 'failed': [{'pre': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'post': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'message': 'comment is JUNOS Software Release [12.1X46-D81]. Junos version is not 15.1', 'actual_node_value': 'JUNOS Software Release [12.1X46-D81]', 'id': {}}], 'command': 'get_software_information', 'result': False, 'passed': []...

'get_alarm_information' (4373921008) = {list} <type 'list'>: [{'xpath': '/alarm-information/alarm-summary', 'count': {'fail': 0, 'pass': 1}, 'testoperation': 'exists', 'node_name': 'no-active-alarms', 'failed': [], 'command': 'get_alarm_information', 'result': True, 'passed': [{'pre': {'no-active-alarms': None}, 'post': {'no-active-alarms': None}, 'message': 'There are no alarms on this system', 'actual_node_value': None, 'id': {}}]}]

'get_software_information' (4373918512) = {list} <type 'list'>: [{'xpath': '/software-information/package-information[name = "junos"]', 'count': {'fail': 1, 'pass': 0}, 'testoperation': 'contains', 'expected_node_value': '15.1', 'node_name': 'comment', 'failed': [{'pre': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'post': {'comment': 'JUNOS Software Release [12.1X46-D81]'}, 'message': 'comment is JUNOS Software Release [12.1X46-D81]. Junos version is not 15.1', 'actual_node_value': 'JUNOS Software Release [12.1X46-D81]', 'id': {}}], 'command': 'get_software_information', 'result': False, 'passed': []}]

__len__ = {int} 2

testname_results = {str} 'Traceback (most recent call last):\n File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 166, in _getPyDictionary\n attr = getattr(var, n)\n File "/Users/bbowen/PycharmProjects/mytest/venv/lib/python2.7/site-p

[jnpr-bowen]

Found a problem in the jsnapy code that broke the association between command/rpc used in the test and the jsnapy test name. As soon as that gets corrected, I can have a callback that produces the kind of output you would like.
My view of the jsnapy callback would not 'exactly' match up with the default jsnapy output. Keeping the test operator and the test details is difficult to understand. My callback would have a way of presenting the err and info message in the output. Since the test developer decided that is the info that is useful to present. Whether the info or error messages would be presented could be controlled by adding a var to the jsnapy task.

[premeshshah]

HI team , is there any update on the issue ?

[keepworld]

Anyone managed to have message info in jsnapy callback plugin ?