proposal to refactor junos_jsnapy

[dgarros]

Hi

I've been playing with the new junos_jsnapy module for some time while and I'm not 100% happy with how it works, especially when some tests failed. I would like to propose another approach for it.

Today the behavior when a test fail is to :

• Mark the task failed when at least 1 jsnapy test FAILs
• Return all informations about the tests executed in JSON formatfor further treatment

This approach as 2 problems from my point of view:

• By default, if a task fails Ansible will not execute remaining tasks. it's possible to workaround that but it does require additional configuration
• By default, all information return by a module reporting as fail, will be printed to the screen. In case of junos_jsnapy, the information returned in case of failure are not easily readable and can represent a large portion of data.

The alternative solution would be to :

• Module Always return Pass, if the execution of the tests are successful, irrespectively of the tests results
• Use a specific Callback plugin to collect test results in JSON and convert them in Human readable format.
• Use a dedicated task at the end to check all tests execution and indicate

I started a prototype, and i will open a pull request soon

In the example below, I'm executing 2 test files on 2 devices. Each test file contains 2 tests.
1 test is reporting fail on QFX5100

The Callback plugin is printing human readable logs after the Summary to indicate what tests failed.

It would be easy to print these logs messages with each task, for now I took the approach to put them at the end.

PLAY [Checking Physical Layer status] ******************************************

TASK [include_vars] ************************************************************
ok: [qfx5100-02]
ok: [qfx5100-01]

TASK [Check BGP neighbor (galaxy/junos_jsnapy)] ********************************
ok: [qfx5100-01]
ok: [qfx5100-02]

TASK [Check Interface (galaxy/junos_jsnapy)] ***********************************
ok: [qfx5100-01]
ok: [qfx5100-02]

TASK [Check JSNAPy tests results] **********************************************
fatal: [qfx5100-01]: FAILED! => {"assertion": "test1.passPercentage == 100", "changed": false, "evaluated_to": false, "failed": true}
ok: [qfx5100-02]

NO MORE HOSTS LEFT *************************************************************
        to retry, use: --limit @pb.check.jsnapy.retry

PLAY RECAP *********************************************************************
qfx5100-01                 : ok=3    changed=0    unreachable=0    failed=1
qfx5100-02                 : ok=4    changed=0    unreachable=0    failed=0


JSNAPy Results for: qfx5100-01 *************************************************
Value of 'peer-state' not 'is-equal' at '//bgp-information/bgp-peer' with {"peer-as": "65200", "peer-state": "Active", "peer-address": "100.0.0.12"}
Value of 'peer-state' not 'is-equal' at '//bgp-information/bgp-peer' with {"peer-as": "60012", "peer-state": "Idle", "peer-address": "192.168.0.2"}

Comments ? Concern ? ideas ?

[vnitinv]

@dgarros makes sense.

Value of 'peer-state' not 'is-equal' at '//bgp-information/bgp-peer' with {"peer-as": "65200", "peer-state": "Active", "peer-address": "100.0.0.12"}

Just wondering what if user want more data, Is there a way they can get them

[dgarros]

Hi @vnitinv

Yes all information are available and returned by the module but they are not printed to the screen by default. You can always uses debug if needed.

To provide more flexibility, it would be great if the log message could be the same as the one provided in the jsnapy test config file. Unfortunately this info is not return by the module currently, I'll open an issue to track that.

Any preference regarding where the log should be printed : At the end or after each task ?

Thanks
Damien

[vnitinv]

Unfortunately this info is not return by the module currently
Yes, we kept that out intentionally. That made the dictionary returned from JSNAPy too bulky.
If that make sense we will add that in 1.1 version.

[vnitinv]

Any preference regarding where the log should be printed : At the end or after each task ?
Can be here https://github.com/Juniper/ansible-junos-stdlib/blob/master/library/junos_jsnapy#L69

[dgarros]

Hi @vnitinv
We can certainly print these messages to a log file but I also see some value to have them "inline"
The good part is that it's easy to support both and to get messages inline it requires the callback plugin anyway so everyone decides

[dgarros]

Closing as the proposal has been merged already