Unittest failures with py-junos-eznc 2.3.0

[mnhauke]

I'm actually in the process of packaging napalm for openSUSE.
Part of their package submission policy is to run all unit tests from upstream.

While the unittests run fine with py-junos-eznc 2.2.1 lot's of junos related tests actually fail with the latest version of py-junos-eznc 2.3.0:

test/junos/test_getters.py::TestGetter::test_get_ntp_servers[normal] PASSED [ 65%]
test/junos/test_getters.py::TestGetter::test_get_interfaces[iffdown] FAILED [ 65%]
test/junos/test_getters.py::TestGetter::test_get_interfaces[normal] FAILED [ 65%]
test/junos/test_getters.py::TestGetter::test_get_firewall_policies[no_test_case_found] SKIPPED [ 66%]
test/junos/test_getters.py::TestGetter::test_get_probes_config[normal] PASSED [ 66%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors_detail[srx4100] FAILED [ 66%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors_detail[ex9208_qfx10008] FAILED [ 66%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors_detail[qfx5110] FAILED [ 67%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors_detail[normal_mx_m_t] FAILED [ 67%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors_detail[normal_ex_fx_srx] FAILED [ 67%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors_detail[missing_port_ids] FAILED [ 67%]
test/junos/test_getters.py::TestGetter::test_get_lldp_neighbors[normal] FAILED [ 68%]
test/junos/test_getters.py::TestGetter::test_get_ntp_stats[normal] PASSED [ 68%]
test/junos/test_getters.py::TestGetter::test_get_facts[normal] FAILED    [ 68%]
test/junos/test_getters.py::TestGetter::test_get_bgp_neighbors_detail[normal] FAILED [ 68%]
test/junos/test_getters.py::TestGetter::test_get_bgp_neighbors_detail[inet_inet6_inetflow_iss139] FAILED [ 69%]
test/junos/test_getters.py::TestGetter::test_get_bgp_config[normal] FAILED [ 69%]
test/junos/test_getters.py::TestGetter::test_get_bgp_config[nhs] FAILED  [ 69%]
test/junos/test_getters.py::TestGetter::test_get_users[normal] PASSED    [ 69%]
test/junos/test_getters.py::TestGetter::test_is_alive[normal] PASSED     [ 70%]
test/junos/test_getters.py::TestGetter::test_traceroute[normal] PASSED   [ 70%]
test/junos/test_getters.py::TestGetter::test_get_environment[normal] FAILED [ 70%]
test/junos/test_getters.py::TestGetter::test_get_environment[ex9208] FAILED [ 70%]
test/junos/test_getters.py::TestGetter::test_get_environment[virtualchassis] FAILED [ 71%]
test/junos/test_getters.py::TestGetter::test_get_config[normal] PASSED   [ 71%]
test/junos/test_getters.py::TestGetter::test_get_snmp_information[normal] FAILED [ 71%]
test/junos/test_getters.py::TestGetter::test_get_arp_table[normal] FAILED [ 71%]
test/junos/test_getters.py::TestGetter::test_get_ntp_peers[normal] PASSED [ 72%]
test/junos/test_getters.py::TestGetter::test_get_network_instances[normal] FAILED [ 72%]
test/junos/test_getters.py::TestGetter::test_get_network_instances[bridge] FAILED [ 72%]
test/junos/test_getters.py::TestGetter::test_get_interfaces_counters[normal] FAILED [ 72%]
test/junos/test_getters.py::TestGetter::test_get_bgp_neighbors[normal] FAILED [ 73%]
test/junos/test_getters.py::TestGetter::test_get_bgp_neighbors[use-send-state-pr209] FAILED [ 73%]
test/junos/test_getters.py::TestGetter::test_get_bgp_neighbors[pr_581] FAILED [ 73%]
test/junos/test_getters.py::TestGetter::test_get_mac_address_table[vlan_l2ng] FAILED [ 73%]
test/junos/test_getters.py::TestGetter::test_get_mac_address_table[bridge_domain] FAILED [ 74%]
test/junos/test_getters.py::TestGetter::test_get_mac_address_table[vlan] FAILED [ 74%]
test/junos/test_getters.py::TestGetter::test_get_route_to[normal] FAILED [ 74%]
test/junos/test_getters.py::TestGetter::test_get_ipv6_neighbors_table[normal] FAILED [ 74%]
test/junos/test_getters.py::TestGetter::test_get_interfaces_ip[normal] FAILED [ 75%]
test/junos/test_getters.py::TestGetter::test_get_interfaces_ip[issue111_default_prefixlen] FAILED [ 75%]
test/junos/test_getters.py::TestGetter::test_ping[normal] PASSED         [ 75%]
test/junos/test_getters.py::TestGetter::test_get_arp_table_with_vrf[no_test_case_found] SKIPPED [ 75%]
test/junos/test_getters.py::TestGetter::test_get_config_filtered[normal] PASSED [ 76%]
test/junos/test_getters.py::TestGetter::test_get_optics[normal] FAILED   [ 76%]
test/junos/test_getters.py::TestGetter::test_get_optics[infinite_with_space] FAILED [ 76%]
test/junos/test_getters.py::TestGetter::test_get_probes_results[normal] FAILED [ 76%]
test/junos/test_getters.py::TestGetter::test_method_signatures PASSED    [ 77%]

The issue is reproducable with napalm 2.4.0 and the latest commit
605784c from the develop branch.

Full buildlog attached
python-napalm-Tumbleweed-buildlog-broken-with-py-junos-eznc-2.3.0.txt

[ktbyers]

Okay, yes it looks like something in Junos PyEZ 2.3.0 broke our unit testing/mocking in some way.

I looked at it briefly.

[mirceaulinic]

Thanks for reporting this @mnhauke. Are you aware of any other issues with junos-eznc 2.3.0 (when using NAPALM)?

[ktbyers]

Temporarily fixed using (by pinning to PyEZ 2.2.1

#1061

FYI, I have not been seeing any NAPALM config-test failures against real devices using PyEZ 2.3.0. I run these nightly.

[mnhauke]

I also haven't seen any further issues with junos-znc 2.3.0 (when used NAPALM) so far.