Add option to enable Genie fuzzy syntax matching

[joewesch]

When using the .send_command() method with use_genie=True the command syntax being sent can sometimes be abbreviated (i.e. sh ip int vs show ip interface), but not always (sh ip int br fails). After digging through the Genie code I was able to determine that it was due to the way they score/weigh each word in the command. I initially created an issue on their repo thinking this was a bug, but realized afterward that this was because I was calling their ._fuzzy_search_command() function with fuzzy=False as this is the default and there is no way in Netmiko to enable this option.

It would be nice to have an option on the .send_command() method that can be passed to Genie to enable fuzzy syntax matching.

[ktbyers]

So Genie tries to guess based on weighting what the abbreviated command represents?

Should we just enable this by default? Are there any downsides to do doing this?

I am not seeing much downside in just enabling this.

[joewesch]

I took every command in their parsers.json file and ran them through the ._fuzzy_search_command() with both fuzzy set to False (the current setting) and then True (the proposed change) and here are my findings:

=======================================================

With regards to using full syntax commands (where there are no variables), I was unable to detect a change. Every parser result returned matched exactly the command that I sent in. This is to be expected as the exact syntax matching is weighed higher than partial syntax or variables.

=======================================================

With regards to commands with variables in them, I substituted every variable with the word "variable" (via re.sub(r"{.*?}", "variable", command)). There were 74 false positives when fuzzy was set to False.

For example:

Command: ping variable variable
Expected result: ping {addr} {intf_name}
Result: ping {addr} {count}

Command: show bgp vrf variable variable
Expected result: show bgp vrf {vrf} {route}
Result: show bgp vrf {vrf} {address_family}

Command: show idprom interface variable
Expected result: show idprom interface {mode}
Result: show idprom interface {interface}

Command: show interface variable counters
Expected result: show interface {intf} counters
Result: show interface {interface} counters

# Etc.

However, when changing fuzzy to True it produced the exact same 74 false positives. My rudimentary substitution of the word "variable" is probably not the best way to test this, but it did prove that changing fuzzy to True produced the same false positives as False.

However, was when setting fuzzy to True the following false negatives did occur whereas they had met expectations when fuzzy was set to False:

Command: show log variable | except variable | match variable
Expected result: show log {filename} | except {except_} | match {match}
No Result

Command: show log variable | match variable | except variable
Expected result: show log {filename} | match {match} | except {except_}
No Result

Command: show running-config variable | sec '^i' | inc variable
Expected result: show running-config {pim} | sec '^i' | inc {pip_str}
No Result

Command: show running-config variable | sec variable | inc variable
Expected result: show running-config {pim} | sec {vrf} | inc {pip_str}
No Result

Again, this could be because of the way I am brute force testing, so not 100% sure.

=======================================================

Lastly, I took every command that didn't have a variable in it again and removed the last letter of each word in the command to simulate partial syntax commands at every word. When I ran them through with fuzzy=False it produced 261 failures (244 ambiguous exceptions and 17 false positives).

When I ran it with fuzzy=True it produced 62 failures (61 false positives and 1 false negative). Quite a bit less than previous.

So, we have false positives...but when looking over the false positives all of them were because that last letter of the syntax is indeed needed. For example:

Full command: show bfd sessions
Partial command: sho bf session
Result: show bfd session

Full command: show bgp neighbors
Partial command: sho bg neighbor
Result: show bgp neighbor

Full command: show interfaces summary
Partial command: sho interface summar
Result: show interface summary

And lastly, as I stated earlier it did produce a single false negative:

Full command: show platform software process slot switch active R0 monitor | inc Mem :|Swap:
Partial command: sho platfor softwar proces slo switc activ R0 monito | in Me :|Swap:
No Result

=======================================================

So, in conclusion: setting fuzzy to True by default seems to correct well over 200 ambiguous errors for partial syntax, but may cause a few false negatives by doing so.