fuzzy matching for manufacturers based on OUI to NetBox list is not very good

[mmguero]

Malcolm gets a big list of manufacturers from here. Some more we preload from our own list.

During enrichment, particularly when we're doing auto-population we use [fuzzy string matching](https://github.com/idaholab/Malcolm/blob/c78a043033eb24dbf05437c84abb875868325bc9/logstash/ruby/netbox_enrich.rb#L179-L189] to match up the list of NetBox manufacturers with the ]list we use to map MAC address to OUI](https://www.wireshark.org/download/automated/data/manuf). The problem is, they don't use the exact syntax. So, e.g., if NetBox has "Dell" and the OUI is "Dell Inc." then you get stuff like "Dell Inc." matching more closely to "Delta" than "Dell" which is not ideal.

So what can we do? Adjust the thresholds? try to standardize the manuf list on import (maybe filter out the suffixes like LTD, INC, LLC, etc.)? Not sure, but it could certainly be improved.

EDIT: What I've done is allowed the fuzzy string matching threshold to be configurable, and also added some cleaning code that uses some simple rules and patterns to scrub the OUI/manufacturer names before comparison. It's much better now.

[mmguero]

From the updated documentation:

Matching device manufacturers to OUIs

Malcolm's NetBox inventory is prepopulated with a collection of community-sourced device type definitions which can then be augmented by users manually or through preloading. During passive autopopulation device manufacturer is inferred from organizationally unique identifiers (OUIs), which make up the first three octets of a MAC address. The IEEE Standards Association maintains the registry of OUIs, which is not necessarily very internally consistent with how organizations specify the name associated with their OUI entry. In other words, there's not a foolproof programattic way for Malcolm to map MAC address OUI organization names to NetBox manufacturer names, barring creating and maintaining a manual mapping (which would be very large and difficult to keep up-to-date).

Malcolm's NetBox lookup code used in the log enrichment pipeline attempts to match OUI organization names against the list of NetBox's manufacturers using "fuzzy string matching", a technique in which two strings of characters are compared and assigned a similarity score between 0 (completely dissimilar) and 1 (identical). The NETBOX_DEFAULT_FUZZY_THRESHOLD environment variable in netbox-common.env can be used to tune the threshold for determining a match. A fairly high value is recommended (above 0.85; 0.95 is the default) to avoid autopopulating the NetBox inventory with devices with manufacturers that don't actually exist in the network being monitored.

Users may select between two behaviors for when the match threshold is not met (i.e., no manufacturer is found in the NetBox database which closely matches the OUI organization name). This behavior is specified by the NETBOX_DEFAULT_AUTOCREATE_MANUFACTURER environment variable in netbox-common.env:

• NETBOX_DEFAULT_AUTOCREATE_MANUFACTURER=false - the autopopulated device will be created with the manufacturer set to Unspecified
• NETBOX_DEFAULT_AUTOCREATE_MANUFACTURER=true - the autopopulated device will be created along with a new manufacturer entry in the NetBox database set to the OUI organization name