Cached indexes are not coherent when changes occur to OUI or IAB data

[mpontillo]

Cross posting this from: Launchpad bug #1655452.

---

For some background, see Launchpad bug #1655447.

The netaddr.eui.ieee package is responsible for indexing IEEE data at build time, and generating index files that the netaddr upstream code ships with.

This works fine until you have a long-running process relying on the data. For long-running processes, the index is cached the first time the OUI class is used (or the netaddr.eui.ieee package is imported). That means if netaddr is updated to a more recent version while the process is running, undefined behavior occurs until the process is restarted. This is because the cache becomes out-of-sync with the IEEE data on disk. (For example, we have observed netaddr crashing with an IndexError in Launchpad bug #1655049).

In order to use netaddr in a long-running process, the cache must be re-read from disk if it changes.

Ideally, netaddr could also employ some type of integrity check, such as metadata in the cache file to ensure it is consistent with the data on-disk. Since a thorough integrity check might be expensive enough to defeat the purpose of an on-disk index, a basic sanity check (such as file size; possibly a hash of a subset of the file) should be sufficient.

Alternatively, the on-disk cache of the index could be dropped, and it could be calculated in memory. (Then recalculated if the data file changes.) However, this may be a problem on embedded systems without much CPU power.

---