Graceful handling of duplicate host enrollment

[zwass]

Context

Osquery hosts provide a configurable identifier when enrolling to a TLS server. Common values for this are uuid (hardware UUID), hostname, and instance (a UUID generated and stored in the osquery database).

Fleet uses this identifier to determine whether the host in an enrollment request is the same as an existing host.

Problems occur when multiple hosts share the same identifier. This is a relatively common case when using hostname (due to clients with the same hostname) and uuid (due to cloned VMs with the same hardware UUID).

This is an issue that users run into as evidenced by #200 and various reports through Slack and other channels.

Because of this, we have added an enrollment cooldown period (#112) and changed the recommended deployment to use --host_identifier=instance. Using instance does prevent duplicates, but it means that Fleet cannot recognize when the same host re-enrolls after the osquery database being deleted or becoming corrupted.

Reports continue to surface that users are having problems with duplicate identifiers.

Proposal

Osquery provides the full contents of the osquery_info table in each enrollment request, which includes both the equivalent of the uuid and the instance identifiers.

We could choose to look only at the instance identifier for determining uniqueness when a host enrolls. This is likely to resolve duplicate issues without folks having to change the startup flags for osquery (often a difficult task).

On the downside, it is no longer possible to determine when the same host has re-enrolled with a fresh osquery database. This can be mitigated by letting the old host record "age out" and be cleaned up by the offline host cleanup feature.

[zwass]

I've gone ahead with not changing the default, but allowing instance to be configured as the host identifier to use in #417.