Test Kitchen should provide telemetry

[thommay]

No description provided.

[cheeseplus]

When and how often is the payload being uploaded?

[coderanger]

To copy over comments from meeting: should dive into implementation details at least a bit on how the actual upload will work, and note that the box name or image name for the driver will only be collected if it's a "standard image" (i.e. not something made by the user where the name might reveal info).

[mbacchi]

How would you deal with a standalone client, which may never be connected to the internet? Is there a throttling or exponential backoff process which would cause the system to never try reporting again even though the user may not have configured the anonymous data collection configuration file (i.e. PR #269) to prevent data collection? Please have a way to keep this in check and even disabled automatically in unexpected circumstances.

[thommay]

Yes, the client will do exponential back off, and will probably only do a relatively small number of retries. I think it's reasonable to expect that once a client detects it can't connect to the stats endpoint it'll stop trying for the rest of the lifetime of that process, but I don't think we'd disable the system from ever trying again; that's the user's responsibility.

[davidski]

Echoing my comments from #269 - I emphatically urge this not to be implemented as opt out (opt in is acceptable to me). Test Kitchen is a core piece of my workflow today, but moving to an opt out approach for this will cause me to seek immediate replacements.

[thommay]

This is a complete event as received by the analytics system:

{
  "anonymousId": "8bea3e08-95b0-42cc-9d59-824039e7240e",
  "event": "create",
  "messageId": "node-JtApvzBp1zgmZYJiZmyNJqHa3XPwgTby",
  "properties": {
    "driver_name": "vagrant",
    "driver_version": "1.2.1",
    "host": "darwin16",
    "instanceId": "00000000-0000-0000-0000-000000000000",
    "licenseId": "00000000-0000-0000-0000-000000000000",
    "product": "test-kitchen",
    "target": "bento/ubuntu-14.04",
    "timestamp": "2017-09-06T11:13:10Z",
    "transport": "ssh",
    "vagrant_provider": "virtualbox",
    "vagrant_version": "1.9.7",
    "version": "1.16.0"
  },
  "timestamp": "2017-09-06T11:13:11.968Z",
  "type": "track",
  "sentAt": "2017-09-06T11:13:11.837Z",
  "receivedAt": "2017-09-06T11:13:11.968Z",
  "originalTimestamp": "2017-09-06T11:13:11.837Z"
}

The current branches of this work are at https://github.com/test-kitchen/kitchen-vagrant/compare/tm/telemetry ; https://github.com/test-kitchen/test-kitchen/compare/tm/telemetry to give examples of what an implementation might look at.

[thommay]

Approved by popular vote @chef/rfc-editors

[adamleff]

Accepting RFC 095