Document "proper" way to add Datadog Tags

[spencergilbert]

We had originally worked on a add_datadog_tag function for VRL, but (IIRC) moved to leverage our existing *_key_value functions that already offered control over delimiters.

If this is our blessed way forward we should document (or add a guide) for how we suggest managing events
Vector sends to Datadog.

Refs:
https://docs.datadoghq.com/agent/vector_aggregation/#overview
https://vector.dev/docs/reference/vrl/functions/#encode_key_value
#7751

[atibdialpad]

++ Observation
I am using Datadog Metrics and Datadog event sinks and I observed that 'tags' behave differently in both.
Datadog Metric Sink : 'tags' are an object/map {"key":value}
Datadog Event Sink: 'tags' are an array of String ["key:value"]

I find the Datadog Event Sink's representation more useful since Datadog internally also treats tags an an array of string. + it allows to multiple keys with different values like ["ip:1.2.3.4", "ip:1.2.3.5"], something which is not possible when tags are an object/dictionary/map.

[jszwedko]

A user noted that DD_TAGS, for the Datadog agent, also allows for space separated tags. They attempted to use space separated tags in Vector but found this didn't work as the Datadog API expects comma separated. We could consider automatically translating.

[spencergilbert]

++ Observation
I am using Datadog Metrics and Datadog event sinks and I observed that 'tags' behave differently in both.
Datadog Metric Sink : 'tags' are an object/map {"key":value}
Datadog Event Sink: 'tags' are an array of String ["key:value"]

I find the Datadog Event Sink's representation more useful since Datadog internally also treats tags an an array of string. + it allows to multiple keys with different values like ["ip:1.2.3.4", "ip:1.2.3.5"], something which is not possible when tags are an object/dictionary/map.

🤔 it would be nice to standardize on that... I'm not sure if there was any reasoning behind why it was done that way.