Inconsistent handling of dots in field names by json_parser

[tyrken]

When the json_parser transform unpacks JSON with dots in a key name, then it handles field names containing dots inconsistently, e.g. see the below config/setup/output. I'm working around this for now but hope it can be fixed & isn't a designed-in ambiguity...

# Convenience
data_dir = "/tmp"

[sources.in]
  type = "file"
  include = ["./input.log"]
  start_at_beginning = true
  # Make vector read the small input file
  fingerprinting.strategy = "device_and_inode"

[transforms.t1]
  inputs = ["in"]
  type = "json_parser"
  drop_field = true
  drop_invalid = false
  field = "message"
  # Uncomment the target_field to see "correct" behaviour when inserting always under a key
  # target_field = "json"

[sinks.out]
    type = "console"
    inputs = ["t1"]
    target = "stdout"
    encoding = "json"

When you run v0.9.2 vector (on Linux FWIW) with the below input file.

echo '{"field.with.dots": true, "sub_field": {"another.one": false}}' >input.log

... you get the below result (pretty printed for clarity), noting "field.with.dots" is now a three-level hierarchy not a string:

{
    "field": {
        "with": {
            "dots": true
        }
    },
    "file": "input.log",
    "host": "my-host-name",
    "source_type": "file",
    "sub_field": {
        "another.one": false
    },
    "timestamp": "2020-06-15T08:14:00.312871271Z"
}

If you uncomment the target_field to make sure it's always not-inserting-at-root then the problem disappears.

[Hoverbear]

Hey @tyrken this is probably caused in https://github.com/timberio/vector/blob/18717d642ebdbdb4ba22a23dd9b0256ad12a8703/src/transforms/json_parser.rs#L99-L129

As you can see in when target_field is present we drop the object in plainly: https://github.com/timberio/vector/blob/18717d642ebdbdb4ba22a23dd9b0256ad12a8703/src/transforms/json_parser.rs#L111

Where without the field we insert them into the root, which causes them to be parsed as a path! Wow that's not awesome :(

https://github.com/timberio/vector/blob/18717d642ebdbdb4ba22a23dd9b0256ad12a8703/src/transforms/json_parser.rs#L120

We'll need to change that to use insert_flat https://github.com/timberio/vector/blob/18717d642ebdbdb4ba22a23dd9b0256ad12a8703/src/event/mod.rs#L132-L138

[tyrken]

Thanks!