Possible incorrect JSON representation of log.level for Pino log format

[hectcastro]

Using v1.3.0 of ecs-pino-format, I get the following output for a message:

{
  "log.level": "info",
  "@timestamp": "2021-08-26T14:45:06.489Z",
  "labels": {
    "env": "prod",
    "app": "test"
  },
  "service": {
    "name": "test"
  },
  "tags": [
    "audit"
  ],
  "ecs": {
    "version": "1.6.0"
  },
  "message": "Hello world"
}

But, it seems like log.level should be an Object instead of a string key, like:

{
  "log": {
    "level": "info"
  },
  "@timestamp": "2021-08-26T14:46:10.475Z",
  "labels": {
    "env": "prod",
    "app": "test"
  },
  "service": {
    "name": "test"
  },
  "tags": [
    "audit"
  ],
  "ecs": {
    "version": "1.6.0"
  },
  "message": "Hello world"
}

Below is the example program in TypeScript I was using to test:

import ecsFormat from "@elastic/ecs-pino-format";
import pino from "pino";

const log = pino({
  base: {
    labels: {
      env: "prod",
      app: "test",
    },
    service: {
      name: "test",
    },
    tags: ["audit"],
  },
  ...ecsFormat(),
});

log.info("Hello world");

[trentm]

@hectcastro Thanks for asking.

Ecs-logging made the decision to allow deep fields to be expressed with dotted names. So these two are equivalent:

{"foo.bar": "baz"}
{"foo": {"bar": "baz}}

And the ecs-logging spec expresses a preference for "log.level" to be a dotted field name.

The typical processor of ECS log records is Elasticsearch which, as of version 5.x, supports dotted fields names in this way. The filebeat configuration instructions for ecs-pino-format suggest using json.expand_keys: true for this.

I'm going to close this, but please feel free to re-open it if I've misinterpreted or you still have concerns/questions.

[hectcastro]

Makes sense. Thanks for the clarification, @trentm. Just started getting familiar with the ECS ecosystem a few days ago, so thanks for the pointers too.

In my use case, the initial/short-term processor for logs will be CloudWatch Logs. I believe that I'll have to ensure some expand_keys type of functionality exists in our clients since the logs will flow directly to CloudWatch Logs from Lambda (no opportunity for filebeat).

[trentm]

In my use case, the initial/short-term processor for logs will be CloudWatch Logs.

Ah. Thanks for mentioning use case.

[felixbarny]

See also elastic/ecs-logging-java#51 (comment) on how to use the dot_expander processor in an Elasticsearch ingest node pipeline.