json logs are not parsed

[lx223]

Hi there, I am trying to hook my k8s cluster to ship logs to logz using this repo. Currently, I am getting issues where the application's json logs are not parsed.

In Kibana table view: message: {"level":"info","msg":"log message","time":"2020-01-12T15:43:28Z"}

I was expecting it to be parsed recursively like the k8s ones. Can you please help me fix it?

Possibly related to something like this: fluent/fluentd-kubernetes-daemonset#328

[idohalevi]

@lx223 thank you for the detailed issue. @yyyogev can you take a look next week?

[yyyogev]

@lx223 we pushed a test image with a fix, try it and let us know if it works out
try logzio/logzio-k8s:test instead of logzio/logzio-k8s:latest

[lx223]

Hi @yyyogev, thanks for the new image. I just tried it. It worked partially.

Most of the fields are now parsed correctly. But just the message field is missing. For example, the following log entry would contain every field such as level, logger but not the message field.

{"level":"INFO","logger":"io.dropwizard.server.DefaultServerFactory","thread":"main","message":"Regstering jersey handler with root path prefix: /","service-name":"main-service","timestamp":"2020-02-16T15:40:08.367Z"}

[yyyogev]

@lx223 is the message itself moved under another field or dropped entirely?

[lx223]

@yyyogev dropped entirely

[yyyogev]

Probably because they have the same name.. Currently, I don't see a way around this problem

[lx223]

Thanks @yyyogev. I can remap the message field to different name. It's probably worth being documented as it would come as a surprise especially when there are third-party software where json fields cannot be easily remapped. Please let me know when the new change is released.

[yyyogev]

@lx223 you could also use our support to parse the json-as-string from a specific field