Relaying from Docker to GELF requires except_fields

[avollmerhaus]

Hi!

I'm trying to use Vector to relay log messages from my local Docker installation to a Graylog server.
I know Docker has native GELF logging capability, but when the Graylog server is unreachable for whatever reason the GELF logging driver prevents Docker from starting containers, unfortunately.
My solution is to scrape the Docker logs with Vector and send them to the Graylog server.

So this is my Vector config:

sources:
  dockerd:
    type: docker_logs

sinks:
  graylog:
    type: "socket"
    inputs: ["dockerd"]
    address: "graylog.example.com:12201"
    mode: "tcp"
    encoding:
      codec: "gelf"

Using it somewhat works, but there are frequent errors like this in the vector log:

vector[19725]: 2023-11-06T15:32:32.746403Z ERROR sink{component_kind="sink" component_id=graylog component_type=socket component_name=graylog}: vector::internal_events::codecs: Failed serializing frame. error=LogEvent contains a value with an invalid type. field = "container_created_at" type = "timestamp" expected type = "string or number" error_type="encoder_failed" stage="sending" internal_log_rate_limit=true

These errors lead to a lot of missing messages in Graylog.
The errors go away when adding except_fields: ["label", "container_created_at"] to the encoding section of the sink, and it doesn't seem like there is anything missing on the Graylog side when omitting these fields.
But I don't fully understand the ramifications.
Should I file a Github issue about this?

[jszwedko]

Hi @avollmerhaus !

This does seem like a possible bug or at least a situation that we should make easier to deal with, would you mind opening an issue? As a workaround, I think you can convert the container_created_at field to a string or integer timestamp with a remap transform that includes .created_at = to_unix_timestamp(timestamp!(.created_at)) (to convert it to a unix timestamp.

[avollmerhaus]

Thanks for the quick feedback!
I've created an issue here: #19114