fluentd-gcp 2.0.5 doesn't work with json output to stdout

[maisem]

1.6.6 shifted the fluentd-gcp image from 2.0.2 to 2.0.5. This seems to have broken the json detection from stdout. Using 2.0.2 the logs are parsed into jsonPayload, however for 2.0.5 they are passed into textPayload.

Reproduction steps:

1. Create a GKE cluster with master version 1.6.4
2. Deploy a dummy pod to echo some JSON to stdout:

 apiVersion: v1
    kind: Pod
    metadata:
      name: blabbermouth
    spec:
      containers:
      - name: busybox
        image: gcr.io/google-containers/busybox
        command: ['sh', '-c', 'while :; do echo "{\"severity\": \"INFO\", \"message\": \"test\"}"; sleep 5; done']

3. Lookup container logs in StackDriver: resource.type="container" and resource.labels.container_name="busybox"
4. Observe that jsonPayload field of each logEntry is populated with the app's output JSON
5. Upgrade the GKE master to 1.6.6.
6. Observe that the log query in 4 now shows each logEntry with the output in a textPayload.

cc: @jlowdermilk @crassirostris @piosz

[k8s-github-robot]

@maisem There are no sig labels on this issue. Please add a sig label by:
(1) mentioning a sig: @kubernetes/sig-<team-name>-misc
e.g., @kubernetes/sig-api-machinery-misc for API Machinery
(2) specifying the label manually: /sig <label>
e.g., /sig scalability for sig/scalability

Note: method (1) will trigger a notification to the team. You can find the team list here and label list here

[k8s-github-robot]

@maisem There are no sig labels on this issue. Please add a sig label by:
(1) mentioning a sig: @kubernetes/sig-<team-name>-misc
e.g., @kubernetes/sig-api-machinery-misc for API Machinery
(2) specifying the label manually: /sig <label>
e.g., /sig scalability for sig/scalability

Note: method (1) will trigger a notification to the team. You can find the team list here and label list here

[crassirostris]

@maisem Thanks! That is indeed a bug with the new version. I'll patch it in the repo shortly

[crassirostris]

Nope, not a bug with the new version, it was actually introduced with #45734

Because of the first clause in https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/blob/master/lib/fluent/plugin/out_google_cloud.rb#L569 additional field in the record breaks JSON logging