Logs from GKE containers use the legacy resource types

[jebbench]

Environment details

1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
   General, Core, and Other are also allowed as types
2. OS type and version: Docker container - adoptopenjdk:11-jre-hotspot
3. Java version: 11
4. logging-logback version(s): 0.118.0-alpha

Steps to reproduce

1. Run a container in a GKE cluster configured to use "Kubernetes Operations Logging" which logs using google-cloud-logging-logback
2. The logs are created using the legacy resource types.

Expexted:

resource: {
  type: "k8s_container"
  labels: {
    cluster_name: "my-cluster"
    project_id: "my-project-XXXXXX"
    pod_name:  "my-pod-XXXXXXXXX-XXXXX"
    location:  "xxx-xxx-x"
    container_name: "my-container"
    namespace_name: "my-namespace"
  }
}

Actual:

resource: {
  type: "container"
  labels: {
    pod_id: "my-pod-XXXXXXXXX-XXXXX"
    namespace_id: ""
    project_id: "my-project-XXXXXX"
    instance_id: "XXX..."
    zone: "xxx-xxx-x"
    cluster_name: "my-cluster"
    container_name: ""
  }
}

[suraj-qlogic]

@jebbench ,Thanks for filling issue.For a Kubernetes container instance set the name of the monitored resource k8s_container as resource_type in logback configuration.Can you please try below configuration.

logback.xml

<configuration>
    <appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">
        <!-- Optional : filter logs at or above a level -->
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        <log>my-log-test</log> <!-- Optional : default java.log -->
        <resourceType>k8s_container</resourceType>
        <flushLevel>WARN</flushLevel> <!-- Optional : default ERROR -->
    </appender>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <root level="debug">
        <appender-ref ref="CLOUD"/>
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

Code:

public class TestLogger {
  private static final Logger logger = LoggerFactory.getLogger(TestLogger.class);

  public static void main(String[] args) {
    logger.info("Logging INFO");
  }
}

output:

[jebbench]

@suraj-qlogic thanks for looking into this.

Adding the setting the resource type correctly sets the resource type however it doesn't fill in any of the resource labels:

resource: {
  type: "k8s_container"
  labels: {
    cluster_name: ""
    pod_name: ""
    container_name: ""
    namespace_name: ""
    location: ""
    project_id: "XXX"
   }
}

My ultimate aim is to have the logs from my Java application correctly associated to the cluster/namespace/pod/container that creates them with the correct log levels.

I can achive the correct association by simply logging to STDOUT however this doesn't correctly pick up the log levels - this library correctly sets the log levels but doesn't associate the logs with the container.

I've tried emiting logs to STDOUT in the logstash JSON format but stackdriver doesn't parse the log level.

[suraj-qlogic]

@jebbench ,please follow here java-logging-206 for progress/work related to your request.

[suraj-qlogic]

@jebbench ,I'm closing this bug seems PR-207 was merged and this feature is available on latest version release of java logging client.
Feel free to comment back if for some reason you feel it's not sufficient.