Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs from GKE containers use the legacy resource types #132

Closed
jebbench opened this issue Jul 2, 2020 · 4 comments
Closed

Logs from GKE containers use the legacy resource types #132

jebbench opened this issue Jul 2, 2020 · 4 comments
Assignees
Labels
api: logging Issues related to the googleapis/java-logging-logback API. type: question Request for information or clarification. Not an issue.

Comments

@jebbench
Copy link

jebbench commented Jul 2, 2020

Environment details

  1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
    General, Core, and Other are also allowed as types
  2. OS type and version: Docker container - adoptopenjdk:11-jre-hotspot
  3. Java version: 11
  4. logging-logback version(s): 0.118.0-alpha

Steps to reproduce

  1. Run a container in a GKE cluster configured to use "Kubernetes Operations Logging" which logs using google-cloud-logging-logback
  2. The logs are created using the legacy resource types.

Expexted:

resource: {
  type: "k8s_container"
  labels: {
    cluster_name: "my-cluster"
    project_id: "my-project-XXXXXX"
    pod_name:  "my-pod-XXXXXXXXX-XXXXX"
    location:  "xxx-xxx-x"
    container_name: "my-container"
    namespace_name: "my-namespace"
  }
}

Actual:

resource: {
  type: "container"
  labels: {
    pod_id: "my-pod-XXXXXXXXX-XXXXX"
    namespace_id: ""
    project_id: "my-project-XXXXXX"
    instance_id: "XXX..."
    zone: "xxx-xxx-x"
    cluster_name: "my-cluster"
    container_name: ""
  }
}
@product-auto-label product-auto-label bot added the api: logging Issues related to the googleapis/java-logging-logback API. label Jul 2, 2020
@suraj-qlogic suraj-qlogic self-assigned this Jul 2, 2020
@yoshi-automation yoshi-automation added triage me I really want to be triaged. 🚨 This issue needs some love. labels Jul 3, 2020
@suraj-qlogic
Copy link
Contributor

@jebbench ,Thanks for filling issue.For a Kubernetes container instance set the name of the monitored resource k8s_container as resource_type in logback configuration.Can you please try below configuration.

logback.xml

<configuration>
    <appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">
        <!-- Optional : filter logs at or above a level -->
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        <log>my-log-test</log> <!-- Optional : default java.log -->
        <resourceType>k8s_container</resourceType>
        <flushLevel>WARN</flushLevel> <!-- Optional : default ERROR -->
    </appender>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <root level="debug">
        <appender-ref ref="CLOUD"/>
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>

Code:

public class TestLogger {
  private static final Logger logger = LoggerFactory.getLogger(TestLogger.class);

  public static void main(String[] args) {
    logger.info("Logging INFO");
  }
}

output:

comment_answer

@suraj-qlogic suraj-qlogic added the type: question Request for information or clarification. Not an issue. label Jul 9, 2020
@yoshi-automation yoshi-automation removed 🚨 This issue needs some love. triage me I really want to be triaged. labels Jul 9, 2020
@jebbench
Copy link
Author

jebbench commented Jul 9, 2020

@suraj-qlogic thanks for looking into this.

Adding the setting the resource type correctly sets the resource type however it doesn't fill in any of the resource labels:

resource: {
  type: "k8s_container"
  labels: {
    cluster_name: ""
    pod_name: ""
    container_name: ""
    namespace_name: ""
    location: ""
    project_id: "XXX"
   }
}

My ultimate aim is to have the logs from my Java application correctly associated to the cluster/namespace/pod/container that creates them with the correct log levels.

I can achive the correct association by simply logging to STDOUT however this doesn't correctly pick up the log levels - this library correctly sets the log levels but doesn't associate the logs with the container.

I've tried emiting logs to STDOUT in the logstash JSON format but stackdriver doesn't parse the log level.

@suraj-qlogic
Copy link
Contributor

suraj-qlogic commented Aug 4, 2020

@jebbench ,please follow here java-logging-206 for progress/work related to your request.

@suraj-qlogic
Copy link
Contributor

@jebbench ,I'm closing this bug seems PR-207 was merged and this feature is available on latest version release of java logging client.
Feel free to comment back if for some reason you feel it's not sufficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: logging Issues related to the googleapis/java-logging-logback API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

3 participants