New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

springboot-appengine-standard logging configuration #284

Open
Athas1980 opened this Issue Sep 11, 2017 · 9 comments

Comments

Projects
None yet
9 participants
@Athas1980

Athas1980 commented Sep 11, 2017

Hi I've cloned the project.

The spring boot application does indeed run as advertised. However I've found that if logback-classic is not excluded all the logging goes over logback classic which doesn't get logged in the app engine.

If logback classic is removed and the slf4j-jdk14 is added which I believe means it should go to the same loggers as used by the app engine. The spring application fails to start due to an out of memory error. However mvn appengine:run and running in the eclipse google tools both work correctly with no problems.


01:54:54.100
Uncaught exception from servlet
java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:3332)
	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
	at java.lang.StringBuilder.append(StringBuilder.java:136)
	at java.lang.StringBuilder.append(StringBuilder.java:76)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:484)
	at java.lang.StringBuilder.append(StringBuilder.java:166)
	at java.lang.StringBuilder.append(StringBuilder.java:76)
	at java.util.Formatter$FormatSpecifier.print(Formatter.java:2769)
	at java.util.Formatter.format(Formatter.java:2508)
	at java.util.Formatter.format(Formatter.java:2455)
	at java.lang.String.format(String.java:2942)
	at java.util.logging.SimpleFormatter.format(SimpleFormatter.java:161)
	at java.util.logging.StreamHandler.publish(StreamHandler.java:211)
	at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:116)
	at java.util.logging.Logger.log(Logger.java:738)
	at com.google.apphosting.runtime.NullSandboxPlugin$LogStream.flush(NullSandboxPlugin.java:603)
	at java.io.PrintStream.flush(PrintStream.java:338)
	at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:300)
	at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
	at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
	at java.util.logging.StreamHandler.flush(StreamHandler.java:259)
	at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:117)
	at java.util.logging.Logger.log(Logger.java:738)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:582)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:632)
	at org.apache.commons.logging.impl.SLF4JLocationAwareLog.error(SLF4JLocationAwareLog.java:216)
	at org.springframework.boot.SpringApplication.reportFailure(SpringApplication.java:771)
	at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:748)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314)
	at org.springframework.boot.web.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:151)
	at org.springframework.boot.web.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:131)

Modified pom.xml is as follows.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.google.appengine.demos</groupId>
  <artifactId>springboot-appengine-standard</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>war</packaging>

  <name>springboot-appengine-standard</name>
  <description>Demo project for Spring Boot</description>

  <parent>                              <!-- Get parameters and allow bulk testing -->
    <artifactId>getting-started-java</artifactId>
    <groupId>com.example</groupId>
    <version>1.0.0</version>
    <relativePath>../..</relativePath>
  </parent>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <spring.boot.version>1.5.6.RELEASE</spring.boot.version> <!-- DO NOT UPDATE w/o MANUAL TESTING -->
    <java.version>1.8</java.version>
    <maven.compiler.source>${java.version}</maven.compiler.source>
    <maven.compiler.target>${java.version}</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
      <version>${spring.boot.version}</version>
            <!-- Exclude Tomcat so that it doesn't conflict w/ Jetty server -->
      <exclusions>
        <exclusion>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-tomcat</artifactId>
        </exclusion>
        <exclusion>
        	<groupId>ch.qos.logback</groupId>
        	<artifactId>logback-classic</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    
    <dependency>
    	<groupId>org.slf4j</groupId>
    	<artifactId>slf4j-jdk14</artifactId>
    </dependency>

    <!-- Exclude any jul-to-slf4j -->
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>jul-to-slf4j</artifactId>
      <scope>provided</scope>
    </dependency>

    <!-- Include Servlet API -->
    <dependency>
      <groupId>javax.servlet</groupId>
      <artifactId>javax.servlet-api</artifactId>
      <version>3.1.0</version>
      <scope>provided</scope>
    </dependency>

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <version>${spring.boot.version}</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <dependencyManagement>
     <dependencies>
        <dependency>
            <!-- Import dependency management from Spring Boot -->
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-dependencies</artifactId>
            <version>${spring.boot.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
  </dependencyManagement>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>${spring.boot.version}</version>
      </plugin>
      <!-- [START cloudplugin] -->
      <plugin>
        <groupId>com.google.cloud.tools</groupId>
        <artifactId>appengine-maven-plugin</artifactId>
        <version>1.3.1</version>
        <configuration>
        </configuration>
      </plugin>
      <!-- [END cloudplugin] -->

       <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>versions-maven-plugin</artifactId>
        <version>2.4</version>
        <executions>
          <execution>
            <phase>compile</phase>
            <goals>
              <goal>display-dependency-updates</goal>
              <goal>display-plugin-updates</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <excludes>
            <exclude>javax.servlet:javax.servlet-api</exclude>
            <exclude>com.google.guava:guava</exclude> <!-- avoid android version -->
          </excludes>
        </configuration>
      </plugin>

      <plugin>
        <artifactId>maven-war-plugin</artifactId>
        <version>3.1.0</version>
        <configuration>
          <failOnMissingWebXml>false</failOnMissingWebXml>
        </configuration>
      </plugin>

    </plugins>
  </build>


</project>

Does anyone know how to get the logging working within spring?

@igmartellPP

This comment has been minimized.

Show comment
Hide comment
@igmartellPP

igmartellPP Sep 28, 2017

Any news on this? I have exactly the same issue.

igmartellPP commented Sep 28, 2017

Any news on this? I have exactly the same issue.

@lesv

This comment has been minimized.

Show comment
Hide comment
@lesv

lesv Oct 13, 2017

Contributor

@jabubake Can you take a look at this?

Contributor

lesv commented Oct 13, 2017

@jabubake Can you take a look at this?

@lesv lesv added bug TRIAGED labels Oct 13, 2017

@pgarneau

This comment has been minimized.

Show comment
Hide comment
@pgarneau

pgarneau Dec 1, 2017

I am having exactly the same issue with Spring Boot 1.5.6. We're trying to run on a F4_1G appengine standard machine and still getting the same exception as you with similar setup.

We even removed all "functional" code and only put in a controller that returns "OK" on a healthcheck route and same thing.

pgarneau commented Dec 1, 2017

I am having exactly the same issue with Spring Boot 1.5.6. We're trying to run on a F4_1G appengine standard machine and still getting the same exception as you with similar setup.

We even removed all "functional" code and only put in a controller that returns "OK" on a healthcheck route and same thing.

@pgarneau

This comment has been minimized.

Show comment
Hide comment
@pgarneau

pgarneau Dec 4, 2017

@igmartellPP @Athas1980
So after a few days if trying to find the solution to this problem, we got an answer!

Three things needed to be added to our project:

  1. Inside src/main/resources, adding a logging.properties file with:
.level = INFO
  1. Inside src/main/webapp/WEB-INF/appengine-web.xml, add a config that points to the new logging.properties file. Ex:
<property name="java.util.logging.config.file" value="WEB-INF/classes/logging.properties"/>
  1. Inside src/main/resources/application.yaml, we added an entry:
logging:
    level.ROOT: INFO
    level.org.springframework.test.web.servlet.result: DEBUG

These changes made the "OutOfMemory" exception disappear when our instances try to get launched. Wish you good luck.

pgarneau commented Dec 4, 2017

@igmartellPP @Athas1980
So after a few days if trying to find the solution to this problem, we got an answer!

Three things needed to be added to our project:

  1. Inside src/main/resources, adding a logging.properties file with:
.level = INFO
  1. Inside src/main/webapp/WEB-INF/appengine-web.xml, add a config that points to the new logging.properties file. Ex:
<property name="java.util.logging.config.file" value="WEB-INF/classes/logging.properties"/>
  1. Inside src/main/resources/application.yaml, we added an entry:
logging:
    level.ROOT: INFO
    level.org.springframework.test.web.servlet.result: DEBUG

These changes made the "OutOfMemory" exception disappear when our instances try to get launched. Wish you good luck.

@jabubake

This comment has been minimized.

Show comment
Hide comment
@jabubake

jabubake Dec 11, 2017

Member

@pgarneau : I'll make sure to update the README to include these instructions. Thanks a lot !

Member

jabubake commented Dec 11, 2017

@pgarneau : I'll make sure to update the README to include these instructions. Thanks a lot !

jabubake added a commit that referenced this issue Dec 11, 2017

@jabubake jabubake closed this in da22898 Dec 11, 2017

@ricmf

This comment has been minimized.

Show comment
Hide comment
@ricmf

ricmf Feb 5, 2018

This doesn't work for me. I did as described in pgarneau's comment and still get an out of memory error when initializing the deployed instance.

ricmf commented Feb 5, 2018

This doesn't work for me. I did as described in pgarneau's comment and still get an out of memory error when initializing the deployed instance.

@jabubake jabubake removed their assignment Feb 5, 2018

@jabubake

This comment has been minimized.

Show comment
Hide comment
@jabubake
Member

jabubake commented Feb 5, 2018

@mlev

This comment has been minimized.

Show comment
Hide comment
@mlev

mlev Feb 27, 2018

The key seems to be the location of the logging.properties file. Historically we've followed the Google docs (https://cloud.google.com/appengine/docs/standard/java/logs/) - put it into /WEB-INF and referenced that location in appengine-web.xml. But for us this causes the OutOfMemoryError as described by the OP. However - if we move the same file into src/main/resources then it works!

Not quite sure what's going on. It appears to work even if I don't set the correct location in appengine-web.xml.

mlev commented Feb 27, 2018

The key seems to be the location of the logging.properties file. Historically we've followed the Google docs (https://cloud.google.com/appengine/docs/standard/java/logs/) - put it into /WEB-INF and referenced that location in appengine-web.xml. But for us this causes the OutOfMemoryError as described by the OP. However - if we move the same file into src/main/resources then it works!

Not quite sure what's going on. It appears to work even if I don't set the correct location in appengine-web.xml.

@mrebasti

This comment has been minimized.

Show comment
Hide comment
@mrebasti

mrebasti Mar 2, 2018

We have faced the same problem, and solved it with the help of the instructions from @pgarneau. Also we had to add the line "java.util.logging.ConsoleHandler.level = FINEST" into logging.properties file to see logs of levels DEBUG and TRACE in StackDriver (JDK is configured with "java.util.logging.ConsoleHandler.level = INFO" by default).

The OutOfMemory is caused by the Java Util Logging support of Spring Boot. When the application uses JUL, Spring Boot tries to read the file logging.properties without verifying its existence and without taking the system property java.util.logging.config.file into account (see [1]). And when Spring Boot tries to log the exception in [2] the OutOfMemory is thrown.

Hope this helps.

[1] https://github.com/spring-projects/spring-boot/blob/1e8b9569d3a3900a0ed61712099823ad735b8078/spring-boot/src/main/java/org/springframework/boot/logging/java/JavaLoggingSystem.java#L85
[2] https://github.com/spring-projects/spring-boot/blob/1e8b9569d3a3900a0ed61712099823ad735b8078/spring-boot/src/main/java/org/springframework/boot/SpringApplication.java#L314

mrebasti commented Mar 2, 2018

We have faced the same problem, and solved it with the help of the instructions from @pgarneau. Also we had to add the line "java.util.logging.ConsoleHandler.level = FINEST" into logging.properties file to see logs of levels DEBUG and TRACE in StackDriver (JDK is configured with "java.util.logging.ConsoleHandler.level = INFO" by default).

The OutOfMemory is caused by the Java Util Logging support of Spring Boot. When the application uses JUL, Spring Boot tries to read the file logging.properties without verifying its existence and without taking the system property java.util.logging.config.file into account (see [1]). And when Spring Boot tries to log the exception in [2] the OutOfMemory is thrown.

Hope this helps.

[1] https://github.com/spring-projects/spring-boot/blob/1e8b9569d3a3900a0ed61712099823ad735b8078/spring-boot/src/main/java/org/springframework/boot/logging/java/JavaLoggingSystem.java#L85
[2] https://github.com/spring-projects/spring-boot/blob/1e8b9569d3a3900a0ed61712099823ad735b8078/spring-boot/src/main/java/org/springframework/boot/SpringApplication.java#L314

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment