Skip to content

Writing Application Logs

Jump to bottom
Karsten Schnitter edited this page Apr 12, 2022 · 17 revisions

The Basics

As outlined in the README.md, we do support applications that write logs using the slf4j API in combination with one of the API implementations logback or log4j2.

So you need to decide

  • which logging implementation you use,
  • set your Maven dependencies, and
  • adjust your logging configurations accordingly

As we're talking about application logs in this section, we're only using the core feature and link it with the specific implementation. Since the core feature is pulled as a dependency in these implementation, we only need to add the dependency to the implementation.

Using logback

Maven Dependencies

<!-- pulls cf-java-logging-support-core feature -->
<dependency>
   <groupId>com.sap.hcp.cf.logging</groupId>
   <artifactId>cf-java-logging-support-logback</artifactId>
   <version>${cf-logging-version}</version>
</dependency>
<!-- logback -->
<dependency>
   <groupId>ch.qos.logback</groupId>
   <artifactId>logback-classic</artifactId>
   <version>1.1.3</version>
</dependency>

Logging Configuration

Our logback implementation provides a dedicated logback encoder which you want to configure as your default in your logback.xml like this:

<configuration debug="false" scan="false">
   <appender name="STDOUT-JSON" class="ch.qos.logback.core.ConsoleAppender">
      <encoder class="com.sap.hcp.cf.logback.encoder.JsonEncoder"/>
   </appender>
   <root level="${LOG_ROOT_LEVEL:-WARN}">
      <appender-ref ref="STDOUT-JSON" />
   </root>
</configuration>

There are several extensions to customise the generated log messages. They can be configured by additional elements within the <encoder> tags. The following example shows the available configurations:

<configuration debug="false" scan="false">
   <appender name="STDOUT-JSON" class="ch.qos.logback.core.ConsoleAppender">
      <encoder class="com.sap.hcp.cf.logback.encoder.JsonEncoder">
         <maxStacktraceSize>56320</maxStacktraceSize>
         <sendDefaultValues>false</sendDefaultValues>
	 <contextFieldSupplier>com.sap.hcp.cf.logging.common.serialization.FullVcapEnvFieldSupplier</contextFieldSupplier>
         <logbackContextFieldSupplier>MyLogbackContextFieldSupplier</logbackContextFieldSupplier>
         <charset>utf-8</charset>
         <jsonBuilder>MyJsonBuilder</jsonBuilder>
         <customField>myCustomFieldName</customField>
         <retainField>myCustomFieldName</retainField>
      </encoder>
   </appender>
   <root level="${LOG_ROOT_LEVEL:-WARN}">
      <appender-ref ref="STDOUT-JSON" />
   </root>
</configuration>
XML-Element Default (if omitted) Explanation
<maxStacktraceSize> 56320 Maximal size of an untruncated stacktrace. If the actual size exceeds the value the library will truncate the middle third of the stacktrace to avoid generation of too large messages.
<sendDefaultValues> false Do not send fields with default values, e.g. "-" for strings. By default this keeps messages small. Set to true, if you want to ensure all fields are always emitted.
<contextFieldSupplier> [] Will create an instance of the configured class implementing ContextFieldSupplier to add additional fields to the log messages. To add CloudFoundry environment variables to identify the application add com.sap.hcp.cf.logging.common.serialization.FullVcapEnvFieldSupplier from the example. Note that this information is usually added as envelope data by CF in either syslog structured data or RLP firehose envelops.
<logbackContextFieldSupplier> [] Will create an instance of the configured class implementing LogbackContextFieldSupplier to add additiional fields to the log messages. This interface allows inspection of the logback ILoggingEvent including parsing the original log message.
<charset> utf-8 Sets the encoding of the output stream used for message creation. If the charset is changed, the JSON.Builder most likely needs changing as well.
<jsonBuilder> JSON.builder() Will create an instance of the configured class as JSON.Builder. This allows modification of the generated JSON, e.g. special escaping or formatting.

For documentation on <customField> and <retainField> see the article on Custom Fields.

Note: To retain the log messages from versions before 3.6.0, you need to configure <sendDefaultValues> and <contextFieldSupplier> as in the example.

Using log4j2

Maven Dependencies

<!-- pulls cf-java-logging-support-core feature -->
<dependency>
   <groupId>com.sap.hcp.cf.logging</groupId>
   <artifactId>java-logging-support-log4j2</artifactId>
   <version>${cf-logging-version}</version>
</dependency>
<!-- log4j2 including the slf4j implementation-->
<dependency>
   <groupId>org.apache.logging.log4j</groupId>
   <artifactId>log4j-slf4j-impl</artifactId>
   <version>2.17.2</version>
</dependency>
<dependency>
   <groupId>org.apache.logging.log4j</groupId>
   <artifactId>log4j-core</artifactId>
   <version>2.17.2</version>
</dependency>

Logging Configuration

Our log4j2 implementation provides a dedicated layout which you want to configure as your default in your log4j2.xml like this:

<Configuration status="warn" strict="true" packages="com.sap.hcp.cf.log4j2.converter,com.sap.hcp.cf.log4j2.layout">
   <Appenders>
      <Console name="STDOUT-JSON" target="SYSTEM_OUT" follow="true">
         <JsonPatternLayout />
      </Console>
   <Loggers>
      <Root level="${LOG_ROOT_LEVEL:-WARN}">
         <!-- Use 'STDOUT' instead for human-readable output -->
         <AppenderRef ref="STDOUT-JSON" />
      </Root>
   </Loggers>
</Configuration>

There are several extensions to customise the generated log messages. They can be configured by additional attributes or elements within the <JsonPatternLayout> tag. The following example shows the available configurations:

<Configuration status="warn" strict="true" packages="com.sap.hcp.cf.log4j2.converter,com.sap.hcp.cf.log4j2.layout">
   <Appenders>
      <Console name="STDOUT-JSON" target="SYSTEM_OUT" follow="true">
         <JsonPatternLayout charset="utf-8" jsonBuilder="MyJsonBuilder" maxStacktraceSize="56329" sendDefaultValues="false">
            <contextFieldSupplier class="com.sap.hcp.cf.logging.common.serialization.FullVcapEnvFieldSupplier" />
            <log4jContextFieldSupplier class="MyLog4jContextFieldSupplier" />
            <customField mdcKeyName="custom-field" retainOriginal="true" />
         </JsonPatternLayout>
      </Console>
   <Loggers>
      <Root level="${LOG_ROOT_LEVEL:-WARN}">
         <!-- Use 'STDOUT' instead for human-readable output -->
         <AppenderRef ref="STDOUT-JSON" />
      </Root>
   </Loggers>
</Configuration>
XML-Attribute/Element Default (if omitted) Explanation
charset utf-8 Sets the encoding of the output stream used for message creation. If the charset is changed, the JSON.Builder most likely needs changing as well.
jsonBuilder JSON.builder() Will create an instance of the configured class as JSON.Builder. This allows modification of the generated JSON, e.g. special escaping or formatting.
maxStacktraceSize 56320 Maximal size of an untruncated stacktrace. If the actual size exceeds the value the library will truncate the middle third of the stacktrace to avoid generation of too large messages.
sendDefaultValues false Do not send fields with default values, e.g. "-" for strings. By default this keeps messages small. Set to true, if you want to ensure all fields are always emitted.
<contextFieldSupplier> [] Will create an instance of the configured class implementing ContextFieldSupplier to add additional fields to the log messages. To add CloudFoundry environment variables to identify the application add com.sap.hcp.cf.logging.common.serialization.FullVcapEnvFieldSupplier from the example. Note that this information is usually added as envelope data by CF in either syslog structured data or RLP firehose envelops.
<log4jContextFieldSupplier> [] Will create an instance of the configured class implementing Log4jContextFieldSupplier to add additiional fields to the log messages. This interface allows inspection of the logback ILoggingEvent including parsing the original log message.

For documentation on <customField> and <retainField> see the article on Custom Fields.

Note: To retain the log messages from versions before 3.6.0, you need to configure <sendDefaultValues> and <contextFieldSupplier> as in the example.

Logging Exceptions

Whenever you need to report an exception, you most likely want to include (a part) of the stack trace in that message. To do so, you should simple use the standard pattern and pass the exception object as a parameter:

logger.error("some accompanying message", ex);

This will produce a formatted, single-line, log message which, instead of writing the stack trace as multiple lines, reports the stack trace within an additional array field stacktrace which will make the life of log parsers much easier.

Invoking [http://localhost:8080/log-sample-app/stacktrace] for the sample application will yield a (fairly long) log line like this:

  
{ "written_at":"2016-03-24T11:49:04.861Z","written_ts":31713016750207,"component_type":"application","component_id":"-","space_name":"-","component_name":"-","component_instance":"0","organization_id":"-","correlation_id":"abbabb88-aea4-42cb-a545-e645f2162736","organization_name":"-","space_id":"-","request_id":"-","container_id":"-","type":"log","logger":"com.iamjambay.cloudfoundry.stickysession.MainServlet","thread":"http-bio-8080-exec-8","level":"ERROR","categories":[],"msg":"Exception occured","stacktrace":["java.lang.NullPointerException","at com.iamjambay.cloudfoundry.stickysession.MainServlet.doGet(MainServlet.java:123)","at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)","at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)","at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)","at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)","at com.sap.hcp.cf.logging.servlet.filter.RequestLoggingFilter.doFilterRequest(RequestLoggingFilter.java:95)","at com.sap.hcp.cf.logging.servlet.filter.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:54)","at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)","at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)","at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)","at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)","at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)","at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)","at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)","at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)","at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)","at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)","at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)","at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)","at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)","at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)","at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)","at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)","at java.lang.Thread.run(Thread.java:745)"] }

Logging Categories

We mentioned in the Overview that there is one additional field categories that will allow you to assign categories to individual log messages. You may, e.g., want to tag any log message that deals with database calls with a category db-call, such that later on, log analysis may use this information as an additional filter facet.

In order to fill that field, you create Marker via the MarkerFactory utility class:

Marker dbCall = MarkerFactory.getMarker("db-call");
String logMsg = createMsg(request);

LOGGER.info(dbCall, logMsg);

Note: You can use almost any string for creating a marker, except for those used internally, which are defined in Markers.java.

Custom Fields

This feature is provided for the use with SAP Cloud Platform Application Logging. You can find detailed information on the Custom Fields page.

Clone this wiki locally