Skip to content
Permalink
Browse files
Update docs to describe log4j setup
Previously we used logback for logging but this changed to be log4j when
we moved to karaf.  As we now only support karaf most logging is now
done using the log4j framework by default.

This commit updates the docs to reflect the default use of log4j,
  • Loading branch information
Duncan Grant committed Mar 5, 2019
1 parent c127d40 commit 7197fb27d5d7cdcc1628e96d1ef84b19bad61bb9
Showing 2 changed files with 46 additions and 130 deletions.
@@ -4,10 +4,10 @@ title: Logging

## Logging: A Quick Overview

For logging, we use **logback** which implements the slf4j API.
For logging, we use **log4j** which implements the slf4j API.
This means you can use any slf4j compliant logging framework,
with a default configuration which just works out of the box
and bindings to the other common libraries (``java.util.logging``, ``log4j``, ...)
and bindings to the other common libraries (``java.util.logging``, ``logback``, ...)
if you prefer one of those.


@@ -29,114 +29,43 @@ then you can modify the config file `etc/org.ops4j.pax.logging.cfg` before hand.
For more information check
[https://ops4j1.jira.com/wiki/display/paxlogging/Configuration](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration).

#### Karaf Log commands

### Classic - non-OSGI based Apache Brooklyn

To use:

* **Users**:
If using a brooklyn binary installation, simply edit the ``logback.xml``
or ``logback-custom.xml`` supplied in the archive, sometimes in a ``conf/``
directory.

* **Developers**:
When setting up a new project, if you want logging it is recommended to include
the ``brooklyn-logback-xml`` project as an *optional* and *provided* maven dependency,
and then to put custom logging configuration in either ``logback-custom.xml`` or ``logback-main.xml``,
as described below.


#### Customizing Your Logging

The project ``brooklyn-logback-xml`` supplies a ``logback.xml`` configuration,
with a mechanism which allows it to be easily customized, consumed, and overridden.
You may wish to include this as an *optional* dependency so that it is not forced
upon downstream projects. This ``logback.xml`` file supplied contains just one instruction,
to include ``logback-main.xml``, and that file in turn includes:

* ``logback-custom.xml``
* ``brooklyn/logback-appender-file.xml``
* ``brooklyn/logback-appender-stdout.xml``
* ``brooklyn/logback-logger-excludes.xml``
* ``brooklyn/logback-debug.xml``

For the most common customizations, simply create a ``logback-custom.xml`` on your classpath
(ensuring it is loaded *before* brooklyn classes in classpath ordering in the pom)
and supply your customizations there:

```xml
<included>
<!-- filename to log to -->
<property name="logging.basename" scope="context" value="acme-app" />

<!-- additional loggers -->
<logger name="com.acme.app" level="DEBUG"/>
</included>
```

For other configuration, you can override individual files listed above.
For example:

* To remove debug logging, create a trivial ``brooklyn/logback-debug.xml``,
containing simply ``<included/>``.
* To customise stdout logging, perhaps to give it a threshhold WARN instead of INFO,
create a ``brooklyn/logback-appender-stdout.xml`` which defines an appender STDOUT.
* To discard all brooklyn's default logging, create a ``logback-main.xml`` which
contains your configuration. This should look like a standard logback
configuration file, except it should be wrapped in ``<included>`` XML tags rather
than ``<configuration>`` XML tags (because it is included from the ``logback.xml``
which comes with ``brooklyn-logback-xml``.)
* To redirect all jclouds logging to a separate file include ``brooklyn/logback-logger-debug-jclouds.xml``.
This redirects all logging from ``org.jclouds`` and ``jclouds`` to one of two files: anything
logged from Brooklyn's persistence thread will end up in a `persistence.log`, everything else
will end up in ``jclouds.log``.

You should **not** supply your own ``logback.xml`` if you are using ``brooklyn-logback-xml``.
If you do, logback will detect multiple files with that name and will scream at you.
If you wish to supply your own ``logback.xml``, do **not** include ``brooklyn-logback-xml``.
(Alternatively you can include a ``logback.groovy`` which causes logback to ignore ``logback.xml``.)

You can set a specific logback config file to use with:

```bash
-Dlogback.configurationFile=/path/to/config.xml
```


#### Assemblies

When building an assembly, it is recommended to create a ``conf/logback.xml`` which
simply includes ``logback-main.xml`` (which comes from the classpath). Users of the assembly
can then edit the ``logback.xml`` file in the usual way, or they can plug in to the configuration
mechanisms described above, by creating files such as ``logback-custom.xml`` under ``conf/``.

Including ``brooklyn-logback-xml`` as an *optional* and *provided* dependency means everything
should work correctly in IDE's but it will not include the extra ``logback.xml`` file in the assembly.
(Alternatively if you include the ``conf/`` dir in your IDE build, you should exclude this dependency.)

With this mechanism, you can include ``logback-custom.xml`` and/or other files underneath
``src/main/resources/`` of a project, as described above (for instance to include custom
logging categories and define the log file name) and it should get picked up,
both in the IDE and in the assembly.

Logging commands are available through the karaf console. These let you interact with the logs and dynamically change
logging configuration in a running application.

#### Tests
Some useful log: commands are:

Brooklyn projects ``test`` scope includes the ``brooklyn-utils-test-support`` project
which supplies a ``logback-test.xml``. logback uses this file in preference to ``logback.xml``
when available (ie when running tests). However the ``logback-test.xml`` Brooklyn uses
includes the same ``logback-main.xml`` call path above, so your configurations should still work.
log:display mylogger -p "%d - %c - %m%n" - Show the log entries for a specific logger with a different pattern.

The only differences of the ``logback-test.xml`` configuration is that:
log:get/set - Show / set the currently configured log levels

* Debug logging is included for all Brooklyn packages
* The log file is called ``brooklyn-tests.log``
log:tail - As display but will show continuously

log:exception-display - Display the last exception

#### Caveats
#### Bundles

You can capture logs from a specific bundle or set of bundles and e.g. write that to a different file.

log4j.appender.sift=org.apache.log4j.sift.MDCSiftingAppender
log4j.appender.sift.key=myBundle
log4j.appender.sift.default=karaf
log4j.appender.sift.appender=org.apache.log4j.FileAppender
log4j.appender.sift.appender.layout=org.apache.log4j.PatternLayout
log4j.appender.sift.appender.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %m%n
log4j.appender.sift.appender.file=${karaf.data}/log/mybundle.debug.log
log4j.appender.sift.appender.append=true

For a detailed reference to the sift appender see [Karaf Advanced configuration](https://karaf.apache.org/manual/latest/#_advanced_configuration)

* logback uses SLF4J version 1.6 which is **not compatible** with 1.5.x.
If you have dependent projects using 1.5.x (such as older Grails) things may break.
#### Tests

For unit testing, where no karaf context exits, Brooklyn uses logback. Brooklyn project's ``test`` scope includes the ``brooklyn-utils-test-support`` project
which supplies a ``logback-test.xml``. logback uses this file in preference to ``logback.xml``
when available (ie when running tests).

#### Caveats

* If you're not getting the logging you expect in the IDE, make sure
``src/main/resources`` is included in the classpath.
@@ -8,40 +8,29 @@ Brooklyn uses the SLF4J logging facade, which allows use of many popular framewo
The convention for log levels is as follows:

* `ERROR` and above: exceptional situations which indicate that something has unexpectedly failed or
some other problem has occured which the user is expected to attend to
some other problem has occurred which the user is expected to attend to
* `WARN`: exceptional situations which the user may which to know about but which do not necessarily indicate failure or require a response
* `INFO`: a synopsis of activity, but which should not generate large volumes of events nor overwhelm a human observer
* `DEBUG` and lower: detail of activity which is not normally of interest, but which might merit closer inspection under certain circumstances.

Loggers follow the ``package.ClassName`` naming standard.


## Using Logback through OSGi Pax Logging

In the OSGi based Apache Brooklyn logging is configured from ops4j pax logging.

See: [Logging - OSGi based Apache Brooklyn]({{book.path.docs}}/dev/tips/logging.md#osgi-based-apache-brooklyn) <br/>
[https://ops4j1.jira.com/wiki/display/paxlogging/Configuration](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration)

The default logging is to write INFO+ messages to `brooklyn.info.log`,
and DEBUG+ to `brooklyn.debug.log`. Each is a rolling log file,
where the past 10 files will be kept. INFO level, and above, messages
will be logged to the karaf console. Use the `log:` commands in the
karaf client, e.g. `log:tail`, to read these messages.

## Standard Configuration

A `logback.xml` file is included in the `conf/` directly of the Brooklyn distro;
this is read by `brooklyn` at launch time. Changes to the logging configuration,
such as new appenders or different log levels, can be made directly in this file
or in a new file included from this.
A `org.ops4j.pax.logging.cfg` file is included in the `etc/` directly of the Brooklyn distro;
this is read by `brooklyn` at launch time. Changes to the logging configuration,
such as new appenders or different log levels, can be made directly in this file.

Karaf logging is highly configurable. For example enable the sift appender to log to separate log files for
each bundle as described here: [Advanced configuration](https://karaf.apache.org/manual/latest/#_advanced_configuration)

## Advanced Configuration

The default `logback.xml` file references a collection of other log configuration files
included in the Brooklyn jars. It is necessary to understand the source structure
in the [logback-includes]({{book.url.brooklyn_sever_git}}/{{"master" if 'SNAPSHOT' in book.brooklyn_version else book.brooklyn_version}}/logging/logback-includes) project.

For example, to change the debug log inclusions, create a folder `brooklyn` under `conf`
and create a file `logback-debug.xml` based on the
[brooklyn/logback-debug.xml]({{book.url.brooklyn_sever_git}}/{{"master" if 'SNAPSHOT' in book.brooklyn_version else book.brooklyn_version}}/logging/logback-includes/src/main/resources/brooklyn/logback-debug.xml)
from that project.
A full explanation of logging in karaf is available [here](https://karaf.apache.org/manual/latest/#_log).


## Log File Backup
@@ -65,15 +54,13 @@ For example (on mac):

## Logging aggregators

Integration with systems like Logstash and Splunk is possible using standard logback configuration.
Logback can be configured to [write to the syslog](http://logback.qos.ch/manual/appenders.html#SyslogAppender),
Integration with systems like Logstash and Splunk is possible using standard log4j configuration.
Log4j can be configured to write to syslog using the SyslogAppender
which can then [feed its logs to Logstash](http://www.logstash.net/docs/1.4.2/inputs/syslog).


## For More Information

The following resources may be useful when configuring logging:

* The [logback-includes]({{book.url.brooklyn_sever_git}}/{{"master" if 'SNAPSHOT' in book.brooklyn_version else book.brooklyn_version}}/logging/logback-includes) project
* [Brooklyn Developer Guide]({{book.path.docs}}/dev/tips/logging.md) logging tips
* The [Logback Project](http://logback.qos.ch/) home page
* [OPS4J Pax Logging](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration)

0 comments on commit 7197fb2

Please sign in to comment.