JSON logging in payara micro #1672

larslorenzen · 2017-06-12T12:53:20Z

Description

Hi, i tried to activate JSON logging in payara micro but could not get it to work.

Expected Outcome

Current Outcome

In version 1: prebootcommand there is no change in the log format
In version 2: postbootcommand there is no change in the log format
In version 3: A slight change in the log format but still no JSON

Steps to reproduce (Only for bug reports)

Version 1

echo "set-log-attributes com.sun.enterprise.server.logging.GFFileHandler.formatter='fish.payara.enterprise.server.logging.JSONLogFormatter'" > prebootcommands.txt && java -jar payara-micro-4.1.2.172.jar --prebootcommandfile prebootcommands.txt

Version 2

echo "set-log-attributes com.sun.enterprise.server.logging.GFFileHandler.formatter='fish.payara.enterprise.server.logging.JSONLogFormatter'" > postbootcommands.txt && java -jar payara-micro-4.1.2.172.jar --postbootcommandfile postbootcommands.txt

Version 3: That's the only time there is a change in the log output format but still no JSON.

cat > logging.properties << EOF
#Payara Micro Logging Properties File

## Handlers
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.formatter=fish.payara.enterprise.server.logging.JSONLogFormatter
java.util.logging.ConsoleHandler.level=FINE

## Global Level
.level=INFO
EOF
java -jar payara-micro-4.1.2.172.jar --logProperties logging.properties

The full payara seems to have the same problem when started in verbose mode ./asadmin start-domain -v domain1

Environment

Payara Version: 4.1.2.172
JDK Version: 1.8.0_72 - Oracle
Operating System: Mac
Database: MongoDB

The text was updated successfully, but these errors were encountered:

leeloo1 · 2017-06-12T13:03:31Z

+1
we need this too :-)

mikecroft · 2017-06-12T13:22:34Z

Hi @larslorenzen - In your Version 3, have you only used your own formatter, or did you try with fish.payara.enterprise.server.logging.JSONLogFormatter set as well?

Before I look into this in any great detail, my thought is that using an asadmin command would not work with a prebootcommandfile, because it's too early in startup; only asadmin set commands will work.

A postbootcommandfile probably wouldn't work since the logging.properties would already have been applied at that point.

There are a couple of ways to "persist" the changes from a postbootcommandfile, though, which would be either to use --rootdir to output to a specific directory which is then reused on the next boot (otherwise a temp directory is generated fresh on each boot) or you could use --outputuberjar my-configured-payara-micro.jar, which you could then use in the same way that you would for any normal payara-micro.jar.

larslorenzen · 2017-06-12T13:31:10Z

Ah yes sorry, was a copy&paste mistake from me, it did not work with the payara logging formatter either.
Changed the initial issue description

smillidge · 2017-06-12T13:36:33Z

Payara Micro logging is different from Payara Server. Payara Micro logging is purely driven by the logging.properties file so 1 and 2 will not work. Not sure why 3 doesn't

larslorenzen · 2017-06-12T13:46:06Z

The full payara has the same behaviour when run in verbose mode ./asadmin start-domain -v domain1.

./asadmin start-domain domain1 
./asadmin set-log-attributes java.util.logging.ConsoleHandler.formatter='fish.payara.enterprise.server.logging.JSONLogFormatter'
./asadmin stop-domain domain1
./asadmin start-domain -v domain1

We are running payara micro inside a docker container on kubernetes. For this i needed to bundle the hazelcast-kubernetes-plugin inside the payara micro jar. After bundling it with the shade plugin, the --outputuberjar did not work anymore.
https://github.com/dreamit-de/payara-micro-hazelcast-k8s
So that is not an option :)

rpofuk · 2017-06-20T12:38:54Z

There are 2 problems:

Payara is taking logging configuration little later then nedded.
If class is not found you drop back to default formater (And this logging does not exist on classpath of payara micro when running as jar)

Quick solution
I fixed it by creating my own JSON logger: https://github.com/loraneo/json-log-formatterr (note: Not logging payara version any more)

just run _maven package_

Dockerfile

...
COPY lib/json-log-formatterr.jar  $PAYARA_PATH/json-log-formatterr.jar


RUN java $JAVA_OPTS \
	 	-Djava.util.logging.config.file=/opt/payara/logging.properties \
	 	-Dcontainer.hostname=$HOSTNAME \
	 	-cp "$PAYARA_PATH/payara-micro.jar:$PAYARA_PATH/json-log-formatterr.jar" \
	 	fish.payara.micro.PayaraMicro \
		--deploy /opt/payara/deployments/<some-war> \
		--logproperties /opt/payara/logging.properties

smillidge · 2017-06-20T20:36:49Z

You could also package this custom JSON Logger and bundle up hazelcast-kubernetes-plugin into an uber jar using the new Payara Micro Maven plugin.
https://docs.payara.fish/documentation/ecosystem/maven-plugin.html

I'll take a look to see why JSON Logger doesn't work.

larslorenzen · 2017-06-20T20:47:45Z

Yes i saw that one also already and wanted to try it out but could not find the plugin artifacts in the maven central repo.

At the moment we package the hazelcast-kubernetes-plugin into the micro payara jar with the maven shade plugin.

  <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>3.0.0</version>
                <executions>
                    <execution>
                        <id>create-archive</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration>
                            <descriptors>
                                <descriptor>src/assembly/hazelcast-kubernetes-discovery.xml</descriptor>
                            </descriptors>
                            <finalName>hazelcast-kubernetes-discovery</finalName>
                            <appendAssemblyId>false</appendAssemblyId>
                        </configuration>
                    </execution>
                    <execution>
                        <id>create-payara</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration>
                            <descriptors>
                                <descriptor>src/assembly/payara-drivers.xml</descriptor>
                            </descriptors>
                            <archive>
                                <manifest>
                                    <mainClass>fish.payara.micro.PayaraMicro</mainClass>
                                </manifest>
                                <manifestEntries>
                                    <Start-Class>fish.payara.micro.impl.PayaraMicroImpl</Start-Class>
                                    <Bundle-SymbolicName>fish.payara.micro</Bundle-SymbolicName>
                                </manifestEntries>
                            </archive>
                            <finalName>payara-micro</finalName>
                            <appendAssemblyId>false</appendAssemblyId>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

and run that resulting jar then with --deploy ROOT.war

The assembly:

<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
    <id>hazelcast-kubernetes</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <unpack>true</unpack>
            <includes>
                <include>com.hazelcast:hazelcast-kubernetes</include>
            </includes>
            <useTransitiveDependencies>true</useTransitiveDependencies>
            <useTransitiveFiltering>true</useTransitiveFiltering>
            <scope>provided</scope>
        </dependencySet>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <unpack>true</unpack>
            <includes>
                <include>org.slf4j:slf4j-jdk14</include>
                <include>org.slf4j:slf4j-api</include>
            </includes>
            <useTransitiveDependencies>false</useTransitiveDependencies>
            <useTransitiveFiltering>true</useTransitiveFiltering>
            <scope>test</scope>
        </dependencySet>
    </dependencySets>
</assembly>

And the payara-drivers:

<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
    <id>packaged-microbadassfish</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <unpack>true</unpack>
            <includes>
                <include>fish.payara.extras:payara-micro</include>
            </includes>
            <scope>provided</scope>
        </dependencySet>
    </dependencySets>
    <files>
        <file>
            <outputDirectory>/MICRO-INF/runtime</outputDirectory>
            <source>target/hazelcast-kubernetes-discovery.jar</source>
        </file>
    </files>

</assembly>

I tried adding a custom json logger the same way but could not get it to work. Maybe providing the logger.jar in the java -cp might do the trick.

smillidge · 2017-06-20T20:55:59Z

I thought we'd pushed the plugin to maven central Doh! I'll get onto it.

leeloo1 · 2017-06-28T10:07:05Z

Hi @smillidge, any status update on this? would be great to have json logging in place for micro.....

mikecroft · 2017-07-13T08:17:54Z

FYI: the Maven plugin is now available in Maven Central

<dependency>
    <groupId>fish.payara.maven.plugins</groupId>
    <artifactId>payara-micro-maven-plugin</artifactId>
    <version>1.0.0</version>
</dependency>

rpofuk · 2017-07-14T10:46:34Z

Another attempt:

https://github.com/loraneo/json-log-formatter

COPY lib/json-log-formater.jar  $PAYARA_PATH/json-log-formatterr.jar

CMD java $JAVA_OPTS \
	 	-Djava.util.logging.config.file=/opt/payara/logging.properties \
	 	-Dcontainer.hostname=$HOSTNAME \
	 	-cp "$PAYARA_PATH/payara-micro.jar:$PAYARA_PATH/json-log-formater.jar" \
	 	fish.payara.micro.PayaraMicro \
		--deploy /opt/payara/deployments/<some-war> \
		--logproperties /opt/payara/logging.properties

Little bit cleaned code.

ljnelson · 2017-08-06T18:38:26Z

(Rookie here.)

Just wanted to add another data point in case it helps. There's also my related StackOverflow question which I suspect is related to this issue.

Running Payara Micro with a --logProperties logging.properties (i.e. no messing about with pre-boot commands, no custom classes, no hand-rolled formatters or anything) where this line is present:

java.util.logging.ConsoleHandler.formatter=fish.payara.enterprise.server.logging.JSONLogFormatter

…also does not work in version 172: the output seems to be the same, in fact, no matter what formatter class is specified. I've tried:

java.util.logging.ConsoleHandler.formatter=fish.payara.enterprise.server.logging.JSONLogFormatter
java.util.logging.ConsoleHandler.formatter=com.sun.enterprise.server.logging.UniformLogFormatter
java.util.logging.ConsoleHandler.formatter=com.sun.enterprise.server.logging.ODLLogFormatter

All result in output that looks like this:

Aug 06, 2017 6:25:07 PM fish.payara.micro.boot.runtime.PayaraMicroRuntimeBuilder build
INFO: Built Payara Micro Runtime

I know that my logging.properties is being picked up, because certain level settings are honored.

Perhaps this is the culprit:

Payara/appserver/extras/payara-micro/payara-micro-core/src/main/java/fish/payara/micro/impl/PayaraMicroImpl.java

Lines 1540 to 1542 in a6a5ea8

    
           for (Handler handler : rootLogger.getHandlers()) { 
        
               handler.setFormatter(new ODLLogFormatter()); 
        
           }

? I'm not sure; just trying to be helpful. 😄

Anyway, hope this helps with the investigation. Thanks for a great product.

smillidge · 2017-08-08T22:27:05Z

This will be fixed in #1835

ljnelson · 2017-11-28T22:23:55Z

The issue described in my comment above is still present in Payara Micro 174.

From debugging into the LogManager, the System classloader is unable to load, for example, fish.payara.enterprise.server.logging.JSONLogFormatter.

That is, as before, my logging.properties file is being picked up, and the java.util.logging.ConsoleHandler.formatter property is being read, and the classname is proper and correct. But the class load operation fails, so SimpleFormatter is used instead.

You can see this if you set a breakpoint in java.util.logging.LogManager's getFormatterProperty(String, Formatter) method.

Note that this comment correctly diagnoses the issue, which is that the fish.payara.enterprise.server.logging.JSONLogFormatter class is not on the classpath of the system classloader when Payara Micro 174 is run via java -jar.

Finally, in case it helps, the following formatter classes are also not loadable by the System classloader:

com.sun.enterprise.server.logging.ODLLogFormatter
com.sun.enterprise.server.logging.UniformLogFormatter

I am using Java 8 if it matters.

ljnelson · 2017-11-28T22:58:40Z

(Ah, looking deeper, it appears that perhaps this set of changes, where you explicitly work around the system classloader requirement, is in effect only if someone does not supply a logging.properties file via the Payara Micro --logProperties command line option. I'm guessing that they should apply in either case.

larslorenzen · 2017-11-30T11:03:25Z

I can confirm that the formatting does not work at all with micro payara 174 if you suppy a --logProperties, even if it's the one that comes with the micro payara.

ApoDBC · 2018-02-09T10:10:45Z

Can we please have an update on this issue? I'm experiencing the same on payara micro 174, and would like to use this in production.

fdreger · 2018-02-09T12:45:22Z

Perhaps it should be reopened? It is obviously not fixed by #1835.

Morten-Staffeldt · 2018-02-13T10:17:33Z

Cant get it to work either, this needs to be reopened.

btrajkovski · 2018-08-08T14:49:23Z

I have payara micro v5 and after a lot of trial and errors, this worked for me:
java -jar payara.jar --deploy build/libs/app.war --noCluster --logProperties logging.properties
And this is the configuration in logging.properties
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.formatter=fish.payara.enterprise.server.logging.JSONLogFormatter
java.util.logging.ConsoleHandler.level=FINEST

mikecroft self-assigned this Jun 12, 2017

mikecroft added 1:Investigating labels Jun 12, 2017

smillidge mentioned this issue Aug 3, 2017

PAYARA-1891 First cut of some Ansi Color on the log formatters #1835

Merged

fturizo assigned smillidge Aug 3, 2017

fturizo added 3:DevInProgress and removed 1:Investigating labels Aug 3, 2017

Pandrex247 closed this as completed in #1835 Sep 8, 2017

fturizo removed the 3:DevInProgress label Sep 8, 2017

smillidge reopened this Feb 16, 2018

smillidge added this to the Payara 5.181 milestone Feb 16, 2018

smillidge mentioned this issue Feb 17, 2018

PAYARA-2512 Ensure formatter added to handler correctly on reading log config #2418

Merged

fturizo added the 3:DevInProgress label Feb 17, 2018

arjantijms closed this as completed in #2418 Feb 18, 2018

fturizo removed the 3:DevInProgress label Feb 18, 2018

btrajkovski unassigned mikecroft Aug 8, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JSON logging in payara micro #1672

JSON logging in payara micro #1672

larslorenzen commented Jun 12, 2017 •

edited

leeloo1 commented Jun 12, 2017

mikecroft commented Jun 12, 2017

larslorenzen commented Jun 12, 2017 •

edited

smillidge commented Jun 12, 2017

larslorenzen commented Jun 12, 2017 •

edited

rpofuk commented Jun 20, 2017 •

edited

smillidge commented Jun 20, 2017 •

edited by mikecroft

larslorenzen commented Jun 20, 2017 •

edited

smillidge commented Jun 20, 2017

leeloo1 commented Jun 28, 2017

mikecroft commented Jul 13, 2017

rpofuk commented Jul 14, 2017 •

edited

ljnelson commented Aug 6, 2017 •

edited

smillidge commented Aug 8, 2017

ljnelson commented Nov 28, 2017 •

edited

ljnelson commented Nov 28, 2017

larslorenzen commented Nov 30, 2017 •

edited

ApoDBC commented Feb 9, 2018

fdreger commented Feb 9, 2018

Morten-Staffeldt commented Feb 13, 2018

btrajkovski commented Aug 8, 2018

JSON logging in payara micro #1672

JSON logging in payara micro #1672

Comments

larslorenzen commented Jun 12, 2017 • edited

Description

Expected Outcome

Current Outcome

Steps to reproduce (Only for bug reports)

Environment

leeloo1 commented Jun 12, 2017

mikecroft commented Jun 12, 2017

larslorenzen commented Jun 12, 2017 • edited

smillidge commented Jun 12, 2017

larslorenzen commented Jun 12, 2017 • edited

rpofuk commented Jun 20, 2017 • edited

smillidge commented Jun 20, 2017 • edited by mikecroft

larslorenzen commented Jun 20, 2017 • edited

smillidge commented Jun 20, 2017

leeloo1 commented Jun 28, 2017

mikecroft commented Jul 13, 2017

rpofuk commented Jul 14, 2017 • edited

ljnelson commented Aug 6, 2017 • edited

smillidge commented Aug 8, 2017

ljnelson commented Nov 28, 2017 • edited

ljnelson commented Nov 28, 2017

larslorenzen commented Nov 30, 2017 • edited

ApoDBC commented Feb 9, 2018

fdreger commented Feb 9, 2018

Morten-Staffeldt commented Feb 13, 2018

btrajkovski commented Aug 8, 2018

larslorenzen commented Jun 12, 2017 •

edited

larslorenzen commented Jun 12, 2017 •

edited

larslorenzen commented Jun 12, 2017 •

edited

rpofuk commented Jun 20, 2017 •

edited

smillidge commented Jun 20, 2017 •

edited by mikecroft

larslorenzen commented Jun 20, 2017 •

edited

rpofuk commented Jul 14, 2017 •

edited

ljnelson commented Aug 6, 2017 •

edited

ljnelson commented Nov 28, 2017 •

edited

larslorenzen commented Nov 30, 2017 •

edited