UserGuideAdvanced

evernat edited this page Jul 20, 2018 · 101 revisions

Advanced documentation of JavaMelody

For standard documentation, see the User's guide

Table of contents

Setup JavaMelody in an ear file

(Written based on a contribution by 'dhartford')

If you deploy your application with a war file or with an equivalent directory, just follow the User's guide.

If you deploy your application with an ear file, probably because you use EJBs, this chapter will guide you to setup JavaMelody in your ear.

1. Jar files

Copy the files javamelody.jar and jrobin-1.5.9.jar to the lib directory of the ear of the application to monitor (lib directory is "required" by the JavaEE 5 specification). If you want to have reports in PDF format or weekly reports by mail, copy also the itext-2.1.7.jar file to the same location. Or if you use Maven, add the javamelody-core and itext dependencies in your pom.xml file.

To declare the jar files in the ear, modify the /META-INF/application.xml file in your ear like this:

<application version="5"
	xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/application_5.xsd">
	....
	<module>
		<java>lib/javamelody.jar</java>
	</module>
	<module>
		<java>lib/jrobin-1.5.9.jar</java>
	</module>
	<module>
		<java>lib/itext-2.1.7.jar</java>
	</module>
	<module>
		<web>
			<web-uri>mywebapp.war</web-uri>
			<context-root>mycontext</context-root>
		</web>
	</module>
	....
</application>

If you use Maven, you may want to use the maven ear plugin and the includeInApplicationXml flag to make this file.

2. web.xml file

You need to have the war of a single monitored webapp in your ear. If you do not have a war, you need to create and deploy a war just for having the monitoring: create it with a simple web.xml file in the WEB-INF directory of the war.

Modify the web.xml file of this war like written in chapter "web.xml file" of the UserGuide.

Make sure that the javamelody.jar file is only in the EAR/lib directory, and not also in the WAR/WEB-INF/lib directory, otherwise you might get issues with class loading (recorded data might be in a different class loader than the one of the monitoring web app.) Note that if you have class loading issues with an ear, the use of a single war instead of an ear could help (and with JavaEE 6, you can even use EJBs in a single war).

3. First results

You can now view the monitoring: deploy your ear and open the following page in a web navigator after starting the server:

http://<host>/<context>/monitoring

where <host> is the name of the server where the application is deployed, followed possibly by the port (for example localhost:8080) and where <context> is the name of the context of the webapp which you have configured in the ear.

Then you can complete the settings as below according to your needs.

4 to 17. Other settings (optional parameters, JDBC, EJB3, etc)

To complete the settings, just follow chapters 4 to 17 of the UserGuide. As you use an ear file, you may be interested by the chapter "Business façades (ejb-jar.xml file if EJB3)" for example.

Optional centralization server setup

A centralized collect server can be used optionally. This server can centralize in a same user interface and with a unique storage the monitoring of several applications (QA and production for example) and / or the monitoring of an application with several instances of servers (a cluster for example). So the reports, the storage and the charts are moved out of the monitored application to the collect server.

Please note that the centralized collect server with javamelody.war only works if the monitoring already works with javamelody.jar in the target application at http://<host>:<port>/<context>/monitoring. An application must not be monitored from two collector servers.

1. War file of the webapp of monitoring

Copy the javamelody.war file from releases to the centralized server. The javamelody.war file should be in general of the same version of monitoring as the version of the javamelody.jar file in the monitored application.

2. Deployment of the webapp of monitoring

Note : An alternative which does not need an application server is described in the following chapter.

Deploy the javamelody.war file in the application server of the centralized server. If it is Tomcat, you can write an xml context file named 'javamelody.xml' in the directory conf/Catalina/localhost of Tomcat as in the following example:

<?xml version="1.0" encoding="UTF-8" ?>
<Context docBase="<pathto>/javamelody.war" path="javamelody" reloadable="false" >
	<Parameter name='javamelody.resolution-seconds' value='120' override='false'/>
</Context>

The optional parameters resolution-seconds, storage-directory, log, warning-threshold-millis, severe-threshold-millis, allowed-addr-pattern, authorized-users, graphite-address, influxdb-url, cloudwatch-namespace, datadog-api-key, statsd-address can be added in the xml context file of the collect server with the prefix javamelody. They can also be added as system properties with the prefix javamelody.

They have the same effects on the collect server than those of a monitored application as written in the UserGuide. In particular the parameter resolution-seconds defines the period of calls to urls of applications from collect server and the resolution of charts in the monitoring. Logs such as connect exceptions are printed in the standard output of the server, and to have more logs you can add the system property -Djavamelody.log=true

The collect server can send weekly, daily or monthly reports by mail for each monitored application. For this, use in the collect server (in Tomcat context for example) exactly the same mail session, the same parameters and the same jar files as those in Weekly, daily or monthly reports by mail.

If xml format is desired instead of java serialization as transport format between a collect server and monitored applications, a parameter transport-format which is specific to the collect server can be added with 'xml' for value. The java serialization is the transport format by default and it is recommended for best performances. The xml transport format needs a dependency on libraries xstream (BSD) and xpp3 (Public Domain) in the monitored webapps.

3. Simpler alternative of deployment of the webapp of monitoring

As an alternative to the deployment described in the last chapter for the webapp of monitoring in a java application server, it is possible to launch the winstone (LGPL) servlet container already included in the javamelody.war file. You only need to use the launch command "java -jar javamelody.war" and to specify in system properties each of the monitoring parameters detailed above.

Examples:

java -jar javamelody.war
java -Djavamelody.resolution-seconds=120 -jar javamelody.war

The settings of http ports, server mode, memory and logs can be done like this:

java -server -Xmx128m -jar javamelody.war --httpPort=8080 --ajp13Port=8009 2>&1 >>javamelody.log

Other parameters exist for the servlet container. You can read them with:

java -jar javamelody.war --usage

On Linux and Unix, it is possible to launch this server as a daemon (in background) and with logs like this (wikipedia):

nohup java -server -jar javamelody.war 0</dev/null 2>&1 >>javamelody.log &

4. Results with a collect server

To view the monitoring: open the following page in a web navigator after starting the servers:

http://<host>/javamelody

where <host> is the name of the collect server, followed possibly by the port (for example localhost:8080) and where 'javamelody' is the name of the context of the webapp as the name of the 'javamelody.xml' file.

With links by application in reports, you can choose the application that you want to monitor.

In the web page, you can add an application to monitor and its access url (for example "http://<host_qa>/myapp/" for "qa" and "http://<host_production>/myapp/" for "production"). If an application to monitor is deployed on several instances of servers (in a cluster or in a farm), urls should be separated by ',' (for example "http://<host1>/myapp/,http://<host2>/myapp/" for "cluster"). Of course, you can otherwise add each instance individually. Note: it is not possible to monitor one application in several collect servers or twice in one collect server.

In order for the collect server to monitor an application, the monitoring should be setup in this application as written at the start of this chapter. When the monitoring is setup in the application, it is not compulsory to include the jrobin-1.5.9.jar file in the WEB-INF/lib directory of the application, as only the collect server will handle the charts for this application.

5. Security with a collect server

And with this setup again, it is possible to restrict access to the monitoring of the application only to the collect server by its ip address, forbidding access to the monitoring of the application to any others. For example, if the monitored application is on the same server as the collect server, the following parameter can be added in the web.xml file of the monitored application to restrict access and to allow only the local collect server:

<init-param>
	<param-name>allowed-addr-pattern</param-name>
	<param-value>127\.0\.0\.1</param-value>
</init-param>

If the monitored application is a Jenkins, JIRA, Confluence or Bamboo server, the system property -Djavamelody.plugin-authentication-disabled=true can be added to the monitored server in order to disable authentication of the monitoring page in the javamelody plugin and to be able to add the server to a centralized collect server. (A system property like -Djavamelody.allowed-addr-pattern=127.0.0.1 can also be added with the ip address of the collect server.)

Instead of using allowed-addr-pattern, for example when you don't know upfront the ip address of the collect server, you may want to secure access with http basic authentication (username and password) in the monitored application. For that, add the following in the web.xml file of the monitored application:

<login-config>
	<auth-method>BASIC</auth-method>
	<realm-name>Monitoring</realm-name>
</login-config>
<security-role>
	<role-name>monitoring</role-name>
</security-role>
<security-constraint>
	<web-resource-collection>
		<web-resource-name>Monitoring</web-resource-name>
		<url-pattern>/monitoring</url-pattern>
	</web-resource-collection>
	<auth-constraint>
		<role-name>monitoring</role-name>
	</auth-constraint>
</security-constraint>

The realm and the user used by the collect server must be defined in the monitored application server, and the user must have the "monitoring" role to have access. For example, if tomcat is used with the default realm in the monitored application server, modify the content of the conf/tomcat-users.xml file as follows:

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
	<role rolename="monitoring"/>
	<user username="monitoring" password="monitoring" roles="monitoring"/>
</tomcat-users>

Or, if you want BASIC authentication with username and password, but do no want to use a realm and "security-constraint" in web.xml, you can add the parameter "authorized-users" in web.xml, in context or in system properties like the other javamelody parameters. For example with a system property -Djavamelody.authorized-users=user1:pwd1,user2:pwd2

Then, when you add the monitored application in the collector server, define the username and the password in the URL. For example, the URL of the monitored application, as given to the collect server, could be: http://myusername:mypassword@myhost:8080/mywebapp

To restrict access to the reports in the collector server, you can also set the -Djavamelody.authorized-users=user1:pwd1,user2:pwd2 system property on the collector server.

Furthermore, if you do not want the people having access to the collect server to be able to add or remove applications to monitor, you can forbid the writing to the file "applications.properties" which contains the list of the applications to monitor. This file is located in the storage directory, "temp/javamelody/applications.properties" for example. The user of the OS running the java process of the collect server must be allowed to read the file but without being allowed to write it. Only you, by using "root" for example, will be able by editing the file to add or to remove applications as the collect server would do. And in this case, the links "add an application" and "remove an application" will not be displayed in the collect server.

6. If reverse proxy

The reports use a Cookie, for example to know the currently displayed application. So if you use Apache as reverse proxy in front of the collector server and if URL are rewritten, the cookie path will not match the path used by the browser. And for example, the charts will not be displayed for the right application.

To fix the cookie path, you may use mod_proxy's top ProxyPassReverseCookiePath directive

7. Automatic registration of applications

When using the optional collect server with ephemeral applications or application's nodes (for example with load-based auto scaling), the addresses and status of the nodes may change dynamically and you may find difficult to manually maintain the list of applications to monitor in the collect server. In that case, you can call an api in your application code to automatically register the application or the application's node when starting and unregister it when stopping. (You need 1.71 in the application and in the collect server for this.)

When starting your webapp:

// name of the application in the collect server (if null, "contextPath_hostname" will be used)
String applicationName = "AwesomeApp";
// url of the collect server
URL collectServerUrl = new URL("http://11.22.33.44:8080");
// url of the application node to be called by the collect server to collect data
URL applicationNodeUrl = new URL("http://myhost:8080/mywebapp");
net.bull.javamelody.MonitoringFilter.registerApplicationNodeInCollectServer(
   applicationName, collectServerUrl, applicationNodeUrl);

When stopping your webapp:

net.bull.javamelody.MonitoringFilter.unregisterApplicationNodeInCollectServer();

You can add basic-authentication in both URLs if needed, for example: URL applicationNodeUrl = new URL("http://myusername:mypassword@myhost:8080/mywebapp");.

Business facades (without EJB3 and without Spring and without Guice)

If the application to monitor contains some business façades with Java interfaces, a counter can be created for statistics of execution of methods on these façades even if these interfaces are neither EJB3 nor Spring nor Guice.

First, if these façades are EJB3, Spring or Guice, then it is simpler to use the configuration described in the user guide. Otherwise let's say that a business façade has an interface with an implementation, which is instanciated in a factory for example as below:

public interface MyFacade { ... }

public class MyFacadeImpl implements MyFacade { ... }

public class Factory {
	public static MyFacade createMyFacade() {
		final MyFacade myFacade = new MyFacadeImpl();
		return myFacade;
	}
}

So it is a standard Java interface (ie POJI or Plain Old Java Interface). Then you just need to modify the code of the instanciation like this:

public class Factory {
	public static MyFacade createMyFacade() {
		final MyFacade myFacade = net.bull.javamelody.MonitoringProxy.createProxy(new MyFacadeImpl());
		return myFacade;
	}
}

As pre-requisites for this, the jar of javamelody must be available in the classpath used to compile sources and façades must have interfaces and not only implementations.

If the name displayed by the monitoring for the class does not please you, you can name the proxy like this:

public class Factory {
	public static MyFacade createMyFacade() {
		final MyFacade myFacade = net.bull.javamelody.MonitoringProxy.createProxy(new MyFacadeImpl(), "my business use case");
		return myFacade;
	}
}

Enable Hotspots detection

When enabled, the Hotspots screen displays CPU hotspots in executed methods for all the JVM.

The overhead of hotspots detection is low: it is based on sampling of stack-traces of threads, without instrumentation. And it is made to be always active if enabled. (It is currently not enabled by default. It may be enabled by default in the future.)

Three parameters can be defined in web.xml, in context or in system properties like the other parameters:

  • "sampling-seconds" to enable the hotspots sampling and to define its period. A period of 10 seconds can be recommended to have the lowest overhead, but then a few hours may be needed to have significant results for a webapp under real use. If you don't mind having a bit more overhead or want a result faster in test, you can use a value of 1 or 0.1 in second for this parameter.
  • "sampling-excluded-packages" to change the list of the excluded packages ("java,sun,com.sun,javax,org.apache,org.hibernate,oracle,org.postgresql,org.eclipse" by default)
  • or "sampling-included-packages" for a white list of included package

MongoDB monitoring

If you use MongoDB V3, you can monitor calls to MongoDB collections (since v1.61).

To enable the MongoDB monitoring, after getting your database in your code

Database database = mongoClient.getDatabase("test")

add the following code:

database = net.bull.javamelody.MongoWrapper.createDatabaseProxy(database)

Then calls to MongoDB will be displayed in the "Services statistics", like "mycollection.find" for example.

JPA monitoring

Inspired by Apache Sirona

If you use JPA with a persistence.xml file, you can monitor main calls on the JPA EntityManager.

To enable the JPA monitoring with JPA statistics in the reports, set net.bull.javamelody.JpaPersistence as JPA Provider in your persistence.xml file (except when using EclipseLink). For example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0"
			xmlns="http://java.sun.com/xml/ns/persistence"
			xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
			xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
			http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
	<persistence-unit name="my-unit">
		<provider>net.bull.javamelody.JpaPersistence</provider>
		...
	</persistence-unit>
</persistence>

If you have in your environment a single "real" JPA provider, it should be found automatically. But if that's not the case or if you want to force the implementation, set the property net.bull.javamelody.jpa.provider to the real implementation you want. For example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0"
			xmlns="http://java.sun.com/xml/ns/persistence"
			xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
			xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
			http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
	<persistence-unit name="my-unit">
		<provider>net.bull.javamelody.JpaPersistence</provider>
		<properties>
			<property name="net.bull.javamelody.jpa.provider" value="org.hibernate.jpa.HibernatePersistenceProvider" />
			...
		</properties>
		...
	</persistence-unit>
</persistence>

Monitoring of sql requests and of jdbc connections in GlassFish v3+

If you use a JDBC DataSource from JNDI with GlassFish v3 or more, the SQL requests and the JDBC connections are probably not monitored.

To monitor them, it is needed to do the following, for each JDBC DataSource:

  • It is supposed that the DataSource is for example currently named "jdbc/MyDataSource" in GlassFish
  • Download javamelody-objectfactory.jar and copy this file in the lib directory of your GlassFish server
  • In the GlassFish admin console, rename your JDBC Resource (DataSource) from "jdbc/MyDataSource" to "jdbc/MyDataSource_uncached" for example, by creating a new identical JDBC Resource then deleting the old one
  • Also in the GlassFish admin console, create a JNDI custom resource like this, without forgetting then "jndi-ref" property:

New Custom Resource

or use the following asadmin command

asadmin create-custom-resource --restype javax.naming.spi.ObjectFactory --factoryclass javamelody.CachedObjectFactory --property jndi-ref=jdbc/MyDataSource_uncached jdbc/MyDataSource

or you can use the asadmin add-resources /path/to/sample-resource.xml command with the file sample-resource.xml

  • Then in the details of the resource in the GlassFish admin console, define the target of the custom resource if the monitored webapp is deployed on another target instance than the Admin Server
  • Test: Webapp uses "jdbc/MyDataSource" as before to lookup from JNDI (and now it is the custom resource referencing "jdbc/MyDataSource_uncached"). After some usage of the webapp, the statistics of the SQL requests should be displayed in the monitoring report.

Usage of JavaMelody in JonAS 5 (which uses OSGI)

JonAS 5 uses OSGI and so you need to add a configuration of felix in JonAS to use JavaMelody.

Copy the content of the latest felix-config.properties for JonAS in a file named "felix-config.properties" and written in the "conf" directory of your JonAS server. If the content of this latest file does not suit your JonAS version, you can find the "felix-config.properties" file for your version in the felix jar file of your server.

In your "felix-config.properties" file, add the following system packages after the existing "org.osgi.framework.system.packages" declaration: "com.sun.management" and "sun.nio.ch".

For example:

org.osgi.framework.system.packages com.sun.management; \
				   sun.nio.ch; \
				   org.osgi.framework; version=1.5.0, \

Then add a system property in your launch file: -Djonas.felix.configuration.file=$JONAS_BASE/conf/felix-config.properties

Deployment on Tomcat without modification of monitored webapps (beta)

If there is one or several webapp(s) to monitor which are deployed on Tomcat 6, it is possible to monitor this or these webapp(s) without modification of this or these webapp(s). That is to say without modification of the war file or of the directory of these webapps.

For this, copy the files javamelody.jar and jrobin-1.5.9.jar and optionally itext-2.1.7.jar in the lib directory of Tomcat 6 (and not in the WEB-INF/lib directories of the webapps).

Then add the following lines in the web.xml file of the conf directory of Tomcat (and not in the WEB-INF/web.xml files of the webapps).

<filter>
	<filter-name>javamelody</filter-name>
	<filter-class>net.bull.javamelody.MonitoringFilter</filter-class>
</filter>
<filter-mapping>
	<filter-name>javamelody</filter-name>
	<url-pattern>/*</url-pattern>
</filter-mapping>
<listener>
	<listener-class>net.bull.javamelody.SessionListener</listener-class>
</listener>

In this conf/web.xml file of tomcat, some parameters can be added as written in the user guide.

So and without modification of the webapps, all webapps deployed in this Tomcat instance are monitored (and it is then not possible to choose which, otherwise than with the url-exclude-pattern parameter described in the user guide). This technique is specific to Tomcat 6+ and does not apply to other JavaEE implementations.

The ClassLoader in application servers is a complex subject. In consequence :

  • This feature is considered as beta for now
  • It works only in Tomcat 6, 7 or 8 (Tomcat 5.5 is not supported)
  • Hot reployment is not supported when using this technique
  • It does not seem advisable to use the monitoring by Spring AOP in this case

Embedding JavaMelody in a standalone app

In general, JavaMelody is installed in a webapp with a server such as Tomcat. But if you want to monitor a standalone application (non webapp and no server), you can embed JavaMelody with a Jetty Server in your application. For that:

  • Add Jetty, and JavaMelody with JRobin dependencies in you application. If you use Maven, you can copy dependencies from this pom.xml.
  • Then, copy this example of EmbeddedServer class in your application.
  • And you can start the server like this. For example, add the following in the main class of your application:
final Map<Parameter, String> parameters = new HashMap<>();
// you can add basic auth:
parameters.put(Parameter.AUTHORIZED_USERS, "admin:pwd");
// you can change the default storage directory:
parameters.put(Parameter.STORAGE_DIRECTORY, "/tmp/javamelody");
// you can enable hotspots sampling with a period of 1 second:
parameters.put(Parameter.SAMPLING_SECONDS, "1.0");

// set the path of the reports:
parameters.put(Parameter.MONITORING_PATH, "/");
// start the embedded http server with javamelody
EmbeddedServer.start(8080, parameters);

When running you application with embedded JavaMelody, open http://localhost:8080 to browse the monitoring reports.

Spring-boot app

Moved to the Spring Boot Starter page.

Debugging logs

The debugging logs, such as javamelody initialization, are displayed at the bottom of the html report.

Logs for debugging are also available using either logback if found, or log4j if found or java.util.logging otherwise. The log messages for debugging are written using the category "net.bull.javamelody" and the level "debug".

Moreover, some stack-traces of unusual exceptions are logged using logback or log4j or java.util.logging, in the same category with the level "info" or "warn".

If the logback library is available in your monitored webapp, you can display debugging logs by adding or modifying the "logback.xml" file in your "WEB-INF/classes" directory. For example:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
	<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
		<encoder>
			<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
			</pattern>
		</encoder>
	</appender>

	<!-- http requests log -->
	<logger name="javamelody">
		<level value="info" />
	</logger>
	
	<!-- debugging log -->
	<logger name="net.bull.javamelody">
		<level value="debug" />
	</logger>

	<root level="debug">
		<appender-ref ref="STDOUT" />
	</root>
</configuration>

Or if the log4j library is available in your monitored webapp, you can display debugging logs by adding or modifying the "log4j.xml" file in your "WEB-INF/classes" directory. For example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" debug="false">
	<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
		<param name="Threshold" value="DEBUG" />
		<layout class="org.apache.log4j.PatternLayout">
			<param name="ConversionPattern" value="%d{ABSOLUTE} %-5p [%c] %m%n" />
		</layout>
	</appender>

	<!-- http requests log  -->
	<category name="javamelody">
		<priority value="info" />
	</category>
	
	<!-- debugging log -->
	<category name="net.bull.javamelody">
		<priority value="debug" />
	</category>
	
	<root>
		<appender-ref ref="CONSOLE" />
	</root>
</log4j:configuration>

Here is an example of debugging logs:

11:11:00.120 [main] DEBUG net.bull.javamelody - JavaMelody listener init started
11:11:00.151 [main] DEBUG net.bull.javamelody - datasources found in JNDI: [java:comp/env/jdbc/TestDB2, java:comp/env/jdbc/TestDB]
11:11:00.167 [main] DEBUG net.bull.javamelody - datasource rebinded: java:comp/env/jdbc/TestDB2 from class org.apache.tomcat.dbcp.dbcp.BasicDataSource to class $Proxy0
11:11:00.167 [main] DEBUG net.bull.javamelody - datasource rebinded: java:comp/env/jdbc/TestDB from class org.apache.tomcat.dbcp.dbcp.BasicDataSource to class $Proxy0
11:11:00.167 [main] DEBUG net.bull.javamelody - JavaMelody listener init done
11:11:00.182 [main] DEBUG net.bull.javamelody - JavaMelody filter init started
11:11:00.182 [main] DEBUG net.bull.javamelody - OS: Windows XP Service Pack 3, x86/32
11:11:00.182 [main] DEBUG net.bull.javamelody - Java: Java(TM) SE Runtime Environment, 1.6.0_21-b06
11:11:00.182 [main] DEBUG net.bull.javamelody - Server: Apache Tomcat/6.0.26
11:11:00.182 [main] DEBUG net.bull.javamelody - Webapp context: /test
11:11:00.182 [main] DEBUG net.bull.javamelody - JavaMelody version: 1.19.0-SNAPSHOT
11:11:00.182 [main] DEBUG net.bull.javamelody - Host: xxx@xxx
11:11:00.182 [main] DEBUG net.bull.javamelody - parameter defined: resolution-seconds=60
11:11:00.182 [main] DEBUG net.bull.javamelody - parameter defined: log=true
11:11:00.182 [main] DEBUG net.bull.javamelody - parameter defined: system-actions-enabled=true
11:11:00.182 [main] DEBUG net.bull.javamelody - parameter defined: mail-session=mail/Session
11:11:00.182 [main] DEBUG net.bull.javamelody - parameter defined: admin-emails=xxx
11:11:00.182 [main] DEBUG net.bull.javamelody - parameter defined: mail-periods=day,week,month
11:11:00.292 [main] DEBUG net.bull.javamelody - log listeners initialized
11:11:00.307 [main] DEBUG net.bull.javamelody - datasources found in JNDI: [java:comp/env/jdbc/TestDB2, java:comp/env/jdbc/TestDB]
11:11:00,386 INFO  [org.quartz.simpl.SimpleThreadPool] Job execution threads will use class loader of thread: main
11:11:00,432 INFO  [org.quartz.core.QuartzScheduler] Quartz Scheduler v.1.5.2 created.
11:11:00,432 INFO  [org.quartz.simpl.RAMJobStore] RAMJobStore initialized.
11:11:00,432 INFO  [org.quartz.impl.StdSchedulerFactory] Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
11:11:00,432 INFO  [org.quartz.impl.StdSchedulerFactory] Quartz scheduler version: 1.5.2
11:11:00.432 [main] DEBUG net.bull.javamelody - job global listener initialized
11:11:00.432 [main] DEBUG net.bull.javamelody - counters initialized
11:11:00.495 [main] DEBUG net.bull.javamelody - counters data read from files in xxx\apache-tomcat-6.0.26\temp\javamelody\test_xxx
11:11:00.511 [main] DEBUG net.bull.javamelody - collect task scheduled every 60s
11:11:00.776 [main] DEBUG net.bull.javamelody - first collect of data done
11:11:00.792 [main] DEBUG net.bull.javamelody - mail report for the day period scheduled with next execution date at Wed Aug 18 00:00:00 CEST 2010
11:11:00.792 [main] DEBUG net.bull.javamelody - mail report for the week period scheduled with next execution date at Sun Aug 22 00:00:00 CEST 2010
11:11:00.792 [main] DEBUG net.bull.javamelody - mail report for the month period scheduled with next execution date at Wed Sep 01 00:00:00 CEST 2010
11:11:00.792 [main] DEBUG net.bull.javamelody - mail reports scheduled for xxx
11:11:00.792 [main] DEBUG net.bull.javamelody - JavaMelody filter init done

Custom reports

You can include links to your custom reports in the floating menu on the right of the report.

For that, add a parameter named "custom-reports" like the other javamelody parameters. In the value of this parameter, put the list of names of your custom reports separated with commas. Then for each custom report, add a parameter with the same name and its path as value.

For example:

<init-param>
	<param-name>custom-reports</param-name>
	<param-value>Send feedback,My custom report</param-value>
</init-param>
<init-param>
	<param-name>Send feedback</param-name>
	<param-value>https://groups.google.com/forum/#!forum/javamelody</param-value>
</init-param>
<init-param>
	<param-name>My custom report</param-name>
	<param-value>/WEB-INF/pages/myCustomReport.jsp</param-value>
</init-param>

If the path starts with '/', it will be considered as a JSP (or a servlet) inside the webapp. Otherwise it will be considered as an URL (http://example.com). As example of JSP, you can add the file "/WEB-INF/pages/myCustomReport.jsp" in your webapp, with a content like this one.

Customizing styles, icons and other resources in the html reports

It can be helpful to customize styles, icons, effects and javascript or help files in the html reports (pdf reports excluded). The reports can be adapted to the styles of some enterprise applications with the CSS file. For example, add the following, before the monitoring filter, in the web.xml file of your webapp, in order to use your own css or icons:

<filter>
	<filter-name>customResourceFilter</filter-name>
	<filter-class>net.bull.javamelody.CustomResourceFilter</filter-class>
	<init-param>
		<param-name>monitoring.css</param-name>
		<param-value>/customMonitoring.css</param-value>
	</init-param>
	<init-param>
		<param-name>bullets/green.png</param-name>
		<param-value>/static/bullets/red.png</param-value>
	</init-param>
</filter>
<filter-mapping>
	<filter-name>customResourceFilter</filter-name>
	<url-pattern>/monitoring</url-pattern>
</filter-mapping>

Then add your files "customMonitoring.css" and "static/bullets/red.png" at the root of the web content in your webapp. You can replace every web resource in this directory and its sub-directories: "src/main/resources/net/bull/javamelody/resource". Then the url "monitoring?resource=monitoring.css" will return the content of your own "customMonitoring.css".

This configuration is for a monitored webapp. For the optional collector server, see issue 414 instead.

Since 1.74, an alternative is to add the /net/bull/javamelody/resource/themedMonitoring.css or /net/bull/javamelody/resource/themedMonitoring.js files as resources in your application to override or add values in the css and js of the monitoring reports. This is the case in the javamelody dark theme which is ready to use as a Maven dependency.

Using a servlet to display the monitoring reports

The MonitoringFilter documented in the user guide is enough to display reports for the "/monitoring" url. You can also change that url with the monitoring-path parameter.

If for some reason, you want to use a servlet to display the reports, then such a servlet already exists and you can use it by adding the following in the WEB-INF/web.xml file of your webapp:

<servlet>
	<servlet-name>monitoringServlet</servlet-name>
	<servlet-class>net.bull.javamelody.ReportServlet</servlet-class>
</servlet>
<servlet-mapping>
	<servlet-name>monitoringServlet</servlet-name>
	<url-pattern>/admin/monitoring</url-pattern>
</servlet-mapping>

You can change the url-pattern above to anything you want, provided that you use the same path in your browser to access the reports.

Sending metrics to Graphite

Metrics are displayed in the charts of the monitoring reports, based on RRD files. As an extra, since 1.68, you can also publish the same metrics to Graphite, if you have a Graphite server installed. Metrics will be sent once per minute (default value of the resolution-seconds parameter) and Graphite will allow custom visualizations in itself or in Grafana. The names of metrics in Graphite are like javamelody.appContext.hostName.metricName, so you will be able to aggregate a metric using wildcards, for example for one application deployed to several hosts: &target=sum(javamelody.appContext.*.metricName).

To enable sending the metrics, add a parameter graphite-address with hostname:port of the Graphite server, in web.xml, in context or in system properties like the other parameters. For example in system properties: -Djavamelody.graphite-address=11.22.33.44:2003.

Sending metrics to InfluxDB

Metrics are displayed in the charts of the monitoring reports, based on RRD files. As an extra like for Graphite, since 1.69, you can also publish the same metrics to InfluxDB, if you have an InfluxDB server installed. Metrics will be sent once per minute (default value of the resolution-seconds parameter) and InfluxDB will allow custom visualizations in Chronograf or in Grafana. The names of metrics in InfluxDB are like javamelody.metricName. Metrics also include application and hostname as tags, so you will be able to filter metrics based on those tags, for example for one application deployed to several hosts.

To enable sending the metrics, add a parameter influxdb-url with the url and db parameter of the InfluxDB server and of the target database in web.xml, in context or in system properties like the other parameters.

For example in system properties: -Djavamelody.influxdb-url=http://11.22.33.44:8086/write?db=mydb&rp=myretentionpolicy (db is required and rp is optional: it will use the DEFAULT retention policy if not specified).

If the InfluxDB server requires basic authentication, include the username and password in the configured URL. For example: -Djavamelody.influxdb-url=http://myusername:mypassword@11.22.33.44:8086/write?db=mydb.

Sending metrics to AWS CloudWatch

Metrics are displayed in the charts of the monitoring reports, based on RRD files. As an extra like for Graphite, since 1.68, you can also publish the same metrics to AWS CloudWatch, if you have AWS EC2 server instance(s) with detailed monitoring in AWS CloudWatch. Metrics will be sent once per minute (default value of the resolution-seconds parameter) and CloudWatch will allow custom visualizations and mail or auto-scaling alarms. The names of metrics in CloudWatch are like javamelody.metricName. Metrics also include application and hostname as dimensions, so you will be able to filter metrics based on those dimensions, for example for one application deployed to several hosts. Note that there is a pricing for CloudWatch metrics (it is supposed that about 25 custom metrics should mean about $10 per month per EC2 instance).

  • To enable sending the metrics, add a parameter cloudwatch-namespace with the CloudWatch namespace, in web.xml, in context or in system properties like the other parameters. For example in system properties: -Djavamelody.cloudwatch-namespace=MyCompany/MyAppDomain (the namespaces starting with AWS/ are reserved for AWS products).
  • AWS CloudWatch dependencies should be included in the webapp (already done in the optional collect server, but not done in JIRA/Confluence/Bamboo/Liferay/Sonar/Alfresco plugins). For example with Maven in your pom.xml file:
	<dependency>
		<groupId>com.amazonaws</groupId>
		<artifactId>aws-java-sdk-cloudwatch</artifactId>
		<version>1.11.136</version>
	</dependency>

Or if Jenkins, download this aws-cloudwatch-library plugin and install it with the Advanced tab of the Plugin manager in Jenkins to add the AWS CloudWatch dependencies.

  • AWS credentials must be provided by either:
    • Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (RECOMMENDED since they are recognized by all the AWS SDKs and CLI except for .NET), or AWS_ACCESS_KEY and AWS_SECRET_KEY (only recognized by Java SDK)
    • Java System Properties - aws.accessKeyId and aws.secretKey
    • Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
    • Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and security manager has permission to access the variable,
    • Instance profile credentials delivered through the Amazon EC2 metadata service
  • AWS region must be provided by either:
    • Environment Variables - AWS_REGION
    • Java System Properties - aws.region
    • Config file at the default location (~/.aws/config)
    • Region delivered through the Amazon EC2 container service
    • Region delivered through the Amazon EC2 metadata service

Sending metrics to Datadog

Metrics are displayed in the charts of the monitoring reports, based on RRD files. As an extra like for Graphite, since 1.71, you can also publish the same metrics to Datadog. Metrics will be sent once per minute (default value of the resolution-seconds parameter) and Datadog dashboards will allow custom visualizations. The names of metrics in Datadog are like javamelody.metricName. Metrics also include host and application as tags.

To enable sending the metrics, add a parameter datadog-api-key with your api key in web.xml, in context or in system properties like the other parameters. For example in system properties: -Djavamelody.datadog-api-key=9775a026f1ca7d1c6c5af9d94d9595a4.

Exposing metrics to Prometheus

As an extra, since 1.70, you can export metrics in a format that can be scraped by Prometheus. Prometheus differs from the other collector integrations in that Prometheus is designed to scrape an available URL rather than receive metrics. The metrics that are exposed by this report are real-time values of the JavaMelody counters, allowing the metrics resolution to be controlled by the Prometheus scrape_interval settings. The metrics then can be visualized via the Prometheus web interface or in Grafana dashboards.

To access the Prometheus formatted metrics, simply request /monitoring?format=prometheus.

Example scrape_config

Here is an example of Prometheus configuration to scrape metrics. See https://prometheus.io/docs/operating/configuration/ for more information.

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'my_app'

    scheme: http
    metrics_path: '/monitoring'

    params:
      format: ['prometheus']

    # basic_auth if the parameter authorized-users is used
    # basic_auth:
    #   username: my_username
    #   password: my_plain_text_password_here

    scrape_interval: 15s

    static_configs:
      - targets: ['myapp:8080']
        labels:
          env: 'unit'

Screenshot of Prometheus: Prometheus

"Last Value" metrics

The other collector integrations export the "lastValue" (the values that make up the "graphs" in the standard JavaMelody report). These values are aggregated internally on a resolution-seconds (default 60) window. Prometheus generally prefers to ingest raw counters and real-time gauges when possible. Because these "lastValue" metrics may be difficult to interpret correctly in Prometheus, and also many are redundant, they are disabled by default. Since 1.73, to enable the "Last Value" metrics, request /monitoring?format=prometheus&includeLastValue=true. That is, change your scrape_config to:

    params:
      format: ['prometheus']
      includeLastValue: ['true']

To enable, set the property prometheus-include-last-value to true in web.xml, in context or system properties the like other parameters. For example, in system properties set -Djavamelody.prometheus-include-last-value=true.

Sending metrics to StatsD

Metrics are displayed in the charts of the monitoring reports, based on RRD files. As an extra like for Graphite, since 1.70, you can also publish the same metrics to StatsD, if you have a StatsD daemon installed. Metrics will be sent once per minute (default value of the resolution-seconds parameter). The names of metrics are like javamelody.appContext.hostName.metricName, so you will be able to filter metrics, for example for one application deployed to several hosts.

To enable sending the metrics, add a parameter statsd-address with hostname:port of the StatsD daemon, in web.xml, in context or in system properties like the other parameters. For example in system properties: -Djavamelody.statsd-address=11.22.33.44:8125.

Exposing data as JMX MBeans

When the javamelody parameter jmx-expose-enabled is set to true, then javamelody mbeans are available with aggregated statistics data about requests. The data given is the statistics since the first start of the server or since the last "Clear" action in the "All" period.

mbeans

The javamelody mbeans can be read with the MBeans screen in 'System actions' of the monitoring report (http://localhost:8080/yourwebapp/monitoring?part=mbeans) or using JMX with the JConsole of the JDK. A mbean value can also be read as text using http, for example:

http://localhost:8080/yourwebapp/monitoring?jmxValue=net.bull.javamelody:type=CounterRequest,context=yourwebapp,name=http.CounterRequestAggregation

You can also get other data, such as last values in the charts, using the External API. For example: http://localhost:8080/yourwebapp/monitoring?part=lastValue&graph=usedMemory

Clearing all statistics and all graphs

The storage files of statistics and of graphs are stored in the temporary directory of the server, unless if you have defined the "storage-directory" parameter. And there is a sub directory for each application.

For example, with tomcat it is the directory <TOMCAT_HOME>/temp/javamelody/. With some other servers on linux, it is the directory /tmp/javamelody/. And for the Jenkins plugin, it is the directory <JENKINS_HOME>/javamelody/.

To clear all statistics and all graphs:

  • Stop the server
  • Delete the storage directory : javamelody in the temporary directory of the server
  • Restart the server. You can then open the report of the monitoring which is now empty.

But if you want to clear only the statistics, delete the *.gz files in the subdirectories of the storage directory. And if you want to clear only the graphs, delete the *.rrd files in the subdirectories of the storage directory.

Report written before last shutdown

Sometimes a server is stopped in emergency or when there is a problem for example with the used memory. And perhaps, you did not thought of saving the JavaMelody report to have precise figures about the state of the server at this moment.

But don't worry, when the application is undeployed, JavaMelody writes a file called last_shutdown.html in the same storage directory as explained in the previous chapter. It is the JavaMelody report in html format, and it contains the statistics for the current day and the system information (such as the used memory) for the current time just before the application is undeployed, but it does not contain charts.

Format of RRD files

Values of charts are stored in RRD (Round Robin Database) format. Many tools exist for this format like RRDtool, close relative of MRTG, or JRobin in Java.

There is a JRobin console, with RRDTool 1.0.x like syntax to view RRD file content. Using the jrobin-1.5.9.jar file and Java 8 (not Java 9) and the command java -jar jrobin-1.5.9.jar, some JRobin console commands are then for example:

last usedMemory.rrd
info usedMemory.rrd
fetch usedMemory.rrd AVERAGE
dump usedMemory.rrd
etc

or in your own java program:

double lastValue = new org.jrobin.core.RrdDb(new File("usedMemory.rrd")).getLastDatasourceValue("usedMemory");

There is also a UI tool to inspect the RRD file content. For example:

java -cp jrobin-1.5.9.jar org.jrobin.inspector.RrdInspector usedMemory.rrd

Note that JRobin format is a bit different than RRDtool format and there is a converter with sources, if you need to use the RRDtool file format.

Summary of parameters

See Parameters.

Compilation and development

See DevGuide