ScriptsAndAlerts

evernat edited this page Jun 28, 2017 · 30 revisions

If you have a webapp monitored by JavaMelody, this pages documents how to get monitoring data using scripts and how to send alerts based on thresholds. We will ask Jenkins for help with this. (Version 1.57 or later is needed.)

Note that if you need scripts and alerts about Jenkins master or slaves instead of a monitored webapp, see this page.

Jenkins

To run scripts and send alerts, you will need Jenkins:

  1. Download jenkins.war and run it: java -jar jenkins.war
  2. Install the Monitoring plugin (v1.57 or later): On the Jenkins home page, as a Jenkins administrator, click "Manage Jenkins", "Manage plugins", the "Available" tab, find the "Monitoring plugin" with the "Filter" box and check the box, click "Install without restart".
  3. To execute a script periodically in a Jenkins job, install the Groovy plugin: in the available plugins again, find and check "Groovy plugin", click "Install without restart".

You can run groovy scripts with the Jenkins script console: click "Manage Jenkins" then "Script console", typically at http://server/jenkins/script

For each script, it is needed to write the url of the monitored webapp such as url = "http://myserver:8080/mywebapp/monitoring";

If your webapp has BASIC authentication for security using "security-constraint" in web.xml or using the parameter authorized-users, include username:password in the url such as url = "http://myusername:mypassword@myserver:8080/mywebapp/monitoring";

To execute a script periodically:

  1. Create a freestyle job in Jenkins by clicking "New item".
  2. Check "Build periodically" and write a schedule. For example, */15 * * * * will run the job every 15 minutes.
  3. Add a build step "Execute system Groovy script" and write a script such as one below.
  4. You can also configure "Discard old builds" and write a description.
  5. Save.
  6. Click "Build now" to test it.

Scripts

You can execute these scripts either in the Jenkins script console or in a Jenkins job as said above. You can get from the monitored webapp, the same data as with the External API, but with objects in scripts. The monitoring values displayed can be used for thresholds of alerts as said at the bottom of this page.

Display some memory data

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
java = new RemoteCall(url).collectJavaInformations();
memory = java.memoryInformations;
println "\nused memory:\n    " + Math.round(memory.usedMemory / 1024 / 1024) + " Mb";
println "\nmax memory:\n    " + Math.round(memory.maxMemory / 1024 / 1024) + " Mb";
println "\nused perm gen:\n    " + Math.round(memory.usedPermGen / 1024 / 1024) + " Mb";
println "\nmax perm gen:\n    " + Math.round(memory.maxPermGen / 1024 / 1024) + " Mb";
println "\nused non heap:\n    " + Math.round(memory.usedNonHeapMemory / 1024 / 1024) + " Mb";
println "\nused physical memory:\n    " + Math.round(memory.usedPhysicalMemorySize / 1024 / 1024) + " Mb";
println "\nused swap space:\n    " + Math.round(memory.usedSwapSpaceSize / 1024 / 1024) + " Mb";

Display some JVM data

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
java = new RemoteCall(url).collectJavaInformations();
println "\nsessions count:\n    " + java.sessionCount;
println "\nactive HTTP threads count:\n    " + java.activeThreadCount;
println "\nthreads count:\n    " + java.threadCount;
println "\nsystem load average:\n    " + java.systemLoadAverage;
println "\nsystem cpu load:\n    " + java.systemCpuLoad; // since 1.59
println "\navailable processors:\n    " + java.availableProcessors;
println "\nhost:\n    " + java.host;
println "\nos:\n    " + java.os;
println "\njava version:\n    " + java.javaVersion;
println "\njvm version:\n    " + java.jvmVersion;
println "\npid:\n    " + java.pid;
println "\nserver info:\n    " + java.serverInfo;
println "\ncontext path:\n    " + java.contextPath;
println "\nstart date:\n    " + java.startDate;

Display some graph last values

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
long usedMemory = new RemoteCall(url).collectGraphLastValue("usedMemory") / 1024 / 1024;
println "usedMemory = " +  usedMemory + " Mb";
double cpu = new RemoteCall(url).collectGraphLastValue("cpu");
println "cpu = " + cpu;
double gc = new RemoteCall(url).collectGraphLastValue("gc");
println "gc = " + gc;

Display some MBean attributes values

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
exampleAttributes = "java.lang:type=OperatingSystem.ProcessCpuTime|java.lang:type=Memory.HeapMemoryUsage";
println exampleAttributes + " = " + new RemoteCall(url).collectMBeanAttribute(exampleAttributes);

Many JMX / MBean attributes are available. From the monitoring report, open the MBeans report and copy the attribute name which you want from the link on "-", on the left of the attribute.

Display a simple threads dump

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
java = new RemoteCall(url).collectJavaInformations();
threads = java.getThreadInformationsList();
println threads.size() + " threads (" + java.activeThreadCount + " http threads active):";
for (thread in threads) {
  println "";
  println thread;
  for (s in thread.getStackTrace())
    println "    " + s;
}

Display deadlocked threads

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
java = new RemoteCall(url).collectJavaInformations();
threads = java.getThreadInformationsList();
deadlocked = new java.util.ArrayList();
for (thread in threads) {
  if (thread.deadlocked)
    deadlocked.add(thread);
}
println deadlocked.size() + " deadlocked threads / " + threads.size() + " threads (" + java.activeThreadCount + " http threads active)";
for (thread in deadlocked) {
  println "";
  println thread;
  for (s in thread.getStackTrace())
    println "    " + s;
}

Display HTTP sessions

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
sessions = new RemoteCall(url).collectSessionInformations(null);
println sessions.size() + " sessions:";
for (session in sessions) {
    println session;
}

Display heap histogram (object instances per class)

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
heapHisto = new RemoteCall(url).collectHeapHistogram();
classes = heapHisto.classes;
println "class    instances    bytes    source";
println "=====================================";
for (c in classes) {
    println c.name + "    " + c.instancesCount + "    " + c.bytes + "    " + c.source;
}

Display requests having mean time greater than severe threshold

(By default, severe threshold = 2 x stddev of all requests durations and warning threshold = 1 x stddev)

import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
counterName = "http"; // or "sql" for example
list = new RemoteCall(url).collectData();
for (object in list) {
   if (object.getClass().getSimpleName().equals("Counter") 
         && object.getName().equals(counterName)) {
      aggreg = new CounterRequestAggregation(object);
      for (request in aggreg.getRequests()) {
         if (request.getMean() >= aggreg.getSevereThreshold() 
            || request.getCpuTimeMean() >= aggreg.getSevereThreshold()) {
                println(request.getName()
                   + ", counter=" + object.getName()
                   + ", hits=" + request.getHits()
                   + ", mean=" + request.getMean()
                   + ", max=" + request.getMaximum()
                   + ", stddev=" + request.getStandardDeviation()
                   + ", cpuTimeMean=" + request.getCpuTimeMean()
                   + ", systemErrorPercentage=" + request.getSystemErrorPercentage());
         }
      }
   }
}

Other values

You can also get other values, such as statistics, current requests, processes, database reports, opened connections, hotspots and more. See the methods in the source.

Alerts

You can send alerts with a Jenkins job, using the Groovy plugin.

Suppose that you want to check every 15 minutes, if the latest GC is above 20% or if the system load average is above 50 or if the active threads count is above 100 or if there are deadlocked threads:

  • In Jenkins, create a freestyle job in Jenkins by clicking "New item".
  • Check "Build periodically" and write a schedule, */15 * * * * for example.
  • Add a build step "Execute system Groovy script" and write a script:
import net.bull.javamelody.*;
import net.bull.javamelody.internal.model.*;

url = "http://myserver:8080/mywebapp/monitoring";
double cpu = new RemoteCall(url).collectGraphLastValue("cpu");
println "cpu = " + cpu;
double gc = new RemoteCall(url).collectGraphLastValue("gc");
println "gc = " + gc;

java = new RemoteCall(url).collectJavaInformations();
memory = java.memoryInformations;
println "used memory = " + Math.round(memory.usedMemory / 1024 / 1024) + " Mb";
println "active HTTP threads count = " + java.activeThreadCount;
println "system load average = " + java.systemLoadAverage;

threads = java.getThreadInformationsList();
deadlocked = new java.util.ArrayList();
for (thread in threads) {
  if (thread.deadlocked)
    deadlocked.add(thread);
}
println deadlocked.size() + " deadlocked threads / " + threads.size() + " threads";
for (thread in deadlocked) {
  println "";
  println thread;
  for (s in thread.getStackTrace())
    println "    " + s;
}

if (gc > 20) throw new Exception("Alert for mywebapp: gc is " + gc);
if (java.systemLoadAverage > 50) throw new Exception("Alert for mywebapp: systemLoadAverage is " + java.systemLoadAverage);
if (java.activeThreadCount > 100) throw new Exception("Alert for mywebapp: activeThreadCount is " + java.activeThreadCount);
if (deadlocked.size() > 0) throw new Exception("Alert for mywebapp: " + deadlocked.size() + " deadlocked threads");

Or any script with monitoring values in this page.

  • Add a post-build action "E-mail Notification" and write your email in "Recipients".
  • You can also configure "Discard old builds" and write a description.
  • Save.
  • Click "Build now" to test it.

Example: Groovy script in Jenkins

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.