JMX collector

[panda87]

JVM based application are widely used so be able to monitor their stats can be very useful

[ktsaou]

If someone can provide the basics (i.e. connect to JVM, get the metrics of interest, etc), I can turn it into a netdata plugin. It can be written in any language (java included).

[jasonwbarnett]

A cheap thing might be to just copy the overall design that scout app uses and just port it to netdata.

• requires Jmxterm
• scoutapp jmx source

[ktsaou]

@paulfantom this seems python too...

[paulfantom]

@ktsaou already added to my list. Right after HAProxy and ElasticSearch.

[ktsaou]

perfect! Please add redis too..

[paulfantom]

Ok, added.

[paulfantom]

Back to this.

Basically: JMX + python = No-go
But there is a solution to this (adopted for example by telegraf ). It is called jolokia and it provides JMX-HTTP bridge and is agent based.

Also I don't think this module can have autodetection, it will need to be configured by user.

I need to know if I can start working on jolokia module or maybe someone have better approach?

[ktsaou]

So JMX can be directly accessed by what? java?

[paulfantom]

There is something that is called jpype which should act as a python-java bridge. But I cannot get it working.

JMX is written in such way that it is very easy to be accessed by java, but it is almost impossible to be accessed by anything else. At least that is what I found.

[wojciechszela]

I second Jolokia. We have our own in-house metrics collector made in Scala (JVM), developed before telegraf was a thing, and we do use Jolokia instead of JMX. It's very easy to add Jolokia to any JVM based project (it's a JAR drop-in) and extra benefit is that it open tons of options in terms of metrics collections, monitoring, alerting... I doubt any JVM admin would object in using Jolokia.

[jasonwbarnett]

never seen or heard of Jolokia, but conceptually on board with this sort of thing. Seems like a great option.

Of course there are always alternatives that can be entertained down the road.

[toddiuszho]

If the Jolokia agents have discovery enabled, we can have auto-discovery.

[sanketplus]

Found something much simpler compared to Jolokia. JMXProxy

Just get the jar running anywhere and query it with host and JMX port for said Java process and get JSON full of metrics.

[johntdyer]

Big fan of Jolokia !

[simonnagl]

I took a look at this too...

Yes, it would be possible to use Jolokia / JMXProxy. But I do not see the need adding an layer. When using such a program the data flow would be
Java Application -> Jolokia -> Netdata Plugin -> netdata

It is possible to write an custom JMX client which connects to a JVM and works as a netdata plugin.
Java Application -> Netdata JMX Client (plugin) -> netdata.

This may be faster but requires adding a java plugin to netdata.

Edit

A tutorial how to create a client (http://docs.oracle.com/javase/tutorial/jmx/remote/custom.html)

[hereTac]

hope anybody's work about monitor jvm on netdata.

[ktsaou]

I am not (sorry). If anyone is working on this, please let us know of your progress.

I would also love to have this plugin. Keep in mind it can be written in any language (including java). We just need someone willing to spend some time to model the basics. If we have the basic structure, I think a lot of people will step in (including me).

[simonnagl]

I am currently working on a Netdata Java Plugin Daemon and I added an initial JMX Plugin.
There are many things to do...

https://github.com/simonnagl/netdata-plugin-java-daemon

[ktsaou]

@simonnagl wow!
Is it working? If you manage to make it publish at least one chart, more people (including me) could step in and start adding things.
Since it is a deamon, how does it connect to netdata?

[simonnagl]

Yes, this is working. It publishes one chart to netdata, monitoring the loaded java classes of the deamon itself. More configuration can be added to collect other Simple Managed Beans getters which return Long or Integer.
The daemon is a Netdata External Plugin which can be started by netdata and which sends metrics to STDOUT and log messages to STDERR.

[archenroot]

I will give it a try as I am installing today NetData

[simonnagl]

Great, if you need help or somethings not working just ask. Remember I not yet released 1.0.0 so it can be considered beta.

[waykar-prashant]

I configured the netdata app to use the java jmx plugin. Not sure how it is supposed to behave and how do i see the charts?
Please suggest

[simonnagl]

Have you done those steps?

git clone https://github.com/simonnagl/netdata-plugin-java-daemon.git
cd netdata-plugin-java-daemon
mvn package
cp src/main/sh/java.d.plugin /your/netdata/plugin/dir # Most likely /usr/libexec/netdata/plugins.d/
cp -r config/* /your/netdata/config/dir # Most likely /etc/netdata
cp target/java-daemon-0.1.0-SNAPSHOT.jar /tmp

After a restart of netdata you should as least see the metrics of the plugin itself. (Section jmx)
To collect other java processes you need to edit imx.conf. This is not very well tested.

As more users are interested in trying this plugin I will write a proper installer which works like ./netdata-installer.sh. One command that sets up everything. I'll post here when it is done.

[waykar-prashant]

/opt/netdata/netdata-configs/java.d/java.d
/opt/netdata/netdata-configs/java.d/jmx.conf
/opt/netdata/netdata-plugins/java.d/java.d.plugin
Are these correct?

[simonnagl]

How did you install metadata? sudo ./netdata-installer.sh without any arguments?

Then the files you mentioned should go there:

/etc/netdata/jmx.conf
/etc/netdata/java.d/java.d.plugin

[simonnagl]

The project now has an auto installer. It needs to be run with the same privilege and config options as the netdata installer.

sudo ./netdata-plugin-java-daemon-installer.sh

It would be very helpful if someone could give feedback if he sees charts after running the installer.
Thank you!

[simonnagl]

@h0lzi Thank you for the feedback again. You are absolutely right. I put your two points into separated issues. (#3046 and #3047)

[mmohrwinkel]

Hi @simonnagl , I found the dead link on the page https://github.com/simonnagl/netdata-plugin-java-daemon under Configuration. There is a link "JmxPluginConfiguration" in the table under "Schema" and it 404s.

#2808 sounds very promising, I'll try it out as soon as it is ready and give you feedback!

[simonnagl]

Hi @mmohrwinkel. This link will be dead until #2808 got merged.

[corporate-gadfly]

@simonnagl: Your solution for gathering JMX statistics spawns a side-by-side JVM which then connects with the target JVM.

What is the overhead of running an extra JVM vs. using the earlier proposed jolokia solution?

Thanks for your work, BTW.

[simonnagl]

Sorry for the late answer.
I was not able to measure it but I would say it depends.
jolokia needs an agent installed in the JVM (i.e. a application server) to do JMX for you. It can also be a proxy agent, which the is also a extra JVM.
If we assume that we monitor the "netdata way" which means only collecting local services I think using jolokia vs. a side-by-side JVM produces way the same overhead.

[cosmix]

@ilyam8 kind reminder, please update this issue (and/or close it) when we're ready to proceed with a Golang-based module for JMX monitoring.

[simonnagl]

@cosmix ist there a plan to implement a golang-based module for JMX? Do you want to build this based on Jolokia?

[cosmix]

@simonnagl hello, yes there is a plan to build a Golang-based collector for JMX. We are still exploring the potential solution space as to how to do it; most likely we won’t be requiring Jolokia.

[ki0]

Is this stopped? any updates?

[ilyam8]

@ki0 no, there is no updates unfortunately.

[Thrameos]

@paulfantom I know this is belated, but what issue were you having with getting JPype running? As far as I am aware JPype should be able to talk to JMX. Though we currently can't extend a Java class it is possible to implement an interface. And simply calling Java methods should be no issue at all.

[paulfantom]

@Thrameos sorry, I don't remember. My last take on this was over 4 years ago and I no longer use nor work on the project.

[ilyam8]

Closing this feature request and moving it to our community forum for further discussion (link will be posted soon).