Hawkular Agents that can be used to monitor managed products
Java Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.mvn Replace srcdeps-maven-plugin with srcdeps core extension (#273) Dec 5, 2016
eap6-support Reflect immutable + in-container attributes at server level. (#295) Feb 15, 2017
hawkular-dmr-client HAWKULAR-1153 Initial support enable/disable/update collection interv… Feb 19, 2017
hawkular-swarm-agents Reflect immutable + in-container attributes at server level. (#295) Feb 15, 2017
hawkular-wildfly-agent-dist [maven-release-plugin] prepare for next development iteration Feb 14, 2017
hawkular-wildfly-agent-download [maven-release-plugin] prepare for next development iteration Feb 14, 2017
hawkular-wildfly-agent-feature-pack Reflect immutable + in-container attributes at server level. (#295) Feb 15, 2017
hawkular-wildfly-agent-installer-full [maven-release-plugin] prepare for next development iteration Feb 14, 2017
hawkular-wildfly-agent-installer [maven-release-plugin] prepare for next development iteration Feb 14, 2017
hawkular-wildfly-agent-itest-parent HAWKULAR-1153 Initial support enable/disable/update collection interv… Feb 19, 2017
hawkular-wildfly-agent-wf-extension [maven-release-plugin] prepare for next development iteration Feb 14, 2017
hawkular-wildfly-agent HAWKULAR-1153 Initial support enable/disable/update collection interv… Feb 19, 2017
wildfly-module-installer [maven-release-plugin] prepare for next development iteration Feb 14, 2017
.gitignore Replace srcdeps-maven-plugin with srcdeps core extension (#273) Dec 5, 2016
.travis.maven.settings.xml fix license Feb 5, 2016
.travis.yml HAWKULAR-1138 move to okhttp 3.x (#278) Dec 16, 2016
LICENSE Initial commit Mar 31, 2015
README.adoc agent readme with some details on configuration (#277) Dec 2, 2016
mvnw Replace srcdeps-maven-plugin with srcdeps core extension (#273) Dec 5, 2016
mvnw.cmd Replace srcdeps-maven-plugin with srcdeps core extension (#273) Dec 5, 2016
pom.xml HAWKULAR-1153 Initial support enable/disable/update collection interv… Feb 19, 2017

README.adoc

Hawkular WildFly Agent

Build Status
Synopsis

Hawkular WildFly Agent can be used to monitor WildFly or EAP applications servers as well as applications running within those application servers (for the rest of this documentation, WildFly will be used for brevity but unless other noted EAP will be implied as well). It can collect inventory and metrics from the native WildFly management interface (DMR API) as well as JMX (via Jolokia if monitoring remote application servers).

Installation

The agent runs embedded within a WildFly application server. Whether the agent monitors its own app server is up to you and how you configure the agent, but it always is deployed in one.

Tip
That isn’t totally true. The agent can run as a microservice in the form of a WildFly Swarm uber-jar. For more details on the swarm agent, see hawkular-swarm-agents/hawkular-swarm-agent-dists/hawkular-swarm-agent-dist

For more information on how to install the agent in a WildFly application server, see hawkular-wildfly-agent-installer.

Configuration

The agent is configured like any other WildFly subsystem - that is, inside WildFly’s XML configuration file (e.g. standalone.xml). Most of the agent’s configuration is located in the agent’s subsystem configuration section:

<subsystem xmlns="urn:org.hawkular.agent:agent:1.0" ...>
   ...
</subsystem>

You can reference the agent subsystem XSD Schema for all the valid settings. Some simple help documentation for each agent subsystem element/attribute can be found at LocalDescriptions.properties.

Subsystem

<subsystem xmlns="urn:org.hawkular.agent:agent:1.0"> is the outer element of the agent subsystem configuration. It defines some global agent configuration settings.

enabled

When true the agent will be enabled and will start running once the WildFly application server initializes. When false the agent subsystem will not start any internal services and will remain dormant unless and until this flag is set to true.

auto-discovery-scan-period-secs

Periodically the agent will perform a discovery scan allowing it to find resources in the servers it is managing. Once this discovery scan completes, the inventory for the managed server that was scanned will be updated. This setting defines how often that discovery scan is performed.

Storage Adapter

<storage-adapter> defines where the data the agent collects gets stored. Essentially, it defines where your Hawkular Server is located and how the agent is to communicate with it.

type

Defines the type of the Hawkular backend the agent will be communicating with. Usually you will want to use HAWKULAR which means the agent will be expected to talk to a full Hawkular Server. This means the agent will store both metric data and inventory data and will connect to the Hawkular Server via a websocket so the agent can send and receive realtime messages. If you set this to METRICS then the agent is expected to talk to a independent Hawkular Metrics server - in this case only metrics are stored, inventory data is not collected, and the agent does not maintain a websocket connection.

server-outbound-socket-binding-ref

The outbound socket binding definition that defines where the Hawkular backend is (hostname and port).

use-ssl

Should be true if the agent should communicate to the Hawkular backend using SSL encryption.

security-realm

If SSL is to be used, this is the name of the configured security realm that provides keystore information for secure communications. Security realms are defined in the WildFly configuration outside of the agent subsystem configuration.

username

User that will log into the Hawkular backend.

password

Credentials of the user that logs into Hawkular backend.

Metrics, Avails, Resource Types

The agent is told what to monitor by defining metrics, avails, and resource types and grouping those things into sets. Metrics are things like gauges (values that go up and down, like "used memory" or "cpu utilization") and counters (values that continually increase, like "number of requests processed"). Avails are things that determine if a resource is up or down. Resource types describe the entities that are deployed in your WildFly application servers (such as "datasources" or "EJBs" or "message queues").

There are two main ways the agent can collect such data: 1) through WildFly’s native management interface and its detyped model representation (DMR) API and 2) through standard JMX. In addition, the agent can collect this data either locally or remotely - that is to say, the agent can collect DMR and JMX data from within the same WildFly application server the agent is running in or from another external WildFly application server.

Within the agent configuration are definitions for all the types of resources the agent can expect to find in the servers it is managing. The metrics and avails that those resource emit are also defined so the agent knows what kinds of data it should collect from the resources it discovered.

The examples below should hopefully illustrate this further.

DMR

This defines a metric set that contains a metric whose value represents heap memory currently being used.

<metric-set-dmr name="WildFly Memory Metrics">
  <!-- This metric is to be collected every 30 seconds.
       This metric's value is in units of bytes.
       This metric is emitted by the owning resource's child at the DMR path of "/core-service=platform-mbean/type=memory".
       This metric is found in the composite attribute "heap-memory-usage" at the subpath of "used". -->
  <metric-dmr name="Heap Used"
              interval="30"
              time-units="seconds"
              metric-units="bytes"
              path="/core-service=platform-mbean/type=memory"
              attribute="heap-memory-usage#used" />
  <!-- You can define other metrics in this set if you want -->
</metric-set-dmr>

You can define avails in a similar way:

<!-- When the agent collects a value of the given attribute "status", it will
     see if that value matches the regular expression defined in up-regex.
     If it matches, this avail value will be stored as UP, otherwise it will be stored as DOWN. -->
<avail-set-dmr name="Server Availability">
  <avail-dmr name="Server Availability"
             interval="1"
             time-units="minutes"
             path="/"
             attribute="server-state"
             up-regex="run.*" />
  <!-- You can define other avails in this set if you want -->
</avail-set-dmr>

Once your DMR metrics and avails are defined, you then define sets of resource types. These declare the types of resources the agent should expect to discover in its managed servers. The description of these resource types include what metrics and avails those resources emit as well as what resource configuration properties they have and what operations they support.

<resource-type-set-dmr name="Standalone Environment">
  <resource-type-dmr name="WildFly Server"
                     resource-name-template="WildFly Server [%ManagedServerName]"
                     path="/"
                     metric-sets="WildFly Memory Metrics"
                     avail-sets="Server Availability">
    <resource-config-dmr name="Name"          attribute="name"/>
    <resource-config-dmr name="Version"       attribute="product-version" />
    <resource-config-dmr name="Home Directory"
                         path="/core-service=server-environment"
                         attribute="home-dir"/>
    <operation-dmr name="Reload" internal-name="reload" >
      <param name="admin-only"                type="bool" default-value="false"/>
      <param name="use-current-server-config" type="bool" default-value="false"/>
    </operation-dmr>
  </resource-type-dmr>
</resource-type-set-dmr>

You can define child resource types - these are resources that can be expected to be found under other resources:

<resource-type-set-dmr name="Clustering" enabled="true">
  <!-- This is a child resource found at the relative path "/subsystem=infinispan"
       under any parent resource whose type is "WildFly Server". Since "WildFly Server"
       resources have a path of "/" then the absolute path of Infinispan resources will
       be "/subsystem=infinispan". -->
  <resource-type-dmr name="Infinispan"
                           resource-name-template="Infinispan"
                           path="/subsystem=infinispan"
                           parents="WildFly Server" />
  <!-- This is a child resource found at the relative path "/cache-container=*"
       under the parent resource whose type is "Infinispan". Since "Infinispan" resources
       have the absolute path "/subsystem=infinispan" then resources of this type
       "Infinispan Cache Container" will have absolute path of "/subsystem=infinispan/cache-container=*"
       Because of the wildcard in the path, the agent will actually look for multiple resources
       that match the wildcard (that is, any resource whose absolute path matches
       "/subsystem=infinispan/cache-container=*".) -->
  <resource-type-dmr name="Infinispan Cache Container"
                     resource-name-template="Infinispan Cache Container [%-]"
                     path="/cache-container=*"
                     parents="Infinispan" />
</resource-type-set-dmr>

JMX

If applications deployed in WildFly expose data via JMX, the agent can collect that data as well. You define JMX metrics, avails, and resource types in a very similar manner as with DMR. Note that the XML elements end with "-jmx" as opposed to "-dmr" and note that rather than using DMR paths, these use JMX object names.

Here are examples of metrics, avails, and resource types:

<metric-set-jmx name="RuntimeMetricsJMX" enabled="true">
  <!-- This defines a metric that is collected every 30 seconds from the MBean "java.lang:type=Memory",
       specifically from the "used" value found in the MBean's composite attribute "HeapMemoryUsage". -->
  <metric-jmx name="Used Heap Memory"
              interval="30"
              time-units="seconds"
              metric-units="bytes"
              object-name="java.lang:type=Memory"
              attribute="HeapMemoryUsage#used" />
</metric-set-jmx>

<avail-set-jmx name="MemoryPoolAvailsJMX" enabled="true">
  <!-- This defines an avail that indicates if a Memory Pool is a valid or not. Because no object name
       is defined, it is assumed the attribute "Valid" is to be found directly on the resource where this
       avail is attached. If the value of the "Valid" attribute matches the regular expression "[tT].*"
       then the availability is stored as UP, otherwise it is stored as DOWN. -->
  <avail-jmx name="Memory Pool Avail"
             interval="30"
             time-units="seconds"
             attribute="Valid"
             up-regex="[tT].*" />
</avail-set-jmx>

<resource-type-set-jmx name="MainJMX" enabled="true">
  <!-- This defines a resource type identified by the MBean object name "java.lang:type=Runtime".
       Since this is note a object name query, this resource type is defining a singletone resource,
       since there can only every be one of them. -->
  <resource-type-jmx name="Runtime MBean"
                     resource-name-template="JMX [%_ManagedServerName%][%type%]"
                     object-name="java.lang:type=Runtime"
                     metric-sets="RuntimeMetricsJMX" >
    <-- This defines a resource configuration property that will be associated with any resource of this type.
        This means any MBean with the object namne "java.lang:type=Runtime" will be assigned a resource
        configuration property named "OS Name" whose value is that of the "Name" attribute found on
        another MBean named "java.lang:type=OperatingSystem". -->
    <resource-config-jmx name="OS Name" object-name="java.lang:type=OperatingSystem" attribute="Name" />
  </resource-type-jmx>
</resource-type-set-jmx>

<resource-type-set-jmx name="MemoryPoolJMX" enabled="true">
  <!-- This defines a resource type that is to be considered a child of its parent "Runtime MBean" resource.
       Because the object-name is a query, this resource type definition tells the agent it can expect
       multiple resources of this type. Any MBean that matches this query object name will be considered
       this type of resource. -->
  <resource-type-jmx name="Memory Pool MBean"
                     parents="Runtime MBean"
                     resource-name-template="JMX [%_ManagedServerName%] %type% %name%"
                     object-name="java.lang:type=MemoryPool,name=*"
                     avail-sets="MemoryPoolAvailsJMX" >
    <resource-config-jmx name="Type" attribute="Type" />
  </resource-type-jmx>
</resource-type-set-jmx>

<managed-servers>

remote-dmr

This defines a remote WildFly server that the agent should monitor. You give it a name, where it is located (host and port) and what resources the agent can expect to find there (by setting the resource-type-sets attribute whose value is a comma-separated list of resource-type-set-dmr names).

local-dmr

This indicates the agent should monitor the same WildFly server where the agent is deployed. Because it is the same WildFly server where the agent is co-located, there is no need to specify a host or port, but you still need to tell the agent what resource types it should look for via the resource-type-sets attribute.

remote-jmx

This defines a remote JMX endpoint that the agent should monitor. This does not have to be a WildFly server! It can be anything with a Jolokia REST endpoint. It is through the Jolokia REST interface that the agent will communicate with the managed server. Similar to remote-dmr, you need to tell the agent where the Jolokia endpoint is (via the url attribute) and you need to tell the agent what resources it can expect to find there (by setting the resource-type-sets attribute whose value is a comma-separated list of resource-type-set-jmx names).

local-jmx

This indicates the agent should monitor the same WildFly server where the agent is deployed, but it will collect data from the JMX MBeans found in the WildFly server. Again, you need to specify the resource-type-sets attribute in order to inform the agent about the resources it can expect to find.