Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Real-time Monitoring
Java JavaScript

This checkin makes the compression and decompression of sample

replacable, but turning SampleCoder into an interface.  Lots of files
got changed because formerly classes that used the coder used static
methods, and now they need to have access to an instance that
implements SampleCoder.

I still need to add the right factory class to select alternative
SampleCoder and TimelineCoder implementations.

util/src/main/java/com/ning/arecibo/util/timeline/samples/SampleCoder.java:

    Morph into an interface that only has coding methods - - the
    sample accumulation part got moved to SampleAccumulator, and the
    coding part got moved toe SampleCoderImpl.

util/src/main/java/com/ning/arecibo/util/timeline/samples/SampleAccumulator.java:

    New unit containing the portions of the former SampleCoder that
    accumulate samples.

util/src/main/java/com/ning/arecibo/util/timeline/samples/SampleCoderImpl.java:

    New unit - - the implementation of new interface SampleCoder.

collector/src/main/java/com/ning/arecibo/collector/guice/CachingDefaultTimelineDAOProvider.java:
collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregator.java:
collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineEventHandler.java:
collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineHostEventAccumulator.java:
collector/src/main/java/com/ning/arecibo/collector/resources/HostDataResource.java:
util/src/main/java/com/ning/arecibo/util/timeline/chunks/TimelineChunk.java:
util/src/main/java/com/ning/arecibo/util/timeline/chunks/TimelineChunkAccumulator.java:
util/src/main/java/com/ning/arecibo/util/timeline/chunks/TimelineChunkDecoded.java:
util/src/main/java/com/ning/arecibo/util/timeline/chunks/TimelineChunkMapper.java:
util/src/main/java/com/ning/arecibo/util/timeline/persistent/DefaultTimelineDAO.java:

    Add SampleCoder data member and constructor arg, and pass it to
    other methods and constructors that need it.

collector/src/main/java/com/ning/arecibo/collector/guice/CollectorModule.java:

    Bind sampleCoder.

collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregatorDAO.java:
util/src/main/java/com/ning/arecibo/util/timeline/persistent/TimelineDAOQueries.java:

    Remove unneeded @RegisterMapper of TimelineChunkMapper.

collector/src/test/java/com/ning/arecibo/collector/TestTimelineChunkAndTimes.java:
collector/src/test/java/com/ning/arecibo/collector/TestTimelineHostEventAccumulator.java:
collector/src/test/java/com/ning/arecibo/collector/persistent/TestDefaultTimelineDAO.java:
collector/src/test/java/com/ning/arecibo/collector/persistent/TestTimelineAggregator.java:
collector/src/test/java/com/ning/arecibo/collector/persistent/TestTimelineEventHandler.java:
collector/src/test/java/com/ning/arecibo/collector/persistent/TimelineLoadGenerator.java:
collector/src/test/java/com/ning/arecibo/collector/process/TestCollectorEventProcessor.java:
collector/src/test/java/com/ning/arecibo/collector/process/TestFileBackedBuffer.java:
collector/src/test/java/com/ning/arecibo/collector/process/TestInMemoryCollectorEventProcessor.java:
collector/src/test/java/com/ning/arecibo/collector/process/TestSamplesReplayer.java:
collector/src/test/java/com/ning/arecibo/collector/resources/TestHostDataResource.java:
util/src/test/java/com/ning/arecibo/util/timeline/chunks/TestTimelineChunkAccumulator.java:
util/src/test/java/com/ning/arecibo/util/timeline/chunks/TestTimelineChunkToJson.java:
util/src/test/java/com/ning/arecibo/util/timeline/samples/TestSampleCoder.java:
util/src/test/java/com/ning/arecibo/util/timeline/samples/TestSampleCompression.java:

    Create and use a SampleCoderImpl instance rather than calling
    static methods on the original SampleCoder.

collector/src/test/java/com/ning/arecibo/collector/TestEventCollectorServer.java:

    Fix test bug - - the time needs to be rounded to the nearest second.

util/src/main/java/com/ning/arecibo/util/timeline/DecimatingSampleFilter.java:

    Reflect the move of getDoubleValue() from SampleCoder to ScalarSample.

util/src/main/java/com/ning/arecibo/util/timeline/samples/ScalarSample.java:

    Move methods getDoubleValue() and sameSampleValue() from
    SampleCoder to here, since they are not coder-specific.
latest commit 7d4c80d99e
@dstryker dstryker authored
Failed to load latest commit information.
agent build: fix compilation issues
aggregator-client-support build: update pom.xml and copyright headers
aggregator-lang build: update pom.xml and copyright headers
aggregator build: fix compilation issues
alert-client-support alertmanager: dashboard: various configuration and startup cleanups
alert-confdata-support alertmanager: alert-client-support: fix remaining UI issues
alert-data-support
alert util: fix broken EmbeddedJettyJerseyModule
alertmanager alertmanager: remove incorrect plugin version
collector-client-support collector: update /rest/1.0/sample_kinds endpoint
collector-rt-support collector-rt-support: initial import
collector This checkin makes the compression and decompression of sample
dashboard-client-support build: update pom.xml and copyright headers
dashboard dashboard: handle edge usecase when refreshing the sample kinds tree
event dashboard: unify lighthouse advertisement with collector code
util This checkin makes the compression and decompression of sample
.gitignore alertmanager: remove Wicket
README.md Update README.md
pom.xml dashboard: add the plumbing for legends and groups configuration

README.md

Overview

  1. To build

    mvn clean install

  2. Detailed documentation: see https://github.com/ning/Arecibo/wiki

Collector

The collector gathers and stores data sent by the agents to the datastore (MySQL by default). It supports multiple APIs, including REST over HTTP and UDP.

Before starting the collector, you need to create a database first, e.g.:

~ > echo 'create database arecibo' | mysql -u root -p
Enter password:
~ > cat collector/src/main/resources/collector.sql | mysql -u root -p arecibo
Enter password:

To start the collector, assuming MySQL is running locally:

java \
    -Dcom.sun.management.jmxremote.authenticate=false \
    -Dcom.sun.management.jmxremote.port=8990 \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Darecibo.events.collector.db.url=jdbc:mysql://127.0.0.1:3306/arecibo \
    -Darecibo.events.collector.db.user=root \
    -Darecibo.events.collector.db.password=root
    -jar target/arecibo-collector-*-jar-with-dependencies.jar

To send data via the REST API, send JSON payloads to /xn/rest/1.0/event, e.g. (send 2 data points):

echo "
{
    \"timestamp\": $(date +%s)000,
    \"eventType\": \"testEvent\",
    \"sourceUUID\": \"550e8400-e29b-41d4-a716-446655440000\",
    \"min_heapUsed\": $RANDOM.515698888E9,
    \"max_heapUsed\": $RANDOM.835511784E9
}" | \
curl -v -H'Content-Type: application/json' -XPOST -d@- http://127.0.0.1:8088/xn/rest/1.0/event

Data is exposed in JSON format, e.g.:

curl -v http://127.0.0.1:8088/rest/1.0/hosts

See com.ning.arecibo.collector.resources for available endpoints.

For convenience, there is a Java library to access the data (see the collector-client-support module).

Dashboard

The dashboard module exposes graphs of collected metrics. It relies on a collector to access the data (using the collector REST api) and can be configured to use the alertmanager (disabled by default).

To start the dashboard, assuming the collector is at http://127.0.0.1:8088:

mvn \
    -Dcom.sun.management.jmxremote.authenticate=false \
    -Dcom.sun.management.jmxremote.port=8989 \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Darecibo.collectorClient.collectorUri=http://127.0.0.1:8088 \
    jetty:run

You can test it by hitting the dashboard's endpoints, e.g.:

curl -v http://127.0.0.1:8080/rest/1.0/hosts

Note that the output is not JSON, but JSONP, for the dashboard AJAX requests. To get straight JSON, use the collector endpoints instead, e.g.:

curl -v http://127.0.0.1:8088/rest/1.0/hosts
Something went wrong with that request. Please try again.