Skip to content

Commit

Permalink
fix(Documentation): Adding documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
addomafi committed Nov 11, 2019
1 parent b3cd05f commit aab0ee5
Show file tree
Hide file tree
Showing 4 changed files with 2,522 additions and 3 deletions.
62 changes: 61 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,64 @@
# spark-monitoring

A tool to extract metrics from Spark Jobs
A tool to extract metrics from Spark Applications, it applies to Streaming and Jobs.

## Current Status

Stable.

## Installation

```sh
npm install spark-monitoring
```

## Features

* Extract metrics from Streaming Jobs;
* Extract metrics from Normal Jobs;
* Send these metrics to Zabbix.

## Send metrics to Zabbix

To get usage details, type this:

```sh
zabbix-spark-monitor --help
```

## To use the tool on your application

Import the required class as follow:
```javascript
import { SparkMonitoring } from '../../lib/spark-monitoring';
```

And use the method `getApplicationStats` to get statistics for an specific application.

The output follow this example:
```json
{
"allocatedMB": 5632,
"allocatedVCores": 4,
"elapsedTime": 1050926592,
"id": "application_1569147216125_1386",
"jobStats": [
{
"max": 25,
"mean": 17.6,
"min": 11,
"name": "DataCollection",
"window": 156
}
],
"memorySeconds": 5918796398,
"name": "51-minimum-gain",
"pendingRecords": 2561,
"processedRecords": 951,
"runningContainers": 4,
"runningJobs": 1,
"streaming": true,
"totalDelay": 30000,
"vcoreSeconds": 4203689
}
```

0 comments on commit aab0ee5

Please sign in to comment.