Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics collecting and reporting support #1321

Merged
merged 23 commits into from Jun 25, 2015

Conversation

Projects
None yet
3 participants
@karalabe
Copy link
Member

commented Jun 23, 2015

The goal of this PR is that - similar to logs - we should be able to add metrics collection to any part of the code without fancy constructs (counter variables, exporting them through public interfaces, adding them to the APIs, consoles, RCP/IPC, etc), and have the just work, be queryable and visualizable.

To that extent, this PR currently adds support for two metric types:

  • Meters: Event counters that can provide rate averages for the past 1, 5, 15 mins, and totals.
  • Timer: Timed meters, where each event has a duration associated and the last 1024 can be queried.

To create and use a meter and/or timer:

meter := metrics.GetOrRegisterMeter("my/fancy/meter", metrics.DefaultRegistry)
meter.Mark(125) // Register 125 events (e.x. 125 bytes written).

timer := metrics.GetOrRegisterTimer("my/fancy/timer", metrics.DefaultRegistry)
timer.Update(time.Second) // Register an event that took a second (e.x. block import)

These are automatically registered into a central metrics registry and can be queried from the javascript console through debug.metrics() for a user friendly pre-formatted output:

> debug.metrics().p2p.InboundTraffic
{
  Avg01Min: '169.12K (2.82K/s)',
  Avg05Min: '1.92M (6.42K/s)',
  Avg15Min: '3.57M (3.96K/s)',
  Total: '5.83M (2.97K/s)'
}
> debug.metrics().core.BlockInsertions
{
  Avg01Min: '10 (0.17/s)',
  Avg05Min: '61 (0.20/s)',
  Avg15Min: '168 (0.19/s)',
  Maximum: '2.157261657s',
  Minimum: '2.271716ms',
  Percentiles: {
    20: '6.993756ms',
    50: '12.342836ms',
    80: '21.765944ms',
    95: '218.500479ms',
    99: '376.015984ms'
  },
  Total: '432 (0.22/s)'
}

Or in a raw format for visualization, reporting, processing, monitoring, etc. via debug.metrics(true):

> debug.metrics(true).p2p.InboundTraffic
{
  AvgRate01Min: 1599.6190029292586,
  AvgRate05Min: 5367.754506658111,
  AvgRate15Min: 3761.057607521597,
  MeanRate: 2907.3919382272857,
  Total: 5901154
}
> debug.metrics(true).core.BlockInsertions
{
  AvgRate01Min: 0.18424704676246578,
  AvgRate05Min: 0.19801030732706026,
  AvgRate15Min: 0.18758183634199713,
  MeanRate: 0.22205942695239234,
  Percentiles: {
    0: 2271716,
    1: 2595259.96,
    // ...
    98: 267196822.1600002,
    99: 362098004.77999866,
    100: 2157261657
  },
  Total: 453
}

The PR furthermore introduces a new command to geth, geth monitor, that allows the real time visualization of any of these metrics:

Sample Metrics

The monitor will plot all requested metrics on individual line charts, expanding user patterns:

  • geth monitor core/BlockInsertions/Percentiles/80: a single metric to chart
  • geth monitor core/BlockInsertions/Percentiles/25 core/BlockInsertions/Percentiles/95: two metrics, side by side
  • geth monitor core/BlockInsertions/Percentiles/25,80,95: three metrics, with a common root
  • geth monitor p2p: expand and display all metrics under the p2p collection
  • geth monitor eth/db/block,state/Reads,Writes: various metrics with various wildcards (image above)

The charts are automatically scaled below 1000, and the scaling units inserted into the chart labels. The colors of the charts are also updated depending on the scale, ranging in [blue, cyan, green, yellow, red], with blue being none scaled, and every consecutive one a 1000 multiplier.

@obscuren obscuren modified the milestone: 0.9.34 Jun 23, 2015

@karalabe karalabe force-pushed the karalabe:cut-it-open-3000 branch to e8d2314 Jun 23, 2015

@karalabe karalabe changed the title Uuuuu, what's inside here? :D Metrics collecting and reporting support Jun 23, 2015

@karalabe karalabe force-pushed the karalabe:cut-it-open-3000 branch to 92ef33d Jun 24, 2015

@karalabe karalabe removed the in progress label Jun 25, 2015

obscuren added a commit that referenced this pull request Jun 25, 2015

Merge pull request #1321 from karalabe/cut-it-open-3000
Metrics collecting and reporting support

@obscuren obscuren merged commit b0a5be4 into ethereum:develop Jun 25, 2015

4 of 5 checks passed

Linux Go pull requests DEV build done.
Details
ARM Go pull requests DEV build done.
Details
OSX Go pull requests DEV build done.
Details
Windows Go pull requests DEV build done.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@obscuren obscuren removed the pr:review label Jun 25, 2015

@obscuren obscuren modified the milestone: 0.9.34 Jun 30, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.