Skip to content

Latest commit

 

History

History
407 lines (249 loc) · 15.5 KB

hdr_histogram.md

File metadata and controls

407 lines (249 loc) · 15.5 KB
Raw

Module hdr_histogram

.

Copyright (c) 2014 Darach Ennis

Version: 0.2.0

Authors: Darach Ennis (darach@gmail.com).

Description

The HDR histogram library is an Erlang native interface function wrapper of Mike Barker's C port of Gil Tene's HDR Histogram utility.

A high dynamic range histogram is one that supports recording and analyzing sampled data points across a configurable range with configurable precision within that range. The precision is expressed as a number of significant figures in the recording.

This HDR histogram implementation is designed for recording histograms of value measurements in latency sensitive environments. Although the native recording times can be as low as single digit nanoseconds there is added overhead in this wrapper/binding due to both the frontend overhead of converting from native C to the NIF interface, and the erlang overhead incurred calling into the NIFs. C'est la vie, I suppose.

A distinct advantage of this histogram implementation is constant space and recording (time) overhead with an ability to recycle and reset instances whilst reclaiming already allocated space for reuse thereby reducing allocation cost and garbage collection overhead in the BEAM where repeated or continuous usage is likely. For example, a gen_server recording metrics continuously and resetting and logging histogram dumps on a periodic or other windowed basis.

The code is released to the public domain, under the same terms as its sibling projects, as explained in the LICENSE.txt and COPYING.txt in the root of this repository, but normatively at:

http://creativecommons.org/publicdomain/zero/1.0/

For users of this code who wish to consume it under the "BSD" license rather than under the public domain or CC0 contribution text mentioned above, the code found under this directory is also provided under the following license (commonly referred to as the BSD 2-Clause License). This license does not detract from the above stated release of the code into the public domain, and simply represents an additional license granted by http://creativecommons.org/publicdomain/zero/1.0/

** Beginning of "BSD 2-Clause License" text. **

Copyright (c) 2012, 2013, 2014 Gil Tene Copyright (c) 2014 Michael Barker Copyright (c) 2014 Darach Ennis All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Data Types

ref()


ref() = binary()

NIF private data (looks like empty binary)

Function Index

add/2Contribute the data points from a HDR histogram to another.
close/1Close this HDR histogram instance and free any system resources.
count_at/2Get the count of values at a given at a given value.
from_binary/1Take a snapshot of HDR histogram internal state as a compressed binary and hydrate/open a reference.
get_memory_size/1Get memory footprint of an HDR histogram.
get_total_count/1Get total count of record values.
log/3Log the histogram to a file in classic or CSV format.
lowest_at/2Get the lowest equivalent value.
max/1Get the maximum recorded data point value.
mean/1Get the mean data point value to a significant figure.
median/1Get the median data point value to a significant figure.
min/1Get the minimum recorded data point value.
open/2Open a fresh instance of a high dynamic range (HDR) histogram.
percentile/2Get the specified percentile data point value to a significant figure.
print/2Print the histogram to standard output in classic or CSV format.
record/2Record an uncorrected histogram data point value in a HDR histogram.
record_corrected/3Record a histogram data point value in a HDR histogram with expected interval for correction for coordinated ommission.
record_many/3Record a histogram data point value and number of occurances.
reset/1Reset the memory backing this HDR histogram instance and zero results.
rotate/2Copy data from Ref to To and return the difference as binary.
same/3Are two data point values considered to be equivalent.
stddev/1Get the standard deviation data point value to a significant figure.
to_binary/1Take a snapshot of HDR histogram internal state as a compressed binary.
to_binary/2Take a snapshot of HDR histogram internal state as an (optionally) compressed binary.

Function Details

add/2


add(To, From) -> integer() | {error, Reason}

Contribute the data points from a HDR histogram to another. Return the number of dropped data point values. That is, the values from From that are higher than the HighestTrackableValue of To.

close/1


close(Ref) -> ok | {error, term()}

Close this HDR histogram instance and free any system resources

count_at/2


count_at(Ref, Value) -> CountAt
  • Ref = ref()
  • Value = integer()
  • CountAt = integer()

Get the count of values at a given at a given value

from_binary/1


from_binary(Binary) -> {ok, Ref} | {error, term()}
  • Binary = binary()
  • Ref = ref()

Take a snapshot of HDR histogram internal state as a compressed binary and hydrate/open a reference. The reference SHOULD be closed when no longer needed to reclaim the memory used

get_memory_size/1


get_memory_size(Ref) -> Size
  • Ref = ref()
  • Size = integer()

Get memory footprint of an HDR histogram

get_total_count/1


get_total_count(Ref) -> Count
  • Ref = ref()
  • Count = integer()

Get total count of record values

log/3


log(Ref, Format, FileName::file:name()) -> ok | {error, Reason}
  • Ref = ref()
  • Format = classic | csv
  • Reason = term()

Log the histogram to a file in classic or CSV format

lowest_at/2


lowest_at(Ref, Value) -> LowestValueAt
  • Ref = ref()
  • Value = integer()
  • LowestValueAt = float()

Get the lowest equivalent value

max/1


max(Ref) -> MaxValue
  • Ref = ref()
  • MaxValue = integer()

Get the maximum recorded data point value

mean/1


mean(Ref) -> MeanValue
  • Ref = ref()
  • MeanValue = float()

Get the mean data point value to a significant figure

median/1


median(Ref) -> MedianValue
  • Ref = ref()
  • MedianValue = float()

Get the median data point value to a significant figure

min/1


min(Ref) -> MinValue
  • Ref = ref()
  • MinValue = integer()

Get the minimum recorded data point value

open/2


open(HighestTrackableValue, SignificantFigures) -> {ok, Ref} | {error, Reason}
  • HighestTrackableValue = integer()
  • SignificantFigures = 1..5
  • Ref = ref()
  • Reason = term()

Open a fresh instance of a high dynamic range (HDR) histogram

percentile/2


percentile(Ref, Percentile) -> PercentileValue
  • Ref = ref()
  • Percentile = float()
  • PercentileValue = float()

Get the specified percentile data point value to a significant figure

print/2


print(Ref, Format) -> ok | {error, Reason}
  • Ref = ref()
  • Format = classic | csv
  • Reason = term()

Print the histogram to standard output in classic or CSV format

record/2


record(Ref, Value) -> ok | {error, Reason}
  • Ref = ref()
  • Value = integer()
  • Reason = term()

Record an uncorrected histogram data point value in a HDR histogram

record_corrected/3


record_corrected(Ref, Value, ExpectedInterval) -> ok | {error, Reason}
  • Ref = ref()
  • Value = integer()
  • ExpectedInterval = integer()
  • Reason = term()

Record a histogram data point value in a HDR histogram with expected interval for correction for coordinated ommission

record_many/3


record_many(Ref, Value, Count) -> ok | {error, Reason}
  • Ref = ref()
  • Value = integer()
  • Count = integer()
  • Reason = term()

Record a histogram data point value and number of occurances

reset/1


reset(Ref) -> ok | {error, term()}

Reset the memory backing this HDR histogram instance and zero results

rotate/2


rotate(Ref, To) -> Diff::binary() | {error, term()}

Copy data from Ref to To and return the difference as binary

same/3


same(Ref, A, B) -> AreEquivalent
  • Ref = ref()
  • A = integer()
  • B = integer()
  • AreEquivalent = boolean()

Are two data point values considered to be equivalent

stddev/1


stddev(Ref) -> StddevValue
  • Ref = ref()
  • StddevValue = float()

Get the standard deviation data point value to a significant figure

to_binary/1


to_binary(Ref) -> binary() | {error, term()}

Take a snapshot of HDR histogram internal state as a compressed binary. The reference HDR instance can be modified after a snapshot is taken in the usual way with no restrictions.

to_binary/2


to_binary(Ref, X2::[{compression, none} | {compression, zlib}]) -> binary() | {error, term()}

Take a snapshot of HDR histogram internal state as an (optionally) compressed binary. The reference HDR instance can be modified after a snapshot is taken in the usual way with no restrictions.