Skip to content
This repository has been archived by the owner on Oct 3, 2023. It is now read-only.

Update stats, zpages, prometheus packages with new Tags API #307

Merged
merged 2 commits into from
Jan 29, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 63 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ All notable changes to this project will be documented in this file.
## Unreleased
- Add Metrics API.
- Add Resource API.
- Add Tags API.
- Add Gauges (`DoubleGauge`, `LongGauge`, `DerivedDoubleGauge`, `DerivedLongGauge`) APIs.
- Add support for supplying instrumentation configuration via tracing option. Option argument added to instrumentation interface.
- Add ignoreIncomingPaths and ignoreOutgoingUrls support to the http and https tracing instrumentations.
Expand All @@ -14,26 +15,82 @@ All notable changes to this project will be documented in this file.

- Modify `Logger` interface: `level` made optional, `silly` removed.
- The ```new Stats()``` has been deprecated on Stats class. The global singleton ```globalStats``` object should be used instead. Also, ```registerView()``` is separated out from ```createView()```.
- Use ```TagKey```, ```TagValue``` and ```TagMap``` to create the tag keys, tag values.

##### Old code
```js
const { Stats } = require("@opencensus/core");
const stats = new Stats();

// Counts/groups the lengths of lines read in.
const mLineLengths = stats.createMeasureInt64(
"demo/line_lengths",
MeasureUnit.BYTE,
"The distribution of line lengths"
);

// Create tag keys
const tagKeys = ["method", "status"];

// Create and register the view
stats.createView(...);
stats.createView(
"demo/lines_in",
mLineLengths,
AggregationType.COUNT,
tagKeys,
"The number of lines from standard input"
);

// Records measurements
stats.record({
measure: mLineLengths,
tags,
value: 2
});

```

##### New code
```js
// Get the global singleton stats object
// Gets the global stats instance
const { globalStats } = require("@opencensus/core");

// Create the view
const view = globalStats.createView(...);

// register the view
// Counts/groups the lengths of lines read in.
const mLineLengths = globalStats.createMeasureInt64(
"demo/line_lengths",
MeasureUnit.BYTE,
"The distribution of line lengths"
);

// Creates the method and status key
const methodKey = {name: "method"};
const statusKey = {name: "status"};

// Creates the view
const view = globalStats.createView(
"demo/lines_in",
mLineLengths,
AggregationType.COUNT,
[methodKey, statusKey],
"The number of lines from standard input"
);

// Registers the view
globalStats.registerView(view);

// Creates tags map -> key/value pair
const tagMap = new TagMap();
tagMap.set(methodKey, {value: 'REPL'});
tagMap.set(statusKey, {value: 'OK'});

// Creates measurements (measure + value)
const measurements = [{
measure: mLineLengths,
value: 2
}];

// Records measurement with tagMap
globalStats.record(measurements, tagMap);
```

## 0.0.8 - 2018-12-14
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

import * as loggerTypes from '../common/types';
import {Measurement, View} from '../stats/types';
import {TagKey, TagValue} from '../tags/types';
import * as modelTypes from '../trace/model/types';

import {ExporterBuffer} from './exporter-buffer';
Expand Down Expand Up @@ -93,7 +94,8 @@ export class ConsoleStatsExporter implements types.StatsEventListener {
* @param view recorded view from measurement
* @param measurement recorded measurement
*/
onRecord(views: View[], measurement: Measurement) {
onRecord(
views: View[], measurement: Measurement, tags: Map<TagKey, TagValue>) {
console.log(`Measurement recorded: ${measurement.measure.name}`);
}

Expand Down
5 changes: 4 additions & 1 deletion packages/opencensus-core/src/exporters/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/

import {Measurement, View} from '../stats/types';
import {TagKey, TagValue} from '../tags/types';
import * as configTypes from '../trace/config/types';
import * as modelTypes from '../trace/model/types';

Expand Down Expand Up @@ -44,7 +45,9 @@ export interface StatsEventListener {
* @param views The views related to the measurement
* @param measurement The recorded measurement
*/
onRecord(views: View[], measurement: Measurement): void;
onRecord(
views: View[], measurement: Measurement,
tags: Map<TagKey, TagValue>): void;

/**
* Starts the exporter that polls Metric from Metrics library and send
Expand Down
19 changes: 10 additions & 9 deletions packages/opencensus-core/src/stats/metric-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@
*/


import {LabelKey, LabelValue, MetricDescriptor, MetricDescriptorType} from '../metrics/export/types';
import {LabelValue, MetricDescriptor, MetricDescriptorType} from '../metrics/export/types';
import {TagValue} from '../tags/types';

import {AggregationType, Measure, MeasureType, Tags, View} from './types';
import {AggregationType, Measure, MeasureType, View} from './types';

/** Utils to convert Stats data models to Metric data models */
export class MetricUtils {
Expand Down Expand Up @@ -61,25 +62,25 @@ export class MetricUtils {
* @returns {MetricDescriptor}
*/
static viewToMetricDescriptor(view: View): MetricDescriptor {
// TODO(mayurkale): add description
return {
name: view.name,
description: view.description,
unit: view.measure.unit,
type: MetricUtils.getType(view.measure, view.aggregation),
labelKeys: view.getColumns().map(
tag => ({key: tag, description: ''} as LabelKey))
// TODO(mayurkale): add description
tagKey => ({key: tagKey.name, description: ''}))
};
}

/**
* Converts tags to label values.
* @param tags
* Converts tag values to label values.
* @param tagValues
* @returns {LabelValue[]} List of label values
*/
static tagsToLabelValues(tags: Tags): LabelValue[] {
return Object.keys(tags).map(key => {
return {value: tags[key]} as LabelValue;
static tagValuesToLabelValues(tagValues: TagValue[]): LabelValue[] {
return tagValues.map((tagValue) => {
return {value: tagValue.value};
});
}
}
15 changes: 15 additions & 0 deletions packages/opencensus-core/src/stats/recorder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
* limitations under the License.
*/

import {TagKey, TagValue} from '../tags/types';
import {AggregationData, AggregationType, CountData, DistributionData, LastValueData, Measurement, MeasureType, SumData} from './types';

export class Recorder {
Expand All @@ -40,6 +41,20 @@ export class Recorder {
}
}

/** Gets the tag values from tags and columns */
static getTagValues(tags: Map<TagKey, TagValue>, columns: TagKey[]):
draffensperger marked this conversation as resolved.
Show resolved Hide resolved
TagValue[] {
const tagValues: TagValue[] = [];

// ignore not found key values.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tag keys that are not found, we should use null or undefined as the tag value. See https://github.com/census-instrumentation/opencensus-java/blob/5a97afdefdb2d9f4213264d91fa67b646adbf5d9/impl_core/src/main/java/io/opencensus/implcore/stats/RecordUtils.java#L86-L87.

The tag values returned from this method should have the same size as tag keys.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This project has strictNullChecks set to false in tsconfig, hence didn't add null for missing tag values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The caveat here is the tag values returned must match the order of tag keys. If we skip any tag values then they became mismatched.

One example: a view is defined to break down latencies by method and caller. The tag keys are [method, caller] in order. If a record is called with only tag caller:c1, this method will return tag values [c1]. Then c1 will be matched with method.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, For tag keys that are not found, replaced with null as the tag value.

columns.forEach((tagKey) => {
if (tags.has(tagKey)) {
tagValues.push(tags.get(tagKey));
}
});
return tagValues;
}

private static addToDistribution(
distributionData: DistributionData, value: number): DistributionData {
distributionData.count += 1;
Expand Down
18 changes: 14 additions & 4 deletions packages/opencensus-core/src/stats/stats.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ import * as loggerTypes from '../common/types';
import {StatsEventListener} from '../exporters/types';
import {Metric} from '../metrics/export/types';
import {Metrics} from '../metrics/metrics';
import {TagMap} from '../tags/tag-map';
import {TagKey} from '../tags/types';

import {MetricProducerForStats} from './metric-producer';
import {AggregationType, Measure, Measurement, MeasureType, MeasureUnit, Stats, View} from './types';
Expand Down Expand Up @@ -83,7 +85,7 @@ export class BaseStats implements Stats {
*/
createView(
name: string, measure: Measure, aggregation: AggregationType,
tagKeys: string[], description: string,
tagKeys: TagKey[], description: string,
bucketBoundaries?: number[]): View {
const view = new BaseView(
name, measure, aggregation, tagKeys, description, bucketBoundaries);
Expand Down Expand Up @@ -156,27 +158,35 @@ export class BaseStats implements Stats {
/**
* Updates all views with the new measurements.
* @param measurements A list of measurements to record
* @param tags optional The tags to which the value is applied.
* tags could either be explicitly passed to the method, or implicitly
* read from current execution context.
*/
record(...measurements: Measurement[]): void {
record(measurements: Measurement[], tags?: TagMap): void {
if (this.hasNegativeValue(measurements)) {
this.logger.warn(`Dropping measurments ${measurements}, value to record
must be non-negative.`);
return;
}

if (!tags) {
// TODO(mayurkale): read tags current execution context
tags = new TagMap();
}

for (const measurement of measurements) {
const views = this.registeredViews[measurement.measure.name];
if (!views) {
break;
}
// Updates all views
for (const view of views) {
view.recordMeasurement(measurement);
view.recordMeasurement(measurement, tags);
}

// Notifies all exporters
for (const exporter of this.statsEventListeners) {
exporter.onRecord(views, measurement);
exporter.onRecord(views, measurement, tags.tags);
}
}
}
Expand Down
30 changes: 14 additions & 16 deletions packages/opencensus-core/src/stats/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@

import {StatsEventListener} from '../exporters/types';
import {Metric} from '../metrics/export/types';
import {TagMap} from '../tags/tag-map';
import {TagKey, TagValue} from '../tags/types';

/** Main interface for stats. */
export interface Stats {
Expand All @@ -31,7 +33,7 @@ export interface Stats {
*/
createView(
name: string, measure: Measure, aggregation: AggregationType,
tagKeys: string[], description: string,
tagKeys: TagKey[], description: string,
bucketBoundaries?: number[]): View;

/**
Expand Down Expand Up @@ -62,8 +64,11 @@ export interface Stats {
/**
* Updates all views with the new measurements.
* @param measurements A list of measurements to record
* @param tags optional The tags to which the value is applied.
* tags could either be explicitly passed to the method, or implicitly
* read from current execution context.
*/
record(...measurements: Measurement[]): void;
record(measurements: Measurement[], tags?: TagMap): void;

/**
* Remove all registered Views and exporters from the stats.
Expand All @@ -83,12 +88,6 @@ export interface Stats {
registerExporter(exporter: StatsEventListener): void;
}


/** Tags are maps of names -> values */
export interface Tags {
[key: string]: string;
}

/**
* Describes the type of the individual values/measurements recorded by an
* application. It includes information such as the type of measurement, the
Expand Down Expand Up @@ -140,8 +139,6 @@ export interface Measurement {
* up to Number.MAX_SAFE_INTERGER.
*/
readonly value: number;
/** The tags to which the value is applied */
readonly tags: Tags;
}

/**
Expand Down Expand Up @@ -176,15 +173,16 @@ export interface View {
*
* Measurements with measurement type INT64 will have its value truncated.
* @param measurement The measurement to record
* @param tags The tags to which the value is applied
*/
recordMeasurement(measurement: Measurement): void;
recordMeasurement(measurement: Measurement, tags: TagMap): void;
/**
* Returns a snapshot of an AggregationData for that tags/labels values.
* @param tags The desired data's tags
* @param tagValues The desired data's tag values.
*/
getSnapshot(tags: Tags): AggregationData;
getSnapshot(tagValues: TagValue[]): AggregationData;
/** Gets the view's tag keys */
getColumns(): string[];
getColumns(): TagKey[];
/** Gets view`s metric */
getMetric(start: number): Metric;
}
Expand All @@ -204,8 +202,8 @@ export enum AggregationType {
export interface AggregationMetadata {
/** The aggregation type of the aggregation data */
readonly type: AggregationType;
/** The tags/labels that this AggregationData collects and aggregates */
readonly tags: Tags;
/** The tagValues that this AggregationData collects and aggregates */
readonly tagValues: TagValue[];
/** The latest timestamp a new data point was recorded */
timestamp: number;
}
Expand Down
Loading