Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proto update to latest to support arrays and maps #1339

Merged
merged 29 commits into from
Jul 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
99d8854
feat: adding proto over http for collector exporter
obecny Jul 10, 2020
6638c32
chore: lint
obecny Jul 10, 2020
dbdd707
chore: lint
obecny Jul 10, 2020
32a01c5
chore: fixing transport type in example
obecny Jul 10, 2020
0b7dc81
chore: adding generated to files in package.json
obecny Jul 10, 2020
11c5a56
chore: adding more info about protoc
obecny Jul 13, 2020
bfeaec7
chore: replacing google-protobuf with protobufjs
obecny Jul 13, 2020
7c450b6
chore: merge branch 'master' into collector_proto
obecny Jul 13, 2020
18b0745
chore: cleanup
obecny Jul 13, 2020
5a0cbc5
chore: updating proto
obecny Jul 14, 2020
84afac3
chore: updating proto
obecny Jul 17, 2020
8fb0274
chore: merge with master
obecny Jul 21, 2020
1b1a9ba
chore: merge collector proto
obecny Jul 21, 2020
7448693
chore: merge branch 'collector_proto' into proto_update
obecny Jul 21, 2020
c321638
chore: merge branch 'master' into proto_update
obecny Jul 22, 2020
ca84624
chore: fixes for tests
obecny Jul 22, 2020
ef6020f
chore: merge branch 'master' into proto_update
obecny Jul 22, 2020
90e995f
chore: fixes after merge
obecny Jul 22, 2020
a61a261
chore: fixes after merge
obecny Jul 22, 2020
48740ea
chore: cleanup
obecny Jul 22, 2020
8acc46f
Merge branch 'proto_update' of github.com:obecny/opentelemetry-js int…
obecny Jul 22, 2020
f2fa357
chore: readme
obecny Jul 22, 2020
b007d88
Merge branch 'master' into proto_update
obecny Jul 22, 2020
21904d6
chore: adding unit test for mixed values with array and maps
obecny Jul 22, 2020
0ad57e8
chore: aligning default headers with node version as when using beaco…
obecny Jul 24, 2020
23af355
Merge branch 'master' into proto_update
obecny Jul 24, 2020
4a16ac6
Merge branch 'master' into proto_update
obecny Jul 27, 2020
905a5d6
Merge branch 'master' into proto_update
dyladan Jul 27, 2020
4d2e1dd
Merge branch 'master' into proto_update
obecny Jul 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions examples/collector-exporter-node/docker/collector-config.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
receivers:
otlp:
{}
# keep it when upgrading to version 0.5+
# protocols:
# grpc:
# http:
# endpoint: localhost:55680
protocols:
grpc:
http:
cors_allowed_origins:
- http://*
- https://*

exporters:
zipkin:
url: "http://zipkin-all-in-one:9411/api/v2/spans"
endpoint: "http://zipkin-all-in-one:9411/api/v2/spans"

processors:
batch:
Expand Down
11 changes: 10 additions & 1 deletion examples/collector-exporter-node/docker/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,26 @@ version: "3"
services:
# Collector
collector:
image: otel/opentelemetry-collector:0.4.0
image: otel/opentelemetry-collector:latest
# image: otel/opentelemetry-collector:0.6.0
command: ["--config=/conf/collector-config.yaml", "--log-level=DEBUG"]
networks:
- otelcol
volumes:
- ./collector-config.yaml:/conf/collector-config.yaml
ports:
- "55680:55680"
- "55681:55681"
depends_on:
- zipkin-all-in-one

# Zipkin
zipkin-all-in-one:
image: openzipkin/zipkin:latest
networks:
- otelcol
ports:
- "9411:9411"

networks:
otelcol:
10 changes: 10 additions & 0 deletions examples/collector-exporter-node/start.js
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
'use strict';

const opentelemetry = require('@opentelemetry/api');
// const { ConsoleLogger, LogLevel} = require('@opentelemetry/core');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this?

Copy link
Member Author

@obecny obecny Jul 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm keeping them for easier debugging only so that I can see which option has been chosen how the spans looks like etc. and recently have been using this quite heavily. Although not strong against not removing if you think they shouldn't be. But this is just an example mainly for devs so I guess it might be useful to have it in one place. WDYT ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mayurkale22

// logger: new ConsoleLogger(LogLevel.DEBUG),

and here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this is fine since it is an example.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mayurkale22 ok to merge?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

const { BasicTracerProvider, ConsoleSpanExporter, SimpleSpanProcessor } = require('@opentelemetry/tracing');
const { CollectorTraceExporter, CollectorProtocolNode } = require('@opentelemetry/exporter-collector');

const exporter = new CollectorTraceExporter({
serviceName: 'basic-service',
// logger: new ConsoleLogger(LogLevel.DEBUG),
// headers: {
// foo: 'bar'
// },
Expand Down Expand Up @@ -48,6 +50,14 @@ function doWork(parent) {
// Set attributes to the span.
span.setAttribute('key', 'value');

span.setAttribute('mapAndArrayValue', [
0, 1, 2.25, 'otel', {
foo: 'bar',
baz: 'json',
array: [1, 2, 'boom'],
},
]);

// Annotate our span to capture metadata about our operation
span.addEvent('invoking doWork');

Expand Down
2 changes: 1 addition & 1 deletion examples/tracer-web/examples/user-interaction/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ const providerWithZone = new WebTracerProvider({
plugins: [
new UserInteractionPlugin(),
new XMLHttpRequestPlugin({
ignoreUrls: [/localhost:8090\/sockjs-node/],
ignoreUrls: [/localhost/],
propagateTraceHeaderCorsUrls: [
'http://localhost:8090'
]
Expand Down
29 changes: 26 additions & 3 deletions packages/opentelemetry-exporter-collector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import { WebTracerProvider } from '@opentelemetry/web';
import { CollectorTraceExporter } from '@opentelemetry/exporter-collector';

const collectorOptions = {
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is http://localhost:55680/v1/trace
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is http://localhost:55681/v1/trace
headers: {}, //an optional object containing custom headers to be sent with each request
};

Expand Down Expand Up @@ -118,10 +118,33 @@ const { CollectorExporter, CollectorTransportNode } = require('@opentelemetry/e
const collectorOptions = {
protocolNode: CollectorTransportNode.HTTP_JSON,
serviceName: 'basic-service',
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is http://localhost:55680/v1/trace
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is http://localhost:55681/v1/trace
headers: {
foo: 'bar'
}, //an optional object containing custom headers to be sent with each request will only work with json over http
}, //an optional object containing custom headers to be sent with each request will only work with http
};

const provider = new BasicTracerProvider();
const exporter = new CollectorExporter(collectorOptions);
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));

provider.register();

```

## Usage in Node - PROTO over http

```js
const { BasicTracerProvider, SimpleSpanProcessor } = require('@opentelemetry/tracing');
const { CollectorExporter, CollectorTransportNode } = require('@opentelemetry/exporter-collector');

const collectorOptions = {
protocolNode: CollectorTransportNode.HTTP_PROTO,
serviceName: 'basic-service',
url: '<opentelemetry-collector-url>', // url is optional and can be omitted - default is http://localhost:55681/v1/trace
headers: {
foo: 'bar'
}, //an optional object containing custom headers to be sent with each request will only work with http
};

const provider = new BasicTracerProvider();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import { sendWithBeacon, sendWithXhr } from './util';
import { parseHeaders } from '../../util';

const DEFAULT_SERVICE_NAME = 'collector-trace-exporter';
const DEFAULT_COLLECTOR_URL = 'http://localhost:55680/v1/trace';
const DEFAULT_COLLECTOR_URL = 'http://localhost:55681/v1/trace';

/**
* Collector Trace Exporter for Web
Expand All @@ -35,9 +35,6 @@ export class CollectorTraceExporter
collectorTypes.opentelemetryProto.collector.trace.v1.ExportTraceServiceRequest
>
implements SpanExporter {
DEFAULT_HEADERS: Record<string, string> = {
[collectorTypes.OT_REQUEST_HEADER]: '1',
};
private _headers: Record<string, string>;
private _useXHR: boolean = false;

Expand All @@ -46,10 +43,13 @@ export class CollectorTraceExporter
*/
constructor(config: CollectorExporterConfigBrowser = {}) {
super(config);
this._headers =
parseHeaders(config.headers, this.logger) || this.DEFAULT_HEADERS;
this._useXHR =
!!config.headers || typeof navigator.sendBeacon !== 'function';
if (this._useXHR) {
this._headers = parseHeaders(config.headers, this.logger);
} else {
this._headers = {};
}
}

onInit(): void {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import { toCollectorExportMetricServiceRequest } from '../../transformMetrics';

const DEFAULT_SERVICE_NAME = 'collector-metric-exporter';
const DEFAULT_COLLECTOR_URL_GRPC = 'localhost:55680';
const DEFAULT_COLLECTOR_URL_JSON = 'http://localhost:55680/v1/metrics';
const DEFAULT_COLLECTOR_URL_JSON = 'http://localhost:55681/v1/metrics';

/**
* Collector Metric Exporter for Node
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import { toCollectorExportTraceServiceRequest } from '../../transform';

const DEFAULT_SERVICE_NAME = 'collector-trace-exporter';
const DEFAULT_COLLECTOR_URL_GRPC = 'localhost:55680';
const DEFAULT_COLLECTOR_URL_JSON = 'http://localhost:55680/v1/trace';
const DEFAULT_COLLECTOR_URL_JSON_PROTO = 'http://localhost:55680/v1/trace';
const DEFAULT_COLLECTOR_URL_JSON = 'http://localhost:55681/v1/trace';
const DEFAULT_COLLECTOR_URL_JSON_PROTO = 'http://localhost:55681/v1/trace';

/**
* Collector Trace Exporter for Node
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ export function removeProtocol(url: string): string {
* @param onSuccess
* @param onError
*/
export function sendDataUsingHttp<ExportItem, ServiceRequest>(
export function sendWithHttp<ExportItem, ServiceRequest>(
collector: CollectorExporterNodeBase<ExportItem, ServiceRequest>,
data: string | Buffer,
contentType: string,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import * as collectorTypes from '../../types';
import { CollectorExporterNodeBase } from './CollectorExporterNodeBase';
import { CollectorExporterConfigNode } from './types';
import { sendDataUsingHttp } from './util';
import { sendWithHttp } from './util';

export function initWithJson<ExportItem, ServiceRequest>(
_collector: CollectorExporterNodeBase<ExportItem, ServiceRequest>,
Expand All @@ -34,7 +34,7 @@ export function sendWithJson<ExportItem, ServiceRequest>(
): void {
const serviceRequest = collector.convert(objects);

sendDataUsingHttp(
sendWithHttp(
collector,
JSON.stringify(serviceRequest),
'application/json',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import * as protobufjs from 'protobufjs';
import * as collectorTypes from '../../types';
import { CollectorExporterNodeBase } from './CollectorExporterNodeBase';
import { CollectorExporterConfigNode } from './types';
import { sendDataUsingHttp } from './util';
import { sendWithHttp } from './util';

let ExportTraceServiceRequestProto: Type | undefined;

Expand Down Expand Up @@ -60,7 +60,7 @@ export function sendWithJsonProto<ExportItem, ServiceRequest>(
if (message) {
const body = ExportTraceServiceRequestProto?.encode(message).finish();
if (body) {
sendDataUsingHttp(
sendWithHttp(
collector,
Buffer.from(body),
'application/x-protobuf',
Expand Down
66 changes: 49 additions & 17 deletions packages/opentelemetry-exporter-collector/src/transform.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,47 +30,79 @@ import {
opentelemetryProto,
CollectorExporterConfigBase,
} from './types';
import ValueType = opentelemetryProto.common.v1.ValueType;

/**
* Converts attributes
* Converts attributes to KeyValue array
* @param attributes
*/
export function toCollectorAttributes(
attributes: Attributes
): opentelemetryProto.common.v1.AttributeKeyValue[] {
): opentelemetryProto.common.v1.KeyValue[] {
return Object.keys(attributes).map(key => {
return toCollectorAttributeKeyValue(key, attributes[key]);
});
}

/**
* Converts key and value to AttributeKeyValue
* Converts array of unknown value to ArrayValue
* @param values
*/
export function toCollectorArrayValue(
values: unknown[]
): opentelemetryProto.common.v1.ArrayValue {
return {
values: values.map(value => toCollectorAnyValue(value)),
};
}

/**
* Converts attributes to KeyValueList
* @param attributes
*/
export function toCollectorKeyValueList(
attributes: Attributes
): opentelemetryProto.common.v1.KeyValueList {
return {
values: toCollectorAttributes(attributes),
};
}

/**
* Converts key and unknown value to KeyValue
* @param value event value
*/
export function toCollectorAttributeKeyValue(
key: string,
value: unknown
): opentelemetryProto.common.v1.AttributeKeyValue {
let aType: opentelemetryProto.common.v1.ValueType = ValueType.STRING;
const AttributeKeyValue: opentelemetryProto.common.v1.AttributeKeyValue = {
): opentelemetryProto.common.v1.KeyValue {
const anyValue = toCollectorAnyValue(value);
return {
key,
type: 0,
value: anyValue,
};
}

/**
* Converts unknown value to AnyValue
* @param value
*/
export function toCollectorAnyValue(
value: unknown
): opentelemetryProto.common.v1.AnyValue {
const anyValue: opentelemetryProto.common.v1.AnyValue = {};
if (typeof value === 'string') {
AttributeKeyValue.stringValue = value;
anyValue.stringValue = value;
} else if (typeof value === 'boolean') {
aType = ValueType.BOOL;
AttributeKeyValue.boolValue = value;
anyValue.boolValue = value;
} else if (typeof value === 'number') {
// all numbers will be treated as double
aType = ValueType.DOUBLE;
AttributeKeyValue.doubleValue = value;
anyValue.doubleValue = value;
} else if (Array.isArray(value)) {
anyValue.arrayValue = toCollectorArrayValue(value);
} else if (value) {
anyValue.kvlistValue = toCollectorKeyValueList(value as Attributes);
}

AttributeKeyValue.type = aType;

return AttributeKeyValue;
return anyValue;
}

/**
Expand Down
Loading