feat(collector-exporter): allow configuration from env #2099 #2101

vmarchaud · 2021-04-11T19:33:51Z

I centralized default configuration inside the main package exporter-collector, each variant of the exporter (for each signal/protocol) is defined with abstract functions.
I'm not sure if everything is spec compliant, there actually a lot of possible mix: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md

I've also changed the default port for HTTP which seems to be 4317 for all protocols: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md#otlphttp-default-port

Since the headers configuration is using the same correlation format that the baggage, i modified the core package to export utils to parse them (i expect more configurations will use this later on)

Fixes #2099

…#2099

codecov · 2021-04-11T19:39:06Z

Codecov Report

Merging #2101 (1f96179) into main (9fc1b10) will increase coverage by 1.31%.
The diff coverage is 92.22%.

@@            Coverage Diff             @@
##             main    #2101      +/-   ##
==========================================
+ Coverage   92.76%   94.07%   +1.31%     
==========================================
  Files         137       50      -87     
  Lines        4904     1721    -3183     
  Branches     1012      379     -633     
==========================================
- Hits         4549     1619    -2930     
+ Misses        355      102     -253

Impacted Files	Coverage Δ
...ckages/opentelemetry-core/src/utils/environment.ts	`100.00% <ø> (ø)`
...ages/opentelemetry-exporter-collector/src/types.ts	`100.00% <ø> (ø)`
...ry-exporter-collector/src/CollectorExporterBase.ts	`89.81% <88.13%> (-2.50%)`	⬇️
...emetry-core/src/baggage/propagation/HttpBaggage.ts	`96.77% <100.00%> (-1.48%)`	⬇️
packages/opentelemetry-core/src/baggage/utils.ts	`100.00% <100.00%> (ø)`
...s/opentelemetry-core/src/platform/node/sdk-info.ts	`0.00% <0.00%> (-100.00%)`	⬇️
...pentelemetry-core/src/platform/node/performance.ts	`0.00% <0.00%> (-100.00%)`	⬇️
...emetry-api-metrics/src/platform/node/globalThis.ts	`0.00% <0.00%> (-100.00%)`	⬇️
...ckages/opentelemetry-exporter-zipkin/src/zipkin.ts	`81.63% <0.00%> (-18.37%)`	⬇️
...s/opentelemetry-metrics/src/BatchObserverResult.ts
... and 88 more

dyladan · 2021-04-12T16:03:44Z

Seems like quite a few changes are here which are not related to environment

vmarchaud · 2021-04-12T16:10:47Z

Seems like quite a few changes are here which are not related to environment

Because all the defaults was repeated in each package and each platform, i refactored it to centralize in a single place the config for all packages.

vmarchaud · 2021-04-14T07:48:06Z

Even though there are conflicts + tests are failing, i would like some reviews to know if we agree on the way of doing it

obecny

I think there are too many changes here that are moved back to base class and they should not be there. Each class of different collector that inherits from base class has its own params that are not needed elsewhere. You have 6 exporters in total. For example headers are - they are not needed in grpc but are needed in in http and browser. I think it will be less errors and changes to just update few places with those few env variables. Trace / span exporter should be interested in 2 variable only, the span exporter for grpc one variable. etc. and the same for metric exporter. Those can be really a small changes.

obecny · 2021-04-14T21:42:04Z

packages/opentelemetry-core/src/baggage/utils.ts

+
+export const parsePairKeyValue = (entry: string) => {
+  const valueProps = entry.split(PROPERTIES_SEPARATOR);
+  if (valueProps.length <= 0) return;


Nit / suggestions

Suggested change

if (valueProps.length <= 0) return;

if (valueProps.length <= 0) {

return;

}

obecny · 2021-04-14T21:43:12Z

packages/opentelemetry-core/src/baggage/utils.ts

+  if (!keyPairPart) return;
+  const keyPair = keyPairPart.split(KEY_PAIR_SEPARATOR);
+  if (keyPair.length !== 2) return;
+  const key = decodeURIComponent(keyPair[0].trim());


Nit: line space

obecny · 2021-04-14T21:44:09Z

packages/opentelemetry-core/src/utils/environment.ts

@@ -69,6 +69,12 @@ export type ENVIRONMENT = {
  OTEL_EXPORTER_JAEGER_USER?: string;
  OTEL_LOG_LEVEL?: DiagLogLevel;
  OTEL_RESOURCE_ATTRIBUTES?: string;
+  OTEL_EXPORTER_OTLP_ENDPOINT?: string;


pls keep it in alphabetical order so it is easier to find / add new one

obecny · 2021-04-14T21:44:17Z

packages/opentelemetry-core/src/utils/environment.ts

@@ -102,6 +108,12 @@ export const DEFAULT_ENVIRONMENT: Required<ENVIRONMENT> = {
  OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT: 1000,
  OTEL_SPAN_EVENT_COUNT_LIMIT: 1000,
  OTEL_SPAN_LINK_COUNT_LIMIT: 1000,
+  OTEL_EXPORTER_OTLP_ENDPOINT: '',


pls keep it in alphabetical order so it is easier to find / add new one

obecny · 2021-04-14T21:45:48Z

packages/opentelemetry-core/src/baggage/utils.ts

+ */
+import { Baggage, baggageEntryMetadataFromString } from '@opentelemetry/api';
+
+export const KEY_PAIR_SEPARATOR = '=';


if you split this already what if you keep all const in separate file ? constants.ts for example ?
and then later in code instead of

baggageUtils.BAGGAGE_HEADER

you will have for example

baggageConst.BAGGAGE_HEADER // or constants.BAGGAGE_HEADER

obecny · 2021-04-14T21:56:43Z

packages/opentelemetry-exporter-collector-grpc/src/util.ts

        collector.serviceClient = new packageObject.opentelemetry.proto.collector.trace.v1.TraceService(
          serverAddress,
          credentials
        );
-      } else {
+      } else if (collector.getExporterType() === 'metric') {


Hmm you have removed the ServiceClientType and then in both metrics and spans added a trace. I think having a predefined enum / const like it was ServiceClientType is pretty straightforward ?

obecny · 2021-04-14T21:58:30Z

packages/opentelemetry-exporter-collector-proto/src/util.ts

@@ -41,15 +40,15 @@ export function onInit<ExportItem, ServiceRequest>(
  root.resolvePath = function (origin, target) {
    return `${dir}/${target}`;
  };
-  if (collector.getServiceClientType() === ServiceClientType.SPANS) {
+  if (collector.getExporterType() === 'trace') {


why removing the ServiceClientType and then using hardcoded value ?

obecny · 2021-04-14T21:59:32Z

packages/opentelemetry-exporter-collector/src/CollectorExporterBase.ts

-    this.url = this.getDefaultUrl(config);
+    this.serviceName = this._getDefaultServiceName(config);
+    this.url = this._getUrl(config);
+    this.headers = this._getHeaders(config);


headers are not needed for grpc so they should not be here ?

obecny · 2021-04-14T22:03:37Z

packages/opentelemetry-exporter-collector/src/CollectorExporterBase.ts

+    }
+  }
+
+  private _getDefaultServiceName(config: T): string {


keeping this in base will get extra overhead for each instance. Like for example metric exporter doesn't need to know the name of trace exporter and opposite. Plus each instance will have extra function for just getting it's name. Whereas you can keep the final name in the specific class.

obecny · 2021-04-14T22:04:34Z

packages/opentelemetry-exporter-collector/src/CollectorExporterBase.ts

+    }
+  }
+
+  private _getHeaders(config: T): Record<string, string> {


the same with headers, base class is used by all 3 protocols, grpc doesn't need this at all. That's why I think the headers should not be a part of base class

obecny · 2021-04-14T22:34:29Z

packages/opentelemetry-exporter-collector/src/CollectorExporterBase.ts

@@ -132,14 +143,106 @@ export abstract class CollectorExporterBase<
    return this._shuttingDownPromise;
  }

+  private _getUrl(config: T): string {


The cyclomatic complexity of this function is really high 16-17 if I counted correctly.
The whole purpose of splitting the classes was to put the final information in the final class like for example name, default url etc.. Imagine now you have 50 different exporters and you want to create a default url for them. I think doing this in base class becomes very complicated as it will produce an enormous complex function, not even mentioning about all possible tests cases.

vmarchaud · 2021-04-17T15:24:51Z

Closing this PR since addressing changes is basically a new PR, see #2117

feat(collector-exporter): allow configuration from env open-telemetry…

1f96179

…#2099

vmarchaud self-assigned this Apr 11, 2021

vmarchaud requested review from dyladan, Flarna, johnbley, legendecas, markwolff, mayurkale22, mwear, naseemkullah, obecny and OlivierAlbertini as code owners April 11, 2021 19:33

obecny reviewed Apr 14, 2021

View reviewed changes

vmarchaud mentioned this pull request Apr 17, 2021

feat(exporter-collector): support config from env #2099 #2117

Merged

vmarchaud closed this Apr 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(collector-exporter): allow configuration from env #2099 #2101

feat(collector-exporter): allow configuration from env #2099 #2101

vmarchaud commented Apr 11, 2021 •

edited

Loading

codecov bot commented Apr 11, 2021

dyladan commented Apr 12, 2021

vmarchaud commented Apr 12, 2021 •

edited

Loading

vmarchaud commented Apr 14, 2021

obecny left a comment

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

obecny Apr 14, 2021

vmarchaud commented Apr 17, 2021

feat(collector-exporter): allow configuration from env #2099 #2101

feat(collector-exporter): allow configuration from env #2099 #2101

Conversation

vmarchaud commented Apr 11, 2021 • edited Loading

codecov bot commented Apr 11, 2021

Codecov Report

dyladan commented Apr 12, 2021

vmarchaud commented Apr 12, 2021 • edited Loading

vmarchaud commented Apr 14, 2021

obecny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vmarchaud commented Apr 17, 2021

vmarchaud commented Apr 11, 2021 •

edited

Loading

vmarchaud commented Apr 12, 2021 •

edited

Loading