Add Prometheus Remote Write Exporter (1/6) #180

shovnik · 2020-11-12T20:35:54Z

Description

This is PR 1/6 of adding a Prometheus Remote Write Exporter in Python SDK and address Issue open-telemetry/opentelemetry-python#1302

👉 Part 1/6

Adds class skeleton
Adds all function signatures

Part 2/6

Adds validation of exporter constructor commands
Add unit tests for validation

Part 3/6

Adds conversion methods from OTel metric types to Prometheus TimeSeries
Add unit tests for conversion

Part 4/6

Adds methods to export metrics to Remote Write endpoint
Add unit tests for exporting

Part 5/6

Add integration tests using sample app and instance of Cortex

Part 6/6

Add README, Design Doc and other necessary documentation.

Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Added class TestValidation in test_prometheus_remote_write_exporter.py

Does This PR Require a Core Repo Change?

Yes. - Link to PR:
No.

Checklist:

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated (Full documentation in final exporter PR)
cc- @AzfaarQureshi , @alolita

codeboten

A few minor changes needed, would be good to add some info regarding the source of the protos for prometheus as well.

codeboten · 2020-11-24T06:24:39Z

exporter/opentelemetry-exporter-prometheus-remote-write/README.rst

+References
+----------
+
+* `Datadog <https://prometheus.io/>`_


Suggested change

* `Datadog <https://prometheus.io/>`_

* `Prometheus <https://prometheus.io/>`_

codeboten · 2020-11-24T06:25:09Z

exporter/opentelemetry-exporter-prometheus-remote-write/README.rst

+OpenTelemetry Prometheus Remote Write Exporter
+==============================================
+
+This library allows exporting metric data to `Prometheus Write Integrated Backends


Should this say Prometheus Remote Write?

codeboten · 2020-11-24T06:25:45Z

exporter/opentelemetry-exporter-prometheus-remote-write/setup.cfg

+long_description_content_type = text/x-rst
+author = OpenTelemetry Authors
+author_email = cncf-opentelemetry-contributors@lists.cncf.io
+url = https://github.com/open-telemetry/opentelemetry-python/exporter/opentelemetry-exporter-datadog


url needs updating

codeboten · 2020-11-24T06:26:02Z

exporter/opentelemetry-exporter-prometheus-remote-write/setup.cfg

+    =src
+packages=find_namespace:
+install_requires =
+    ddtrace>=0.34.0


is ddtrace necessary?

No, it is a remnant from the datadog exporter I used as template, removed.

lzchen · 2020-11-24T14:29:12Z

exporter/opentelemetry-exporter-prometheus-remote-write/CHANGELOG.md

@@ -0,0 +1,3 @@
+# Changelog
+
+## Unreleased


I would include "initial release" and the PRs associated.

Added Initial Release in CHANGELOG and will update it with each subsequent PR.

lzchen · 2020-11-24T14:30:37Z

Did we decide to create this in the contrib repo? I thought we wanted to keep this in the same place as the prometheus exporter to avoid confusion?

shovnik · 2020-11-24T16:44:24Z

We were not sure where the exporter would reside so thought we would make the PRs on the contrib repo for now to get the reviews going. Once the standard location for prometheus exporters is decided, we can move one or the other.

shovnik · 2020-11-24T17:22:21Z

I addressed all the requested changes @lzchen @codeboten

toumorokoshi · 2020-11-25T05:48:56Z

@lzchen @codeboten from a high level, how do you want to handle reviewing this six-parter?

It may be more efficient for the same people to review all PRs. happy to volunteer.

Currently I'm not too inclined to review 3/6 which I was assigned, since I have to go read 1 and 2 first anyway.

toumorokoshi · 2020-11-25T05:52:05Z

...-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/gen/remote.proto

@@ -0,0 +1,82 @@
+// Copyright 2016 Prometheus Team


it looks like these are copied from the prometheus library itself. Probably fine.

Are there any thoughts around what the update story would look like? just copy the protos over I suppose.

just copy the protos over I suppose.

Yup! thats the plan currently

lzchen · 2020-11-25T15:42:08Z

@toumorokoshi

I'm going to be reviewing all 6. Others are more than welcome to chime in as well.

codeboten · 2020-11-26T00:09:48Z

exporter/opentelemetry-exporter-prometheus-remote-write/CHANGELOG.md

+
+## Unreleased
+
+## Initial Release


nit:
this change can just be under the ## Unreleased section, when the release is done, the correct version tag title will be applied to the changelog

Rebased and updated CHANGELOG to use Unreleased

codeboten

Changes look good, holding off on merging until we get this release out to prevent releasing the prometheus package before the other PRs are merged

codeboten

Please update the branch and we can merge!

shovnik requested a review from a team as a code owner November 12, 2020 20:35

shovnik requested review from codeboten and ocelotl and removed request for a team November 12, 2020 20:35

shovnik force-pushed the SetupPrometheusRWExporter branch 20 times, most recently from 0c4ba24 to 31b910d Compare November 16, 2020 19:12

shovnik changed the title ~~Setup Prometheus Remote Write Exporter (WIP)~~ Setup Prometheus Remote Write Exporter PR 1/7 Nov 19, 2020

shovnik changed the title ~~Setup Prometheus Remote Write Exporter PR 1/7~~ Setup Prometheus Remote Write Exporter PR 1/7 (WIP) Nov 19, 2020

shovnik force-pushed the SetupPrometheusRWExporter branch 4 times, most recently from d0cab05 to a8c4ffc Compare November 20, 2020 00:06

shovnik mentioned this pull request Nov 23, 2020

Add Prometheus Remote Write Exporter (3/6) #207

Merged

8 tasks

AzfaarQureshi mentioned this pull request Nov 23, 2020

Prometheus Remote Write Exporter (4/7) #208

Closed

8 tasks

codeboten suggested changes Nov 24, 2020

View reviewed changes

lzchen reviewed Nov 24, 2020

View reviewed changes

shovnik force-pushed the SetupPrometheusRWExporter branch from a1f5c7d to 61bc919 Compare November 24, 2020 16:50

shovnik force-pushed the SetupPrometheusRWExporter branch from 61bc919 to 9ca8b3e Compare November 24, 2020 17:34

shovnik changed the title ~~Prometheus Remote Write Exporter PR (1/7)~~ Prometheus Remote Write Exporter PR (1/6) Nov 24, 2020

shovnik mentioned this pull request Nov 24, 2020

Add Prometheus Remote Write Exporter (4/6) #212

Merged

8 tasks

AzfaarQureshi mentioned this pull request Nov 24, 2020

Prometheus Remote Write Exporter (5/6) open-o11y/opentelemetry-python-contrib#13

Closed

8 tasks

shovnik changed the title ~~Prometheus Remote Write Exporter PR (1/6)~~ Add Prometheus Remote Write Exporter (1/6) Nov 24, 2020

toumorokoshi reviewed Nov 25, 2020

View reviewed changes

lzchen approved these changes Nov 25, 2020

View reviewed changes

AzfaarQureshi mentioned this pull request Nov 25, 2020

Prometheus Remote Write Exporter (5/6) #216

Merged

8 tasks

codeboten reviewed Nov 26, 2020

View reviewed changes

ocelotl approved these changes Nov 26, 2020

View reviewed changes

shovnik force-pushed the SetupPrometheusRWExporter branch from d063662 to 7d7883f Compare November 26, 2020 17:40

codeboten approved these changes Nov 26, 2020

View reviewed changes

shovnik force-pushed the SetupPrometheusRWExporter branch 2 times, most recently from 59b0634 to 83e15ea Compare November 27, 2020 17:27

Setup Prometheus Remote Write Exporter

40fc34e

shovnik force-pushed the SetupPrometheusRWExporter branch from 83e15ea to 40fc34e Compare November 27, 2020 17:39

codeboten merged commit 9b71f8b into open-telemetry:master Nov 27, 2020

This was referenced Nov 27, 2020

Prometheus Remote Write Exporter (6/6) open-o11y/opentelemetry-python-contrib#14

Closed

Prometheus Remote Write Exporter (6/6) #227

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Prometheus Remote Write Exporter (1/6) #180

Add Prometheus Remote Write Exporter (1/6) #180

shovnik commented Nov 12, 2020 •

edited

Loading

codeboten left a comment

codeboten Nov 24, 2020

shovnik Nov 24, 2020

codeboten Nov 24, 2020

shovnik Nov 24, 2020

codeboten Nov 24, 2020

shovnik Nov 24, 2020

codeboten Nov 24, 2020

shovnik Nov 24, 2020

lzchen Nov 24, 2020

shovnik Nov 24, 2020

lzchen commented Nov 24, 2020

shovnik commented Nov 24, 2020

shovnik commented Nov 24, 2020

toumorokoshi commented Nov 25, 2020

toumorokoshi Nov 25, 2020

AzfaarQureshi Nov 25, 2020 •

edited

Loading

lzchen commented Nov 25, 2020

codeboten Nov 26, 2020

shovnik Nov 26, 2020

codeboten left a comment

codeboten left a comment

	* `Datadog <https://prometheus.io/>`_
	* `Prometheus <https://prometheus.io/>`_

Add Prometheus Remote Write Exporter (1/6) #180

Add Prometheus Remote Write Exporter (1/6) #180

Conversation

shovnik commented Nov 12, 2020 • edited Loading

Description

Type of change

How Has This Been Tested?

Does This PR Require a Core Repo Change?

Checklist:

codeboten left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzchen commented Nov 24, 2020

shovnik commented Nov 24, 2020

shovnik commented Nov 24, 2020

toumorokoshi commented Nov 25, 2020

Choose a reason for hiding this comment

AzfaarQureshi Nov 25, 2020 • edited Loading

Choose a reason for hiding this comment

lzchen commented Nov 25, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codeboten left a comment

Choose a reason for hiding this comment

codeboten left a comment

Choose a reason for hiding this comment

shovnik commented Nov 12, 2020 •

edited

Loading

AzfaarQureshi Nov 25, 2020 •

edited

Loading