Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[processor/deltatocumulative]: progress tracking #30705

Open
12 of 25 tasks
sh0rez opened this issue Jan 22, 2024 · 2 comments
Open
12 of 25 tasks

[processor/deltatocumulative]: progress tracking #30705

sh0rez opened this issue Jan 22, 2024 · 2 comments
Labels
Accepted Component New component has been sponsored admin issues tracker issues etc.

Comments

@sh0rez
Copy link
Contributor

sh0rez commented Jan 22, 2024 •
edited

Important

This is strictly for progress tracking. Please do any discussion in #30479
@sh0rez sh0rez added the needs triage New item requiring triage label Jan 22, 2024
This was referenced Jan 22, 2024
Merged
Merged
Open
@github-actions github-actions bot mentioned this issue Jan 23, 2024
Closed
@crobert-1 crobert-1 added admin issues tracker issues etc. Accepted Component New component has been sponsored and removed needs triage New item requiring triage labels Jan 23, 2024
@sh0rez sh0rez mentioned this issue Feb 15, 2024
Closed
6 tasks
jpkrohling pushed a commit that referenced this issue Feb 15, 2024
@sh0rez
[processor/deltatocumulative]: Sums (#30707)
c0fe1b8 
**Description:**
Implements major component functionality:

- [x] metrics identification (`metrics.Ident`) and stream identification
(`streams.Ident`)
- [x] abstract data layer `data.Point` that keeps processing code data
type (sum, histogram, exp histogram) agnostic
- [x] `delta.Accumulator` stream processor for accumulating any
`data.Point`

**Link to tracking Issue:**
#30705

**Testing:**  Done

**Documentation:** <kbd>TODO</kbd>
djaglowski pushed a commit to djaglowski/opentelemetry-collector-contrib that referenced this issue Feb 19, 2024
@sh0rez @djaglowski
[processor/deltatocumulative]: Sums (open-telemetry#30707)
64a6f88 
**Description:**
Implements major component functionality:

- [x] metrics identification (`metrics.Ident`) and stream identification
(`streams.Ident`)
- [x] abstract data layer `data.Point` that keeps processing code data
type (sum, histogram, exp histogram) agnostic
- [x] `delta.Accumulator` stream processor for accumulating any
`data.Point`

**Link to tracking Issue:**
open-telemetry#30705

**Testing:**  Done

**Documentation:** <kbd>TODO</kbd>
djaglowski pushed a commit to djaglowski/opentelemetry-collector-contrib that referenced this issue Feb 19, 2024
@sh0rez @djaglowski
[processor/deltatocumulative]: Sums (open-telemetry#30707)
d0b5e68 
**Description:**
Implements major component functionality:

- [x] metrics identification (`metrics.Ident`) and stream identification
(`streams.Ident`)
- [x] abstract data layer `data.Point` that keeps processing code data
type (sum, histogram, exp histogram) agnostic
- [x] `delta.Accumulator` stream processor for accumulating any
`data.Point`

**Link to tracking Issue:**
open-telemetry#30705

**Testing:**  Done

**Documentation:** <kbd>TODO</kbd>
This was referenced Feb 20, 2024
Merged
Merged
jpkrohling pushed a commit that referenced this issue Mar 5, 2024
@sh0rez
[processor/deltatocumulative]: expire stale series (#31337)
f003e0e 
**Description:** Removes stale series from tracking (and thus frees
their memory) using staleness logic from
#31089

**Link to tracking Issue:**
#30705,
#31016

**Testing:** `TestExpiry`
**Documentation:** README updated
@burnjake
Copy link

burnjake commented Mar 7, 2024

Just adding that I have a use case where we are trying to send metrics from a high number of mobile clients but our metrics backend has a hard limit on the frequency of sample ingestion (GCP, once every 5s). This means we need to aggregate before sending them which sounds like an intended use for this ;) I volunteer as tribute to help test if this would help at all.

@sh0rez
Copy link
Contributor Author

sh0rez commented Mar 7, 2024

@burnjake this is also in progress! see #29461 for details.

on a side note, please keep ideas / requests / discussions to the design issue (#30479), as this one is just for tracking progress ;)

DougManton pushed a commit to DougManton/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
@sh0rez @DougManton
[processor/deltatocumulative]: expire stale series (open-telemetry#31337
a9fdc0b 
)

**Description:** Removes stale series from tracking (and thus frees
their memory) using staleness logic from
open-telemetry#31089

**Link to tracking Issue:**
open-telemetry#30705,
open-telemetry#31016

**Testing:** `TestExpiry`
**Documentation:** README updated
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
@sh0rez @XinRanZhAWS
[processor/deltatocumulative]: Sums (open-telemetry#30707)
2a3e0f0 
**Description:**
Implements major component functionality:

- [x] metrics identification (`metrics.Ident`) and stream identification
(`streams.Ident`)
- [x] abstract data layer `data.Point` that keeps processing code data
type (sum, histogram, exp histogram) agnostic
- [x] `delta.Accumulator` stream processor for accumulating any
`data.Point`

**Link to tracking Issue:**
open-telemetry#30705

**Testing:**  Done

**Documentation:** <kbd>TODO</kbd>
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this issue Mar 13, 2024
@sh0rez @XinRanZhAWS
[processor/deltatocumulative]: expire stale series (open-telemetry#31337
d97dcaa 
)

**Description:** Removes stale series from tracking (and thus frees
their memory) using staleness logic from
open-telemetry#31089

**Link to tracking Issue:**
open-telemetry#30705,
open-telemetry#31016

**Testing:** `TestExpiry`
**Documentation:** README updated
@sh0rez sh0rez mentioned this issue Mar 28, 2024
Open
4 tasks
jpkrohling pushed a commit that referenced this issue Apr 10, 2024
@sh0rez
[processor/deltatocumulative]: observe accumulation metrics (#31363)
655bfa7 
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->

Uses the otel meter to observe some key metrics about the accumulation:

| Name | Description |

|------------------------------------------|---------------------------------------------------------------------------------------|
| `streams_count` | Number of streams currently tracked by the
aggregation state |
| `datapoints_processed` | Total number of datapoints processed, whether
successful or not |
| `datapoints_dropped` | Faulty datapoints that were dropped due to the
reason given in the `reason` attribute |
| `seconds_lost` | Total length of all gaps in the streams, which occur
e.g. due to lost in transit |

**Link to tracking Issue:**

#30705

**Testing:** None

**Documentation:** Readme updated
@zhehao-grafana zhehao-grafana mentioned this issue Apr 25, 2024
Open
rimitchell pushed a commit to rimitchell/opentelemetry-collector-contrib that referenced this issue May 8, 2024
@sh0rez @rimitchell
[processor/deltatocumulative]: observe accumulation metrics (open-tel…
38c39ea 
…emetry#31363)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->

Uses the otel meter to observe some key metrics about the accumulation:

| Name | Description |

|------------------------------------------|---------------------------------------------------------------------------------------|
| `streams_count` | Number of streams currently tracked by the
aggregation state |
| `datapoints_processed` | Total number of datapoints processed, whether
successful or not |
| `datapoints_dropped` | Faulty datapoints that were dropped due to the
reason given in the `reason` attribute |
| `seconds_lost` | Total length of all gaps in the streams, which occur
e.g. due to lost in transit |

**Link to tracking Issue:**

open-telemetry#30705

**Testing:** None

**Documentation:** Readme updated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Accepted Component New component has been sponsored admin issues tracker issues etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@burnjake @sh0rez @crobert-1