Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] [exporterhelper] Remove retry sender -> queue sender callback #8942

Closed

Conversation

dmitryax
Copy link
Member

Use returned error instead to simplify the senders feedback loop

@dmitryax dmitryax requested a review from a team as a code owner November 17, 2023 02:38
Copy link

codecov bot commented Nov 17, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (7e3e725) 91.55% compared to head (d84c40a) 91.55%.

Files Patch % Lines
...porter/exporterhelper/internal/persistent_queue.go 0.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8942      +/-   ##
==========================================
- Coverage   91.55%   91.55%   -0.01%     
==========================================
  Files         315      315              
  Lines       17117    17105      -12     
==========================================
- Hits        15672    15660      -12     
  Misses       1150     1150              
  Partials      295      295              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dmitryax dmitryax force-pushed the remove-retry-queue-callback branch 6 times, most recently from ae14cff to c25c88e Compare November 18, 2023 00:02
@dmitryax dmitryax marked this pull request as draft November 18, 2023 18:03
@dmitryax dmitryax force-pushed the remove-retry-queue-callback branch 2 times, most recently from 39c1e38 to 94a70c0 Compare November 18, 2023 19:59
@dmitryax dmitryax marked this pull request as ready for review November 18, 2023 19:59
@dmitryax
Copy link
Member Author

The coverage will be improved after #8960 is merged

@dmitryax
Copy link
Member Author

This PR introduces a significant change in behavior: re-enqueueing in the persistent queue will be always enabled even if the retry sender is disabled. I'm turning it into a draft for now until it's properly addressed.

@dmitryax dmitryax marked this pull request as draft November 19, 2023 21:52
bogdandrutu pushed a commit that referenced this pull request Nov 21, 2023
)

Update the test validating failed re-enqueue to ensure that the request
is sent through the queue sender, not the retry sender. It'll unblock
removing the dependency on the retry sender for the re-enqueue as done
in #8942
@dmitryax
Copy link
Member Author

This PR introduces a significant change in behavior: re-enqueueing in the persistent queue will be always enabled even if the retry sender is disabled. I'm turning it into a draft for now until it's properly addressed.

This is now resolved by introducing a separate config option for re-enqueue.

@dmitryax dmitryax force-pushed the remove-retry-queue-callback branch 2 times, most recently from 2b3035d to 9727f41 Compare November 22, 2023 01:23
@dmitryax dmitryax marked this pull request as ready for review November 22, 2023 01:38
@dmitryax
Copy link
Member Author

I'm going to break this PR down

@dmitryax dmitryax marked this pull request as draft November 22, 2023 01:47
dmitryax added a commit that referenced this pull request Nov 22, 2023
Remove the QueueRequest wrapper. Pass consumeFunc to the queue instead
of asking for the QueueRequest wrapper with callback. The callback isn't
needed for the memory queue, and the field for the callback is occupying
the memory even it's always empty. The bounded queue benchmark results
show the RAM savings:

Before:
```
Benchmark_QueueUsage_50000_1_50000
Benchmark_QueueUsage_50000_1_50000-10    	       1	69813548417 ns/op	 2806232 B/op	   99753 allocs/op
```

After:
```
Benchmark_QueueUsage_50000_1_50000
Benchmark_QueueUsage_50000_1_50000-10    	       1	65498055709 ns/op	 2404832 B/op	   99754 allocs/op
```

Cut from
#8942
This change changes the re-enqueue capability of the queue sender to not depend on the retry sender and have a separate configuration for that
@dmitryax
Copy link
Member Author

Closing in favor of #8985

@dmitryax dmitryax closed this Nov 22, 2023
@dmitryax dmitryax deleted the remove-retry-queue-callback branch November 27, 2023 05:38
pantuza pushed a commit to pantuza/opentelemetry-collector that referenced this pull request Dec 8, 2023
…en-telemetry#8960)

Update the test validating failed re-enqueue to ensure that the request
is sent through the queue sender, not the retry sender. It'll unblock
removing the dependency on the retry sender for the re-enqueue as done
in open-telemetry#8942
pantuza pushed a commit to pantuza/opentelemetry-collector that referenced this pull request Dec 8, 2023
…etry#8982)

Remove the QueueRequest wrapper. Pass consumeFunc to the queue instead
of asking for the QueueRequest wrapper with callback. The callback isn't
needed for the memory queue, and the field for the callback is occupying
the memory even it's always empty. The bounded queue benchmark results
show the RAM savings:

Before:
```
Benchmark_QueueUsage_50000_1_50000
Benchmark_QueueUsage_50000_1_50000-10    	       1	69813548417 ns/op	 2806232 B/op	   99753 allocs/op
```

After:
```
Benchmark_QueueUsage_50000_1_50000
Benchmark_QueueUsage_50000_1_50000-10    	       1	65498055709 ns/op	 2404832 B/op	   99754 allocs/op
```

Cut from
open-telemetry#8942
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant