Skip to content

Conversation

@shreyamalpani
Copy link
Contributor

@shreyamalpani shreyamalpani commented Oct 27, 2025

Use a channel-based aggregator for traces to avoid locking on aggregation. Adds an aggregator service which receives incoming commands on a channel. Commands can be to insert payloads, get batches, and shutdown.

Follows a similar pattern as #879. Tested with self-monitoring and tested that dual shipping still works as well.

SVLS-7647

@shreyamalpani shreyamalpani marked this pull request as ready for review October 28, 2025 14:16
@shreyamalpani shreyamalpani requested a review from a team as a code owner October 28, 2025 14:16
}
if response_tx.send(batches).is_err() {
error!("Failed to send trace flush response - receiver dropped");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be great to log the error details.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the error here just contains the data that failed to send

Copy link
Contributor

@litianningdatadog litianningdatadog Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am thinking of sth like below. It will give us more context in case of errors.

if let Err(e) = response_tx.send(batches) {
   error!("Failed to send trace flush response : {}", e);
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this, but here the error is the Vec<Vec<SendDataBuilder>> of data that failed to send and doesn't have additional context I believe

Copy link
Contributor

@litianningdatadog litianningdatadog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM

@shreyamalpani shreyamalpani merged commit f766c46 into main Oct 28, 2025
39 checks passed
@shreyamalpani shreyamalpani deleted the shreya.malpani/remove-lock-on-traces branch October 28, 2025 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants