feat(metrics): Actually batch by partition, set request header [INGEST-1562] #1440
Conversation
| fut.spawn(context); | ||
| } | ||
| }); | ||
| let num_partitions = self.config.flush_partitions.unwrap_or(1); |
There was a problem hiding this comment.
This means that we always set the X-Sentry-Relay-Shard header, even though no partitioning was requested by config. We could easily change this behavior but I thought this would keep the code nice and simple.
| @@ -554,6 +554,10 @@ impl UpstreamRelay { | |||
| } | |||
| } | |||
|
|
|||
| if let Some(partition_key) = request.request.partition_key() { | |||
| builder.header("X-Sentry-Relay-Shard", partition_key); | |||
There was a problem hiding this comment.
TODO: Decide on the actual name for the header.
There was a problem hiding this comment.
Going with X-Sentry-Relay-Shard. The proposal mentions X-Relay-Shard, but other headers we set also start with X-Sentry-Relay-.
| @@ -66,6 +66,7 @@ pub struct SendEnvelope { | |||
| pub http_encoding: HttpEncoding, | |||
| pub response_sender: Option<oneshot::Sender<Result<(), SendEnvelopeError>>>, | |||
| pub project_key: ProjectKey, | |||
| partition_key: Option<String>, | |||
There was a problem hiding this comment.
Having a String here means that other envelopes could theoretically use the same header with a different partitioning strategy.
relay-server/src/actors/envelopes.rs
Outdated
| @@ -353,6 +362,7 @@ impl fmt::Debug for SendMetrics { | |||
| .field("buckets", &self.buckets) | |||
There was a problem hiding this comment.
@jan-auer any idea why we don't use derive(Debug) here?
There was a problem hiding this comment.
I think we couldn't because SendMetrics used to hold an Addr<Project>. Now that Project is not an actor anymore, and this struct only holds a ProjectKey, I think we can use derive(Debug).
Convert the dry run implemented in #1425 into an actual batching mechanism that splits metrics buckets into logical partitions.
The
partition_keyhas to be passed throughProjectCache,EnvelopeManagerandUpstreamRelayto be set as a header on the outgoing envelope request.