ref(server): Transform EnvelopeManager into a sequential pipeline

[jan-auer]

Removes the long-running handler future in EnvelopeManager, and instead pass envelopes through explicit queues the ingestion stages. Actual error handling is now localized in every step of the pipeline.

This helps to improve Relay's response time when buffering requests.

Endpoint

The store endpoint handler remains unchanged. It calls the same messages and eventually invokes QueueEnvelope. That message splits up the Envelope and immediately sends it to the ProjectCache. In a follow-up, this can be moved directly into the endpoint, the EnvelopeManager is no longer needed for this.

Project Cache Queues

The biggest change is within the ProjectCache and projects. Instead of responding to CheckEnvelope, the projects now take ownership of envelopes and put them into queues if they are not up-to-date. The precise sequence is:

1. Move envelopes into the project via the new ValidateEnvelope message. If the project state is valid, this immediately validates and forwards the envelopes to the next stage (see 3). Otherwise, put them on a local queue on the project.
2. Trigger a fire-and-forget update of the project state in the background. The update will eventually invoke update_state on the project, which flushes the queues of pending envelopes. This is guaranteed to run eventually, if project fetching fails or times out.
3. If the envelope needs dynamic sampling, send it back into the project cache to retrieve the second project state via AddSamplingState. Otherwise, directly forward to processing (step 5).
4. Identical queueing behavior as in step 2. To match the prior behavior this still forwards envelopes if the sampling state cannot be fetched. Such envelopes will all be ingested.
5. Send the envelope via ProcessEnvelope to the envelope processor, along with the project states.

Remaining Queueing

Control flow of the remaining pipeline steps did not change:

• The project cache submits envelopes as ProcessEnvelope to the EnvelopeProcessor.
• Internal errors in the processing pool are handled locally and logged immediately.
• The envelope is finally passed to the EnvelopeManager with a new message SubmitEnvelope. This message runs the final dispatch between HTTP upstream (SendEnvelope), Kafka (StoreEnvelope), or capturing (Capture).
• Errors during submission are handled in a guarded future in the EnvelopeManager. This future is expected to last significantly shorter than project fetching.

Drop Guards

There are now 3 places where futures can be dropped, which are individually monitored:

• Projects gained a Drop implementation that logs how many events have been dropped. In normal operation, only empty projects are evicted from the cache.
• The EnvelopeProcessor does not have any drop guarding. This is complicated to achieve, and the processor's queues are usually near empty. We can introduce a drop guard when migrating the processor to tokio.
• The SubmitEnvelope still features the former drop guarded future.

Resources

Prepared by:

• ref(server): Remove overall envelope handling timeout #1398
• ref(server): Localize outcome emission in EnvelopeContext #1406
• ref(server): Use an atomic semaphore to track buffer usage #1408
• ref(server): Add a message to capture envelopes in capture mode #1409

Follow-ups:

• Remove unneeded response structs or fields for CheckEnvelope and ProcessEnvelope.
• Remove the QueueEnvelope message and instead invoke this as function directly in the endpoint handler.
• Tie EnvelopeContext into ProcessingError, clean up metrics emission, and improve control flow.
• Do not overwrite a project state if it's not expired and incoming is an error in update_state.
• Update the graph and documentation in relay-server.

#skip-changelog

[jan-auer]

Guide for Reviewers: Start in endpoints/common.rs and then follow the messages QueueEnvelope -> ValidateEnvelope -> AddSamplingState -> ProcessEnvelope -> SubmitEnvelope.

[jjbayer]

Great stuff!

[jjbayer]

What would be super-nice is a diagram with Actors and Messages and how they changed. I would even volunteer to draw it :)

[jan-auer]

👍 much appreciated. One of the follow-ups mentioned in the description includes updating the documentation, which contains a graph that would need updating: https://getsentry.github.io/relay/relay_server/index.html#sequence-diagram-of-components-inside-relay