Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add a .no-cache option to event sending #911

Merged
merged 7 commits into from
Jan 28, 2021
Merged

Conversation

mitsuhiko
Copy link
Member

@mitsuhiko mitsuhiko commented Jan 13, 2021

This pull request adds a no-cache flag that forces the internal project cache to refresh before processing the envelope. The flag is encoded into the DSN or public key so that it works with any SDK and third-party (including minidumps).

For external Relays including PoPs, this has the potential to significantly slow down the end-to-end ingestion timing, but makes it possible to debug issues better by forcing a clean cache before sending. This is useful for iterating on grouping or filter settings. Request handling is not impacted by this change, so SDK event submission should not be impacted.

In general, DSN public keys can now specify optional flags. Flags are appended to the public key in dotted syntax:

https://public-key.flag1.flag2@example.com/1234

The flag for cache bypassing is no-cache.

@mitsuhiko mitsuhiko requested a review from a team January 13, 2021 16:36
@jan-auer jan-auer self-requested a review January 13, 2021 17:02
Copy link
Contributor

@RaduW RaduW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this something we want to make available for everybody?
I think we should at least have a configuration option to disable this flag on our production Relays since this has a serious impact on performance and it may be abused (by mistake or deliberately).

@untitaker
Copy link
Member

I'm fine with this but possibly this should not force a refresh if the project is rate limited.

@mitsuhiko
Copy link
Member Author

@RaduW in this form it's not safe to merge but something like this I want to have available on production relays for folks to be able to iterate on grouping configs. Currently the experience testing grouping (and data scrubber) changes is awful because you need to wait what seems to be more than a minute for changes to apply.

* master:
  fix: Build proper 64-bit Linux wheel
  release: 0.8.2
  chore: Switch to manylinux2010 (#917)
  meta(py): Changelog for 0.8.2
  ci: Add an action to release the library (#916)
  ref: Remove Compatibility mode for Session Aggregates (#913)
  release: 21.1.0
  ci: Fix clippy command invocation
  ci: Run clippy via the cargo action
  feat(server): Ingest and normalize sample rates (#910)
@jan-auer
Copy link
Member

Changed this now to apply later in the pipeline. These changes also ensure that we don't force a refresh if the project is rate limited, @untitaker.

  1. The no_cache flag is now only checked when CheckEnvelope::fetched is used. This happens in the EventManager, but not in the fast path.
  2. If no_cache is specified, then any cached state is ignored.
  3. The no_cache flag is passed via upstream requests. This means that even if a request without no_cache is in flight already, we will start a new one regardless.
  4. There is no separation of no_cache in batched upstream requests. In practice, this won't be a problem, so we can look at this at a later stage.

@jan-auer jan-auer changed the title feat: Added a .no-cache option to event sending feat: Add a .no-cache option to event sending Jan 27, 2021
Copy link
Member

@untitaker untitaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have high confidence in this not regressing anything, but it seems ok. Would add a basic integration test.

@jan-auer jan-auer merged commit 0ac80af into master Jan 28, 2021
@jan-auer jan-auer deleted the feature/no-cache branch January 28, 2021 11:40
jan-auer added a commit that referenced this pull request Jan 28, 2021
* master:
  fix(server): Log when recovering from network outages (#918)
  feat: Add a .no-cache option to event sending (#911)
  fix: Build proper 64-bit Linux wheel
  release: 0.8.2
  chore: Switch to manylinux2010 (#917)
  meta(py): Changelog for 0.8.2
  ci: Add an action to release the library (#916)
  ref: Remove Compatibility mode for Session Aggregates (#913)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants