Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jetstream configs should support easy downsampling #1867

Closed
danielkberry opened this issue Jul 31, 2023 · 3 comments
Closed

Jetstream configs should support easy downsampling #1867

danielkberry opened this issue Jul 31, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@danielkberry
Copy link
Contributor

danielkberry commented Jul 31, 2023

Currently, if a job fails due to memory issues, a DS must manually override the enrollments query to do downsampling. This is complex, time-consuming, and error-prone. It would be helpful if support could be built for easy downsampling. Something like:

[experiments]
sample_id_range = [0,10]

to explicitly use 0 <= sample_id <= 10
or

[experiments]
sample_percentage = 75

to implicitly use sample_id <= 75

┆Issue is synchronized with this Jira Task

@danielkberry danielkberry added the enhancement New feature or request label Jul 31, 2023
@mikewilli
Copy link
Contributor

See related mozilla/mozanalysis#187

@mikewilli
Copy link
Contributor

@mikewilli
Copy link
Contributor

For posterity: the solution for this issue was to add the config option sample_size, which is an integer 1-100 representing the sample rate (e.g., sample_size = 20 would use a 20% sample).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants