New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase event_property threshold #2478
Conversation
@@ -108,7 +108,7 @@ per_event_property AS ( | |||
USING | |||
(category, event, event_property) | |||
WHERE | |||
event_property_value_index <= 1000 | |||
event_property_value_index <= 5000 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also open to setting a different threshold
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am slightly nervous about unintended consequences here of raising the limit by 5x, but I don't see why it would a problem apriori. It does seem reasonable to set a higher limit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not aware of any issues with increasing the threshold. However, I believe the right approach is to modify the templating here and regenerate the queries:
bigquery-etl/bigquery_etl/events_daily/query_templates/event_types_history_v1/templating.yaml
Lines 12 to 17 in 821f9d4
telemetry_derived: | |
name: Firefox | |
dataset: telemetry | |
source_table: telemetry_derived.deanonymized_events | |
start_date: 2020-01-01 | |
max_property_values: 1000 |
65dec29
to
3cba89e
Compare
Right, right |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
r+ with the yaml changes to non-Firefox reverted
@@ -7,28 +7,28 @@ fenix_derived: | |||
start_date: 2020-01-01 | |||
skipped_properties: | |||
- time_ms | |||
max_property_values: 1000 | |||
max_property_values: 5000 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep these values the same for everything other than Firefox desktop.
@@ -108,7 +108,7 @@ per_event_property AS ( | |||
USING | |||
(category, event, event_property) | |||
WHERE | |||
event_property_value_index <= 1000 | |||
event_property_value_index <= 5000 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am slightly nervous about unintended consequences here of raising the limit by 5x, but I don't see why it would a problem apriori. It does seem reasonable to set a higher limit.
3cba89e
to
02ceeba
Compare
I'm not sure if increasing this threshold has any implications?