Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DENG-774 Change Control for active_users_aggregates #3687

Merged
merged 22 commits into from
Apr 14, 2023

Conversation

lucia-vargas-a
Copy link
Contributor

@lucia-vargas-a lucia-vargas-a commented Mar 28, 2023

Implements DENG-774

For modifications to schemas in restricted namespaces (see CODEOWNERS):

Comment on lines 33 to 35
def is_github_identity(s):
"""Check if the given string matches the format of a Github identity."""
return re.match(r"[mozilla]+\/[a-zA-Z0-9]+", s)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Airflow task we need to filter out the Github teams otherwise DAGs cannot be generated

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer we include the @ prefix that github uses for github identities. I assume we already have some normalization logic for labels that propagate to GCP if that's the reason we've omitted it.

Copy link
Contributor

@bochocki bochocki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM! I don't think I fully understand how all the pieces fit together with the bigger picture (e.g. I wouldn't have caught @scholtzan's comment about the Airflow task), but from what I'm able to follow it looks great. I just had a few minor refactoring suggestions.

bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
tests/cli/test_cli_metadata.py Outdated Show resolved Hide resolved
tests/cli/test_cli_metadata.py Show resolved Hide resolved
@dataops-ci-bot
Copy link

Integration report for "Merge branch 'main' into DENG-774_map_labels_and_owners"

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:45:03.046239135 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:35:45.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:45:04.222245974 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:35:46.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:45:04.546247858 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:35:47.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:45:05.078250952 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:35:47.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:45:05.158251417 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:35:47.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:45:05.410252883 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 18:35:47.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- bigquery-etl-main/dags/bqetl_analytics_aggregations.py	2023-03-29 18:45:54.042525979 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2023-03-29 18:38:01.000000000 +0000
@@ -118,6 +118,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -134,6 +135,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter="submission_date",
@@ -149,6 +151,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -165,6 +168,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -181,6 +185,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -197,6 +202,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,

@dataops-ci-bot
Copy link

Integration report for "DENG-774 Update test property."

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:12:47.535260344 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:03:18.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:12:48.435258229 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:03:18.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:12:48.711257580 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:03:19.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:12:49.027256863 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:03:19.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:12:49.099256702 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:03:19.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:12:49.383256067 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-29 19:03:19.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/monitoring/average_ping_sizes: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/monitoring/shredder_rows_deleted: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/monitoring/stable_and_derived_table_sizes: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/monitoring/structured_distinct_docids: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- bigquery-etl-main/dags/bqetl_analytics_aggregations.py	2023-03-29 19:13:38.915145023 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2023-03-29 19:05:05.000000000 +0000
@@ -118,6 +118,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -134,6 +135,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter="submission_date",
@@ -149,6 +151,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -165,6 +168,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -181,6 +185,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -197,6 +202,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,

Copy link
Member

@whd whd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had some trouble reading through the validation section. I think it would be simpler to reason about if we required all github identities listed as owners in metadata to be listed as CODEOWNERS for tables subject to this change control.

CODEOWNERS Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
Comment on lines 33 to 35
def is_github_identity(s):
"""Check if the given string matches the format of a Github identity."""
return re.match(r"[mozilla]+\/[a-zA-Z0-9]+", s)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer we include the @ prefix that github uses for github identities. I assume we already have some normalization logic for labels that propagate to GCP if that's the reason we've omitted it.

bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
bigquery_etl/metadata/parse_metadata.py Outdated Show resolved Hide resolved
@dataops-ci-bot
Copy link

Integration report for "DENG-774 Syntax improvements and use of readlines() to read CODEOWNERS."

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:48:34.205784083 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:39:53.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:48:35.261784778 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:39:54.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:48:35.609785007 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:39:54.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:48:35.977785249 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:39:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:48:36.065785307 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:39:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:48:36.405785531 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-03-31 15:39:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- bigquery-etl-main/dags/bqetl_analytics_aggregations.py	2023-03-31 15:49:29.145819296 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2023-03-31 15:41:56.000000000 +0000
@@ -118,6 +118,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -134,6 +135,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter="submission_date",
@@ -149,6 +151,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -165,6 +168,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -181,6 +185,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -197,6 +202,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,

dags/bqetl_analytics_aggregations.py Outdated Show resolved Hide resolved
@dataops-ci-bot
Copy link

Integration report for "Merge branch 'main' into DENG-774_map_labels_and_owners"

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:53:01.952455480 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:44:13.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:53:03.356458599 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:44:14.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:53:03.860459718 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:44:14.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:53:04.444461017 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:44:14.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:53:04.560461274 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:44:14.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:53:05.032462323 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-03 09:44:15.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- bigquery-etl-main/dags/bqetl_analytics_aggregations.py	2023-04-03 09:54:09.860606610 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2023-04-03 09:46:09.000000000 +0000
@@ -118,6 +118,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -134,6 +135,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter="submission_date",
@@ -149,6 +151,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -165,6 +168,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -181,6 +185,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -197,6 +202,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,

CODEOWNERS Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
@dataops-ci-bot
Copy link

Integration report for "Merge branch 'main' into DENG-774_map_labels_and_owners"

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:32:01.868931827 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:22:41.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:32:02.900944544 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:22:42.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:32:03.232948636 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:22:42.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:32:03.600953171 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:22:43.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:32:03.708954502 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:22:43.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:32:04.116959530 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 10:22:43.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 10:26:01.696502114 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 10:22:20.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM devices WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM devices WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 10:26:01.696502114 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 10:22:20.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 10:26:01.696502114 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 10:22:20.000000000 +0000
@@ -11,7 +11,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM users WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM users WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _current
 FULL JOIN
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 10:26:01.768502999 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 10:22:12.000000000 +0000
@@ -6,19 +6,47 @@
   ssl_loads_v1 AS ssl_loads,
   http_pageload_is_ssl_ratio_v1 AS reporting_ratio
 FROM
-  {{
-  metrics.calculate(
-    metrics=["non_ssl_loads_v1", "ssl_loads_v1", "http_pageload_is_ssl_ratio_v1"],
-    platform="firefox_desktop",
-    group_by={"os": "environment.system.os.name", "country": "metadata.geo.country"},
-    where="""
+  (
+    WITH main AS (
+      SELECT
+        submission_date AS submission_date,
+        environment.system.os.name AS os,
+        metadata.geo.country AS country,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 0)
+        ) AS non_ssl_loads_v1,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 1)
+        ) AS ssl_loads_v1,
+        COUNT(payload.histograms.http_pageload_is_ssl) / COUNT(*) AS http_pageload_is_ssl_ratio_v1,
+      FROM
+        (
+          SELECT
+            *,
+            DATE(submission_timestamp) AS submission_date,
+            environment.experiments
+          FROM
+            `moz-fx-data-shared-prod.telemetry_stable.main_v4`
+        )
+      WHERE
       sample_id = 42
       AND normalized_channel = 'release'
       AND environment.system.os.name IN ('Windows_NT', 'Darwin', 'Linux')
       AND application.name = 'Firefox'
       AND DATE(submission_timestamp) > DATE '2016-11-01'
       AND (DATE(submission_timestamp) = @submission_date OR @submission_date IS NULL)
-    """,
-    group_by_client_id=False
+      GROUP BY
+        os,
+        country,
+        submission_date
+    )
+    SELECT
+      main.submission_date,
+      main.os AS os,
+      main.country AS country,
+      non_ssl_loads_v1,
+      ssl_loads_v1,
+      http_pageload_is_ssl_ratio_v1,
+    FROM
+      main
   )
-}}
diff -bur --no-dereference bigquery-etl-main/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- bigquery-etl-main/dags/bqetl_analytics_aggregations.py	2023-04-13 10:32:57.677619877 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2023-04-13 10:24:51.000000000 +0000
@@ -118,6 +118,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -134,6 +135,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter="submission_date",
@@ -149,6 +151,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -165,6 +168,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -181,6 +185,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,
@@ -197,6 +202,7 @@
         email=[
             "gkaberere@mozilla.com",
             "lvargas@mozilla.com",
+            "mozilla/kpi_table_reviewers",
             "telemetry-alerts@mozilla.com",
         ],
         date_partition_parameter=None,

@dataops-ci-bot
Copy link

Integration report for "DENG-774 Update DAGs and notification."

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:44:43.467196883 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:35:07.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:44:44.443195407 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:35:08.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:44:44.723194984 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:35:08.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:44:45.043194500 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:35:09.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:44:45.119194385 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:35:09.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:44:45.447193889 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 12:35:09.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 12:38:17.563811079 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 12:34:39.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM devices WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM devices WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 12:38:17.563811079 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 12:34:39.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 12:38:17.563811079 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 12:34:39.000000000 +0000
@@ -11,7 +11,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM users WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM users WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _current
 FULL JOIN
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 12:38:17.615810991 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 12:34:57.000000000 +0000
@@ -6,19 +6,47 @@
   ssl_loads_v1 AS ssl_loads,
   http_pageload_is_ssl_ratio_v1 AS reporting_ratio
 FROM
-  {{
-  metrics.calculate(
-    metrics=["non_ssl_loads_v1", "ssl_loads_v1", "http_pageload_is_ssl_ratio_v1"],
-    platform="firefox_desktop",
-    group_by={"os": "environment.system.os.name", "country": "metadata.geo.country"},
-    where="""
+  (
+    WITH main AS (
+      SELECT
+        submission_date AS submission_date,
+        environment.system.os.name AS os,
+        metadata.geo.country AS country,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 0)
+        ) AS non_ssl_loads_v1,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 1)
+        ) AS ssl_loads_v1,
+        COUNT(payload.histograms.http_pageload_is_ssl) / COUNT(*) AS http_pageload_is_ssl_ratio_v1,
+      FROM
+        (
+          SELECT
+            *,
+            DATE(submission_timestamp) AS submission_date,
+            environment.experiments
+          FROM
+            `moz-fx-data-shared-prod.telemetry_stable.main_v4`
+        )
+      WHERE
       sample_id = 42
       AND normalized_channel = 'release'
       AND environment.system.os.name IN ('Windows_NT', 'Darwin', 'Linux')
       AND application.name = 'Firefox'
       AND DATE(submission_timestamp) > DATE '2016-11-01'
       AND (DATE(submission_timestamp) = @submission_date OR @submission_date IS NULL)
-    """,
-    group_by_client_id=False
+      GROUP BY
+        os,
+        country,
+        submission_date
+    )
+    SELECT
+      main.submission_date,
+      main.os AS os,
+      main.country AS country,
+      non_ssl_loads_v1,
+      ssl_loads_v1,
+      http_pageload_is_ssl_ratio_v1,
+    FROM
+      main
   )
-}}

@lucia-vargas-a lucia-vargas-a requested a review from whd April 13, 2023 14:55
@dataops-ci-bot
Copy link

Integration report for "DENG-774 Require at least on owner instead of exactly one."

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:13:40.023915772 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:01:41.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:13:41.051913346 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:01:42.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:13:41.379912572 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:01:42.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:13:41.787911608 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:01:43.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:13:41.875911401 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:01:43.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:13:42.311910372 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 15:01:43.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 15:05:13.717108526 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 15:01:28.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM devices WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM devices WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 15:05:13.717108526 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 15:01:28.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 15:05:13.717108526 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 15:01:28.000000000 +0000
@@ -11,7 +11,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM users WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM users WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _current
 FULL JOIN
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 15:05:13.777108381 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 15:01:23.000000000 +0000
@@ -6,19 +6,47 @@
   ssl_loads_v1 AS ssl_loads,
   http_pageload_is_ssl_ratio_v1 AS reporting_ratio
 FROM
-  {{
-  metrics.calculate(
-    metrics=["non_ssl_loads_v1", "ssl_loads_v1", "http_pageload_is_ssl_ratio_v1"],
-    platform="firefox_desktop",
-    group_by={"os": "environment.system.os.name", "country": "metadata.geo.country"},
-    where="""
+  (
+    WITH main AS (
+      SELECT
+        submission_date AS submission_date,
+        environment.system.os.name AS os,
+        metadata.geo.country AS country,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 0)
+        ) AS non_ssl_loads_v1,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 1)
+        ) AS ssl_loads_v1,
+        COUNT(payload.histograms.http_pageload_is_ssl) / COUNT(*) AS http_pageload_is_ssl_ratio_v1,
+      FROM
+        (
+          SELECT
+            *,
+            DATE(submission_timestamp) AS submission_date,
+            environment.experiments
+          FROM
+            `moz-fx-data-shared-prod.telemetry_stable.main_v4`
+        )
+      WHERE
       sample_id = 42
       AND normalized_channel = 'release'
       AND environment.system.os.name IN ('Windows_NT', 'Darwin', 'Linux')
       AND application.name = 'Firefox'
       AND DATE(submission_timestamp) > DATE '2016-11-01'
       AND (DATE(submission_timestamp) = @submission_date OR @submission_date IS NULL)
-    """,
-    group_by_client_id=False
+      GROUP BY
+        os,
+        country,
+        submission_date
+    )
+    SELECT
+      main.submission_date,
+      main.os AS os,
+      main.country AS country,
+      non_ssl_loads_v1,
+      ssl_loads_v1,
+      http_pageload_is_ssl_ratio_v1,
+    FROM
+      main
   )
-}}

@lucia-vargas-a lucia-vargas-a requested a review from whd April 13, 2023 18:22
@dataops-ci-bot
Copy link

Integration report for "DENG-774 Add tests scenarios and consider lines in CODEOWNERS starting with #."

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:27:49.903945087 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:17:53.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:27:51.139959061 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:17:54.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:27:51.515963312 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:17:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:27:51.903967699 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:17:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:27:51.991968694 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:17:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:27:52.391973217 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:17:55.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 18:21:15.227098425 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 18:17:16.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM devices WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM devices WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 18:21:15.227098425 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 18:17:16.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 18:21:15.227098425 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 18:17:16.000000000 +0000
@@ -11,7 +11,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM users WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM users WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _current
 FULL JOIN
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 18:21:15.287099384 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 18:17:37.000000000 +0000
@@ -6,19 +6,47 @@
   ssl_loads_v1 AS ssl_loads,
   http_pageload_is_ssl_ratio_v1 AS reporting_ratio
 FROM
-  {{
-  metrics.calculate(
-    metrics=["non_ssl_loads_v1", "ssl_loads_v1", "http_pageload_is_ssl_ratio_v1"],
-    platform="firefox_desktop",
-    group_by={"os": "environment.system.os.name", "country": "metadata.geo.country"},
-    where="""
+  (
+    WITH main AS (
+      SELECT
+        submission_date AS submission_date,
+        environment.system.os.name AS os,
+        metadata.geo.country AS country,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 0)
+        ) AS non_ssl_loads_v1,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 1)
+        ) AS ssl_loads_v1,
+        COUNT(payload.histograms.http_pageload_is_ssl) / COUNT(*) AS http_pageload_is_ssl_ratio_v1,
+      FROM
+        (
+          SELECT
+            *,
+            DATE(submission_timestamp) AS submission_date,
+            environment.experiments
+          FROM
+            `moz-fx-data-shared-prod.telemetry_stable.main_v4`
+        )
+      WHERE
       sample_id = 42
       AND normalized_channel = 'release'
       AND environment.system.os.name IN ('Windows_NT', 'Darwin', 'Linux')
       AND application.name = 'Firefox'
       AND DATE(submission_timestamp) > DATE '2016-11-01'
       AND (DATE(submission_timestamp) = @submission_date OR @submission_date IS NULL)
-    """,
-    group_by_client_id=False
+      GROUP BY
+        os,
+        country,
+        submission_date
+    )
+    SELECT
+      main.submission_date,
+      main.os AS os,
+      main.country AS country,
+      non_ssl_loads_v1,
+      ssl_loads_v1,
+      http_pageload_is_ssl_ratio_v1,
+    FROM
+      main
   )
-}}

@dataops-ci-bot
Copy link

Integration report for "DENG-774 Syntax update."

sql.diff

Click to expand!
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:38:59.169854264 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:28:47.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: fenix_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:39:00.017857668 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:28:48.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_desktop_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:39:00.285858744 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:28:48.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: firefox_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:39:00.585859948 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:28:48.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_android_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:39:00.657860237 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:28:49.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: focus_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:39:00.965861473 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/active_users_aggregates_v1/metadata.yaml	2023-04-13 18:28:49.000000000 +0000
@@ -9,12 +9,17 @@
   For Firefox Desktop, qualified daily active users QDAU
   are calculated. For mobile apps QDAU = DAU.
 
+  The table is labeled as "change_controlled", which implies
+  that changes require the approval of at least one owner.
+
   Proposal:
   https://docs.google.com/document/d/1qvWO49Lr_Z_WErh3I3058A3B1YuiuURx19K3aTdmejM/edit?usp=sharing
 owners:
   - lvargas@mozilla.com
+  - mozilla/kpi_table_reviewers
 labels:
   incremental: true
+  change_controlled: true
 scheduling:
   dag_name: bqetl_analytics_aggregations
   task_name: klar_ios_active_users_aggregates
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 18:32:11.452222029 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/devices_v1/query.sql	2023-04-13 18:28:28.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM devices WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM devices WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 18:32:11.452222029 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/subscriptions_v1/query.sql	2023-04-13 18:28:28.000000000 +0000
@@ -6,7 +6,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM subscriptions WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _update
 FULL JOIN
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 18:32:11.452222029 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_external/users_v1/query.sql	2023-04-13 18:28:28.000000000 +0000
@@ -11,7 +11,7 @@
     -- The external_database_query argument in EXTERNAL_QUERY must be a literal string or query
     -- parameter, and cannot be generated at runtime using function calls like CONCAT or FORMAT,
     -- so the entire value must be provided as a STRING query parameter to handle specific dates:
-    -- {% raw %} "SELECT * FROM users WHERE DATE(updated_at) = DATE '{{ds}}'" {% endraw %}
+    --  "SELECT * FROM users WHERE DATE(updated_at) = DATE ''"
     @external_database_query
   ) AS _current
 FULL JOIN
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/regrets_reporter_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/xfocsp_error_report: schema.yaml
diff -bur --no-dereference bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql
--- bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 18:32:11.536222372 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/ssl_ratios_v1/query.sql	2023-04-13 18:28:21.000000000 +0000
@@ -6,19 +6,47 @@
   ssl_loads_v1 AS ssl_loads,
   http_pageload_is_ssl_ratio_v1 AS reporting_ratio
 FROM
-  {{
-  metrics.calculate(
-    metrics=["non_ssl_loads_v1", "ssl_loads_v1", "http_pageload_is_ssl_ratio_v1"],
-    platform="firefox_desktop",
-    group_by={"os": "environment.system.os.name", "country": "metadata.geo.country"},
-    where="""
+  (
+    WITH main AS (
+      SELECT
+        submission_date AS submission_date,
+        environment.system.os.name AS os,
+        metadata.geo.country AS country,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 0)
+        ) AS non_ssl_loads_v1,
+        SUM(
+          mozfun.map.get_key(mozfun.hist.extract(payload.histograms.http_pageload_is_ssl).values, 1)
+        ) AS ssl_loads_v1,
+        COUNT(payload.histograms.http_pageload_is_ssl) / COUNT(*) AS http_pageload_is_ssl_ratio_v1,
+      FROM
+        (
+          SELECT
+            *,
+            DATE(submission_timestamp) AS submission_date,
+            environment.experiments
+          FROM
+            `moz-fx-data-shared-prod.telemetry_stable.main_v4`
+        )
+      WHERE
       sample_id = 42
       AND normalized_channel = 'release'
       AND environment.system.os.name IN ('Windows_NT', 'Darwin', 'Linux')
       AND application.name = 'Firefox'
       AND DATE(submission_timestamp) > DATE '2016-11-01'
       AND (DATE(submission_timestamp) = @submission_date OR @submission_date IS NULL)
-    """,
-    group_by_client_id=False
+      GROUP BY
+        os,
+        country,
+        submission_date
+    )
+    SELECT
+      main.submission_date,
+      main.os AS os,
+      main.country AS country,
+      non_ssl_loads_v1,
+      ssl_loads_v1,
+      http_pageload_is_ssl_ratio_v1,
+    FROM
+      main
   )
-}}

Copy link
Member

@whd whd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r+wc

Thank you for adding the tests. I think it's important to decide whether to require only owners in CODEOWNERS, but I don't think that needs another round of review if a test is added.

After this is merged https://github.com/mozilla/bigquery-etl/settings/branch_protection_rules/4491948 should be updated to require the validate-metadata check since it isn't required presently.

tests/cli/test_cli_metadata.py Outdated Show resolved Hide resolved
bigquery_etl/metadata/validate_metadata.py Outdated Show resolved Hide resolved
@lucia-vargas-a lucia-vargas-a enabled auto-merge (squash) April 14, 2023 13:42
@lucia-vargas-a lucia-vargas-a merged commit c346537 into main Apr 14, 2023
@lucia-vargas-a lucia-vargas-a deleted the DENG-774_map_labels_and_owners branch April 14, 2023 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants