Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FX10946 [v103] Tabs Integration Telemetry #10915

Merged

Conversation

lougeniaC64
Copy link
Contributor

@lougeniaC64 lougeniaC64 commented Jun 3, 2022

This PR is based on #10553 and should only be merged after that PR has been merged. This PR creates telemetry to help track the sync success metrics of the appservices tabs component and the native clients engine.

@lougeniaC64
Copy link
Contributor Author

lougeniaC64 commented Jun 7, 2022

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
  • This will help us to separately gauge the success rates of the native iOS tabs component and the application services tabs component as well as the client engine's sync success rates.
  1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
  • We need to answer these questions to ensure that replacing the native tabs component with the application services tabs component doesn't result in degraded syncing for users. We also want to start tracking the sync success rates of the client engine ahead of its upcoming replacement.
  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?
  • There are no other alternatives.

  1. Can current instrumentation answer these questions?
  • No.

  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

    Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.
Measurement Name Measurement Description Data Collection Category Tracking Bug
rust_tabs_sync.uid The user's hashed Firefox Account ID. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
rust_tabs_sync.incoming Records incoming tabs record counts. applied is the number of incoming records that were successfully stored or updated in the local database. failed_to_apply is the number of records that were ignored due to errors. reconciled is the number of merged records. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
rust_tabs_sync.outgoing Records outgoing tabs record counts. uploaded is the number of records that were successfully sent to the server. failed_to_upload is the number of records that weren't uploaded, and will be retried on the next sync. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
rust_tabs_sync.failure_reason Records why the tabs sync failed. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.uid The user's hashed Firefox Account ID. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.incoming Records incoming clients record counts. applied is the number of incoming records that were successfully stored or updated in the local database. failed_to_apply is the number of records that were ignored due to errors. reconciled is the number of merged records. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.outgoing Records outgoing clients record counts. uploaded is the number of records that were successfully sent to the server. failed_to_upload is the number of records that weren't uploaded, and will be retried on the next sync. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.failure_reason Records why the clients sync failed. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
  1. How long will this data be collected?
  • This collection has expiry '2023-01-01'
  1. What populations will you measure?
  • All channels, countries, and locales. No filters.
  1. If this data collection is default on, what is the opt-out mechanism for users?
  • These collections are Glean. The opt-out can be found in the product's preferences.
  1. Please provide a general description of how you will analyze this data.
  • This data will be analyzed through existing and existing Redash dashboard.
  1. Where do you intend to share the results of your analysis?
  • The results will be Mozilla internal.
  1. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
  • No.

@lougeniaC64 lougeniaC64 changed the title [WIP] Update sync telemetry for tabs Add FX10946 v102 Tabs Integration Telemetry Jun 7, 2022
@lougeniaC64 lougeniaC64 changed the title Add FX10946 v102 Tabs Integration Telemetry Add FX10946 [v102] Tabs Integration Telemetry Jun 7, 2022
@lougeniaC64 lougeniaC64 marked this pull request as ready for review June 7, 2022 23:14
@lougeniaC64
Copy link
Contributor Author

@eliserichards Ready for data review! Let me know if I need to make any changes.

@nishant2718 This PR is includes the telemetry changes for the tabs integration and shouldn't be merged before #10553. Because this PR is based on that tabs integration PR, it will decrease to the changes in this commit once the tabs integration has been merged.

@mergify
Copy link
Contributor

mergify bot commented Jun 8, 2022

This pull request has conflicts when rebasing. Could you fix it @lougeniaC64?

@lougeniaC64 lougeniaC64 force-pushed the integrate-tabs-with-telemetry branch from 0ff4db3 to 9a65081 Compare June 8, 2022 19:11
@eliserichards
Copy link

Just a small suggestion, then I'm happy to data-review+:

  1. How long will this data be collected?
  • This collection has expiry '2023-01-01'

I would recommend changing this to '2023-01-07' or something similar. Things can get lost at the end of the year and I don't want these metrics to accidentally expire while we are all on break 😋

@eliserichards
Copy link

eliserichards commented Jun 14, 2022

Whoops, looks like I don't have commit rights for this branch. Hopefully this will make your life easier :) Just download, unzip, and git apply /path/to/patch1.patch

Updates dates for metrics.yaml: patch1.patch.zip

Add data review URL for metrics.yaml and pings.yaml: patch2.patch.zip

@eliserichards
Copy link

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?


* This will help us to separately gauge the success rates of the native iOS tabs component and the application services tabs component as well as the client engine's sync success rates.


2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?


* We need to answer these questions to ensure that replacing the native tabs component with the application services tabs component doesn't result in degraded syncing for users. We also want to start tracking the sync success rates of the client engine ahead of its upcoming replacement.


3. What alternative methods did you consider to answer these questions? Why were they not sufficient?


* There are no other alternatives.


4. Can current instrumentation answer these questions?


* No.


5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.
   
   Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Name Measurement Description Data Collection Category Tracking Bug
rust_tabs_sync.uid The user's hashed Firefox Account ID. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
rust_tabs_sync.incoming Records incoming tabs record counts. applied is the number of incoming records that were successfully stored or updated in the local database. failed_to_apply is the number of records that were ignored due to errors. reconciled is the number of merged records. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
rust_tabs_sync.outgoing Records outgoing tabs record counts. uploaded is the number of records that were successfully sent to the server. failed_to_upload is the number of records that weren't uploaded, and will be retried on the next sync. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
rust_tabs_sync.failure_reason Records why the tabs sync failed. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.uid The user's hashed Firefox Account ID. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.incoming Records incoming clients record counts. applied is the number of incoming records that were successfully stored or updated in the local database. failed_to_apply is the number of records that were ignored due to errors. reconciled is the number of merged records. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.outgoing Records outgoing clients record counts. uploaded is the number of records that were successfully sent to the server. failed_to_upload is the number of records that weren't uploaded, and will be retried on the next sync. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170
clients_sync.failure_reason Records why the clients sync failed. interaction https://mozilla-hub.atlassian.net/jira/software/c/projects/SYNC/boards/211?modal=detail&selectedIssue=SYNC-3170

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.


* This collection is Glean so is documented [in the Glean Dictionary](https://dictionary.telemetry.mozilla.org/).


7. How long will this data be collected?


* This collection has expiry '2023-01-01'


8. What populations will you measure?


* All channels, countries, and locales. No filters.


9. If this data collection is default on, what is the opt-out mechanism for users?


* These collections are Glean. The opt-out can be found in the product's preferences.


10. Please provide a general description of how you will analyze this data.


* This data will be analyzed through existing and existing Redash dashboard.


11. Where do you intend to share the results of your analysis?


* The results will be Mozilla internal.


12. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?


* No.

Data Review Form (to be filled by Data Stewards)

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

  2. Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.

    • Yes, through the "Send Usage Data" preference in the application settings.
  3. If the request is for permanent data collection, is there someone who will monitor the data over time?

    • No, the data will be collected until January 7, 2023 with the option to remove/renew at that time.
  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

    • Category 2 - interaction data
  5. Is the data collection request for default-on or default-off?

    • Default-on
  6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

    • No
  7. Is the data collection covered by the existing Firefox privacy notice?

    • Yes
  8. Does the data collection use a third-party collection tool? If yes, escalate to legal.

    • No

Result

data-review+

@mergify
Copy link
Contributor

mergify bot commented Jun 15, 2022

This pull request has conflicts when rebasing. Could you fix it @lougeniaC64?

@adudenamedruby adudenamedruby self-requested a review June 16, 2022 17:23
@adudenamedruby adudenamedruby changed the title Add FX10946 [v102] Tabs Integration Telemetry Add FX10946 [v103] Tabs Integration Telemetry Jun 16, 2022
Copy link
Contributor

@adudenamedruby adudenamedruby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heya. Requesting just a few minor changes for standardization.

The nit's are up to you however. Generally we prefer small statements on one line, for future reference. There's a few places where lines are really long. Again, for future reference, if we can line break these and indent them appropriately - I'd have tried to fix them here, but GH's indentation is hot trash so I'd do more harm than good. I'd recommend going to Xcode preferences -> Text Editing and enabling "Page column guide at:" The recommended for the codebase is 120, but the closer you are to 90/100, the better for readability. :)

Providers/Profile.swift Outdated Show resolved Hide resolved
Providers/Profile.swift Outdated Show resolved Hide resolved
Providers/Profile.swift Outdated Show resolved Hide resolved
Providers/Profile.swift Outdated Show resolved Hide resolved
Storage/Rust/RustRemoteTabs.swift Outdated Show resolved Hide resolved
Storage/Rust/RustRemoteTabs.swift Outdated Show resolved Hide resolved
Providers/Profile.swift Outdated Show resolved Hide resolved
@lougeniaC64 lougeniaC64 force-pushed the integrate-tabs-with-telemetry branch 2 times, most recently from f027092 to f2151be Compare June 24, 2022 15:57
@adudenamedruby adudenamedruby merged commit 804720d into mozilla-mobile:main Jun 27, 2022
@lougeniaC64 lougeniaC64 deleted the integrate-tabs-with-telemetry branch June 28, 2022 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants