Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

Telemetry for #18855, #18851 and #18857 #18982

Merged
merged 1 commit into from
Apr 15, 2021

Conversation

Amejia481
Copy link
Contributor

Pull Request checklist

  • Tests: This PR includes thorough tests or an explanation of why it does not
  • Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
  • Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

To download an APK when reviewing a PR:

  1. click on Show All Checks,
  2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
  3. click on the "Fenix - assemble" task, then click "Run Artifacts".
  4. the APK links should be on the left side of the screen, named for each CPU architecture

@Amejia481 Amejia481 requested review from a team as code owners April 13, 2021 05:36
app/metrics.yaml Outdated Show resolved Hide resolved
@Amejia481
Copy link
Contributor Author

Amejia481 commented Apr 13, 2021

Request for data collection review form #18851

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
    This will help us understand user interaction with the "set as default browser" menu item on the toolbar in the default browser experiment.

  2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:
    Determine how many people interact directly with our message.

  3. What alternative methods did you consider to answer these questions? Why were they not sufficient?
    No other alternative.

  4. Can current instrumentation answer these questions?
    No.

  5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description Data Collection Category Tracking Bug #
User clicks on set default browser category 2 - interaction data #18851
  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
    https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md

  2. How long will this data be collected? Choose one of the following:
    This is scoped to a time-limited experiment/project until date 10-01-2021 or earlier, when the experiment is removed.

  3. What populations will you measure?
    The audience is controlled by a Nimbus experiment which will probably limit the reach.

  4. If this data collection is default on, what is the opt-out mechanism for users?
    Default Glean SDK opt-out mechanism.

  5. Please provide a general description of how you will analyze this data.
    Glean / Amplitude

  6. Where do you intend to share the results of your analysis?
    Only on Glean, Amplitude and with mobile teams.

  7. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection? If so:
    
No third-party tool will use this data.

@Amejia481
Copy link
Contributor Author

@tdsmith 👋🏽 when you have some time would you mind taking a look to this PR, thanks in advance!

@Amejia481 Amejia481 added the needs:review PRs that need to be reviewed label Apr 13, 2021
app/metrics.yaml Outdated Show resolved Hide resolved
@Amejia481
Copy link
Contributor Author

Amejia481 commented Apr 13, 2021

Request for data collection review form #18855

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
    This will help us understand user interaction with the "set as default browser" menu item on the toolbar in the default browser experiment.

  2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:
    How many users will see, saw, or could have seen the message we will place on the toolbar menu?

  3. What alternative methods did you consider to answer these questions? Why were they not sufficient?
    No other alternative.

  4. Can current instrumentation answer these questions?
    No.

  5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description Data Collection Category Tracking Bug #
The set default browser menu item is shown in the toolbar category 2 - interaction data #18855
  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
    https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md

  2. How long will this data be collected? Choose one of the following:
    This is scoped to a time-limited experiment/project until date 10-01-2021 or earlier, when the experiment is removed.

  3. What populations will you measure?
    The audience is controlled by a Nimbus experiment which will probably limit the reach.

  4. If this data collection is default on, what is the opt-out mechanism for users?
    Default Glean SDK opt-out mechanism.

  5. Please provide a general description of how you will analyze this data.
    Glean / Amplitude

  6. Where do you intend to share the results of your analysis?
    Only on Glean, Amplitude and with mobile teams.

  7. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection? If so:
    
No third-party tool will use this data.

Copy link
Contributor

@tdsmith tdsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data-review+; note that these are interaction data, not technical data, although that doesn't affect the conclusion and the Glean metadata looks correct.

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, in the probe definition files and the Glean probe dictionary.

  1. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, the Firefox telemetry opt-out.

  1. If the request is for permanent data collection, is there someone who will monitor the data over time?

n/a

  1. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Cat 2, interaction data.

  1. Is the data collection request for default-on or default-off?

Default-on.

  1. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

Nope.

  1. Is the data collection covered by the existing Firefox privacy notice?

Yes.

  1. Does there need to be a check-in in the future to determine whether to renew the data?

The team should renew as necessary.

  1. Does the data collection use a third-party collection tool?

No.

app/metrics.yaml Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Apr 14, 2021

Codecov Report

Merging #18982 (2d64f1e) into master (80d2bec) will increase coverage by 0.17%.
The diff coverage is 13.63%.

❗ Current head 2d64f1e differs from pull request most recent head c076fa4. Consider uploading reports for the commit c076fa4 to get more accurate results
Impacted file tree graph

@@             Coverage Diff              @@
##             master   #18982      +/-   ##
============================================
+ Coverage     34.20%   34.38%   +0.17%     
+ Complexity     1578     1543      -35     
============================================
  Files           535      526       -9     
  Lines         21660    21469     -191     
  Branches       3229     3210      -19     
============================================
- Hits           7409     7382      -27     
+ Misses        13348    13191     -157     
+ Partials        903      896       -7     
Impacted Files Coverage Δ Complexity Δ
...pp/src/main/java/org/mozilla/fenix/HomeActivity.kt 23.93% <0.00%> (-0.59%) 28.00 <0.00> (ø)
...java/org/mozilla/fenix/components/metrics/Event.kt 38.82% <0.00%> (-0.39%) 0.00 <0.00> (ø)
...la/fenix/components/metrics/GleanMetricsService.kt 21.34% <0.00%> (-0.07%) 5.00 <0.00> (ø)
...components/toolbar/BrowserToolbarMenuController.kt 57.71% <0.00%> (-0.08%) 0.00 <0.00> (ø)
...lla/fenix/components/toolbar/DefaultToolbarMenu.kt 43.92% <0.00%> (+43.92%) 20.00 <0.00> (+20.00)
.../src/main/java/org/mozilla/fenix/utils/Settings.kt 72.27% <35.29%> (-0.40%) 36.00 <0.00> (ø)
...illa/fenix/tabstray/browser/BaseBrowserTrayList.kt 15.38% <0.00%> (-59.62%) 1.00% <0.00%> (-3.00%)
.../java/org/mozilla/fenix/experiments/NimbusSetup.kt 0.00% <0.00%> (-52.39%) 0.00% <0.00%> (ø%)
...a/org/mozilla/fenix/tabstray/TabsTrayViewHolder.kt 0.00% <0.00%> (-50.69%) 0.00% <0.00%> (-11.00%)
app/src/main/java/org/mozilla/fenix/ext/Nimbus.kt 0.00% <0.00%> (-44.45%) 0.00% <0.00%> (ø%)
... and 73 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80d2bec...c076fa4. Read the comment docs.

@Amejia481 Amejia481 force-pushed the for-18851 branch 5 times, most recently from cb143b8 to b1b76a9 Compare April 14, 2021 02:06
@Amejia481 Amejia481 changed the title For #18851 [Telemetry] Send an event when users click on the "set as default browser" entry in the toolbar menu Telemetry for #18855, #18851 and #18857 Apr 14, 2021
@Amejia481
Copy link
Contributor Author

@tdsmith the PR is updated with the recommended changes :)

@mergify
Copy link
Contributor

mergify bot commented Apr 14, 2021

This pull request has conflicts when rebasing. Could you fix it @Amejia481? 🙏

@tdsmith
Copy link
Contributor

tdsmith commented Apr 14, 2021

Thanks, Arturo!

…mobile#18857

For mozilla-mobile#18857 [Telemetry] Send a Glean event when users change their default browser

For mozilla-mobile#18855 [Telemetry] Send an event when users open the toolbar menu

For mozilla-mobile#18851 [Telemetry] Send an event when users click on the "set as default browser" entry in the toolbar menu
Copy link
Contributor

@mcarare mcarare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mcarare mcarare merged commit a3f4cee into mozilla-mobile:master Apr 15, 2021
@Amejia481
Copy link
Contributor Author

Thanks :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs:review PRs that need to be reviewed
Projects
None yet
4 participants