Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

For #11273: Add tabs tray telemetry #12036

Merged
merged 1 commit into from Jun 26, 2020

Conversation

sblatz
Copy link
Contributor

@sblatz sblatz commented Jun 26, 2020

Pull Request checklist

  • Tests: This PR includes thorough tests or an explanation of why it does not
  • Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
  • Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

After merge

  • Milestone: Make sure issues finished by this pull request are added to the milestone of the version currently in development.

To download an APK when reviewing a PR:

  1. click on Show All Checks,
  2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
  3. click on the "Fenix - assemble" task, then click "Run Artifacts".
  4. the APK links should be on the left side of the screen, named for each CPU architecture

@sblatz
Copy link
Contributor Author

sblatz commented Jun 26, 2020

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
    -What are the most common user flows? (taps on + open a tab, opening an exiting tab, closing a tab, closing the tab tray)
    -What % of users interact with the tabs tray 3 dot menu to save to a collection, share all tabs, or close all tabs?
    -What % of users switch to private mode and open a private tab (taps on the private mode icon and + open a private tap)

  2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

  • We need to know how useful these actions are so we know to dedicate more or less time to tabs tray development.
  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?
  • We do not have telemetry to track the tabs tray. We used to have similar telemetry for the home screen, so this is replacing that functionality!
  1. Can current instrumentation answer these questions?
  • No, see above.
  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the found on the Mozilla wiki.
  • All data is Category 2.
  1. How long will this data be collected?
  • Until 09/01/2020
  1. What populations will you measure?
  • All release, beta, and nightly users with telemetry enabled.
  1. Please provide a general description of how you will analyze this data.
  • Glean / Amplitude
  1. Where do you intend to share the results of your analysis?
  • Only on Glean, Amplitude and with mobile teams.

@sblatz sblatz requested a review from liuche June 26, 2020 22:53
@sblatz sblatz added the needs:data-review PR is awaiting a data review label Jun 26, 2020
Copy link
Contributor

@eliserichards eliserichards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! One outstanding question

notification_emails:
- fenix-core@mozilla.com
expires: "2020-09-01"
save_to_collection:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this save a single tab (e.g. the one you have opened/selected) to the collection? Or can you multiselect, save all, etc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All tabs, That's how the three dot menu works as it stands :)

Copy link
Contributor

@liuche liuche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data-review+ only
a few nits in the documentation clarity

Data Review Form (to be filled by Data Stewards)

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, tabs tray telemetry is documented in the docs/metrics.md

  1. Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.

Yes, user can turn this off in fenix telemetry settings

  1. If the request is for permanent data collection, is there someone who will monitor the data over time?

has expiry

  1. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Type 2, interaction with tabs tray

  1. Is the data collection request for default-on or default-off?

default on

  1. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

  1. Is the data collection covered by the existing Firefox privacy notice?

yes

  1. Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**

has an expiry

  1. Does the data collection use a third-party collection tool? If yes, escalate to legal.
    no

private_mode_tapped:
type: event
description: |
A user switched to private mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is from tabs tray only? assuming a data science person will see the whole sequence tabs_tray.private_mode_tapped then that should be fine, otherwise we might want to clarify that this doesn't include homescreen mode switching

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes tabs tray only.

normal_mode_tapped:
type: event
description: |
A user switched to normal mode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above ^

@liuche liuche removed the needs:data-review PR is awaiting a data review label Jun 26, 2020
@sblatz sblatz merged commit 101ecfb into mozilla-mobile:master Jun 26, 2020
@liuche liuche mentioned this pull request Jun 27, 2020
12 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants