Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

Closes #18173: Add Telemetry When User Opens a Bookmark #18174

Merged
merged 1 commit into from
Mar 8, 2021

Conversation

rocketsroger
Copy link
Contributor

@rocketsroger rocketsroger commented Feb 25, 2021

Fixes #18173

Pull Request checklist

  • Tests: This PR includes thorough tests or an explanation of why it does not
  • Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
  • Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

To download an APK when reviewing a PR:

  1. click on Show All Checks,
  2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
  3. click on the "Fenix - assemble" task, then click "Run Artifacts".
  4. the APK links should be on the left side of the screen, named for each CPU architecture

@rocketsroger rocketsroger added needs:data-review PR is awaiting a data review needs:review PRs that need to be reviewed labels Feb 25, 2021
@rocketsroger rocketsroger requested review from a team as code owners February 25, 2021 16:38
@rocketsroger
Copy link
Contributor Author

Request for data collection review

  1. What questions will you answer with this data?
    More understanding if user behaviour related to bookmarks
  2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

    This allows us to understand if bookmark management needs improvement
  3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

    No alternative is available to answer this question.
  4. Can current instrumentation answer these questions?

    No current instrumentation can answer this question.
  5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.
Measurement Description Data Collection Category Tracking Bug #
Bookmark opened category 2 - interaction data #18173
  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
    https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md
  2. How long will this data be collected? Choose one of the following:

    This is scoped to a time-limited experiment/project until date 08-01-2021.
  3. What populations will you measure?

    No filters - all locales, channels, etc.
  4. If this data collection is default on, what is the opt-out mechanism for users?

    Default Glean SDK opt-out mechanism.
  5. Please provide a general description of how you will analyze this data.

    Glean / Amplitude
  6. Where do you intend to share the results of your analysis?

    Only on Glean, Amplitude and with mobile teams.
  7. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
    
No third-party tool will use this data.

@mergify
Copy link
Contributor

mergify bot commented Mar 2, 2021

This pull request has conflicts when rebasing. Could you fix it @rocketsroger? 🙏

@travis79
Copy link
Member

travis79 commented Mar 8, 2021

@rocketsroger In regards to the data-review, I have a question about this one. It looks like Fenix already records counts of when bookmarks are opened (in several different contexts) and I am just confirming that having an event here is necessary to answer the questions for this experiment/project.

Request for data collection review

1. What questions will you answer with this data?
   More understanding if user behaviour related to bookmarks

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
   
   This allows us to understand if bookmark management needs improvement

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?
   
   No alternative is available to answer this question.

4. Can current instrumentation answer these questions?
   
   No current instrumentation can answer this question.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description Data Collection Category Tracking Bug #
Bookmark opened category 2 - interaction data #18173

1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
   [https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md](https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md?rgh-link-date=2021-02-25T16%3A45%3A43Z)

2. How long will this data be collected? Choose one of the following:
   
   This is scoped to a time-limited experiment/project until date 08-01-2021.

3. What populations will you measure?
   
   No filters - all locales, channels, etc.

4. If this data collection is default on, what is the opt-out mechanism for users?
   
   Default Glean SDK opt-out mechanism.

5. Please provide a general description of how you will analyze this data.
   
   Glean / Amplitude

6. Where do you intend to share the results of your analysis?
   
   Only on Glean, Amplitude and with mobile teams.

7. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
   
   No third-party tool will use this data.

Data Review

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, via the metrics.md and other Glean tools.

  1. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes

  1. If the request is for permanent data collection, is there someone who will monitor the data over time?

Not applicable, the collection is set to end 2021-08-01.

  1. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, User Interaction

  1. Is the data collection request for default-on or default-off?

Default on

  1. Does the instrumentation include the addition of any new identifiers?

No

  1. Is the data collection covered by the existing Firefox privacy notice?

Yes

  1. Does there need to be a check-in in the future to determine whether to renew the data?

No, unless the owner decides to extend beyond 2021-08-01

  1. Does the data collection use a third-party collection tool?

No

Result:

data-review+

@travis79 travis79 removed the needs:data-review PR is awaiting a data review label Mar 8, 2021
@rocketsroger
Copy link
Contributor Author

rocketsroger commented Mar 8, 2021

@rocketsroger In regards to the data-review, I have a question about this one. It looks like Fenix already records counts of when bookmarks are opened (in several different contexts) and I am just confirming that having an event here is necessary to answer the questions for this experiment/project.

Yes, we already do track when bookmarks are opened. However, Fenix didn't distinguish those events correctly and so we couldn't tell which method the user used to opened the bookmarks. This change is to make sure each method have a different event so we can know what feature is being used to open bookmarks.

Copy link
Contributor

@jonalmeida jonalmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing the tests for everything! ✅

Copy link
Member

@gabrielluong gabrielluong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also see we have unit tests for this in BookmarkFragmentInteractorTest.

open:
type: event
description: |
A user opened a bookmark.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add where the bookmark was opened from? I can imagine in the future we might want to extend top sites to allow bookmarks so it would be good to know where its source is from the description.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do track where the bookmark is opened from. But for some reason we never tracked the normal use case where the user taps on the bookmark directly. This is to fix it so we have all 5 cases covered. The 5 cases have the following events:
open (User tapes on the bookmark directly)
open_in_new_tab (User taps on the 3 dot menu and opens the bookmark)
open_in_new_tabs (User selected multiple bookmark and then tap on the 3 dot menu to open the bookmarks)
open_in_private_tab (User taps on the 3dot menu to open the bookmark in a private tab)
open_in_private_tabs (User taps on the 3 dot menu to open multiple bookmarks in private tabs)

I added this sheet in the meta issue but forgot to include it here. https://docs.google.com/spreadsheets/d/1EB6Mme0LR3GRE00YkX7Vm1VJOuTSnhwrSc4MOoySX5A/edit?usp=sharing. This will help clear what event tracks what use case. Thanks

@codecov-io
Copy link

Codecov Report

Merging #18174 (6f85b54) into master (2322cbd) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #18174      +/-   ##
============================================
+ Coverage     33.17%   33.21%   +0.03%     
- Complexity     1415     1416       +1     
============================================
  Files           461      461              
  Lines         19514    19518       +4     
  Branches       3005     3005              
============================================
+ Hits           6474     6483       +9     
- Misses        12191    12196       +5     
+ Partials        849      839      -10     
Impacted Files Coverage Δ Complexity Δ
...la/fenix/components/metrics/GleanMetricsService.kt 18.87% <100.00%> (+4.39%) 5.00 <0.00> (ø)
...rg/mozilla/fenix/components/toolbar/ToolbarMenu.kt 69.56% <0.00%> (-17.40%) 0.00% <0.00%> (ø%)
...java/org/mozilla/fenix/perf/PerformanceInflater.kt 80.00% <0.00%> (-15.00%) 4.00% <0.00%> (-1.00%)
...components/toolbar/BrowserToolbarMenuController.kt 60.20% <0.00%> (-9.95%) 0.00% <0.00%> (ø%)
...g/mozilla/fenix/tabhistory/TabHistoryViewHolder.kt 100.00% <0.00%> (ø) 4.00% <0.00%> (+2.00%)
...java/org/mozilla/fenix/components/metrics/Event.kt 38.69% <0.00%> (+1.78%) 0.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2322cbd...6f85b54. Read the comment docs.

@rocketsroger rocketsroger merged commit 56b08ab into mozilla-mobile:master Mar 8, 2021
@rocketsroger rocketsroger deleted the FB_18173 branch March 8, 2021 22:12
@rocketsroger rocketsroger removed the needs:review PRs that need to be reviewed label Mar 8, 2021
@rocketsroger rocketsroger added this to the 88 milestone Mar 8, 2021
@gabrielluong gabrielluong removed this from the 88 milestone Mar 9, 2021
pkirakosyan pushed a commit to gexsi/user-agent-android that referenced this pull request Aug 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Telemetry When User Opens a Bookmark
5 participants