For #17418 - Added telemetry for Google Default Top Site #17637

codrut-topliceanu · 2021-01-26T14:20:05Z

Pull Request checklist

Tests: This PR doesn't include any tests as it is only a minor change
Screenshots: The code in this PR does not include any user facing features.
Accessibility: The code in this PR does not include any user facing features.

To download an APK when reviewing a PR:

click on Show All Checks,
click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
click on the "Fenix - assemble" task, then click "Run Artifacts".
the APK links should be on the left side of the screen, named for each CPU architecture

codrut-topliceanu · 2021-01-26T14:21:34Z

@lwa-moz I'm not entirely sure what to write in the data review form.

lwa-moz · 2021-01-26T19:13:45Z

@codrut-topliceanu this PR has a good example of a completed data collection review forms. I believe the form inputs in this case are almost identical to those listed in the linked PR.

codecov-io · 2021-01-27T12:48:52Z

Codecov Report

Merging #17637 (bbe9251) into master (ffc89c5) will decrease coverage by 0.78%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master   #17637      +/-   ##
============================================
- Coverage     33.34%   32.56%   -0.79%     
+ Complexity     1345     1314      -31     
============================================
  Files           458      455       -3     
  Lines         19200    19091     -109     
  Branches       2685     2670      -15     
============================================
- Hits           6403     6217     -186     
- Misses        12265    12383     +118     
+ Partials        532      491      -41

Impacted Files	Coverage Δ	Complexity Δ
...java/org/mozilla/fenix/components/metrics/Event.kt	`33.85% <0.00%> (-1.12%)`	`0.00 <0.00> (ø)`
...la/fenix/components/metrics/GleanMetricsService.kt	`12.20% <0.00%> (+0.29%)`	`5.00 <0.00> (ø)`
...ix/home/sessioncontrol/SessionControlController.kt	`73.86% <0.00%> (-1.57%)`	`0.00 <0.00> (ø)`
...la/fenix/customtabs/CustomTabToolbarIntegration.kt	`0.00% <0.00%> (-100.00%)`	`0.00% <0.00%> (-1.00%)`
...lla/fenix/components/toolbar/BrowserToolbarView.kt	`0.00% <0.00%> (-50.84%)`	`0.00% <0.00%> (-15.00%)`
...g/mozilla/fenix/customtabs/CustomTabToolbarMenu.kt	`32.32% <0.00%> (-44.45%)`	`3.00% <0.00%> (ø%)`
...zilla/fenix/settings/sitepermissions/Extensions.kt	`0.00% <0.00%> (-29.42%)`	`0.00% <0.00%> (ø%)`
...enix/home/intent/OpenSpecificTabIntentProcessor.kt	`56.25% <0.00%> (-18.75%)`	`3.00% <0.00%> (ø%)`
...lla/fenix/components/toolbar/ToolbarIntegration.kt	`0.00% <0.00%> (-15.56%)`	`0.00% <0.00%> (-1.00%)`
...ava/org/mozilla/fenix/tabtray/TabTrayViewHolder.kt	`46.07% <0.00%> (-12.68%)`	`13.00% <0.00%> (ø%)`
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffc89c5...9ba6ece. Read the comment docs.

eliserichards · 2021-01-27T19:38:23Z

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt

+        if (url == SupportUtils.GOOGLE_XX_URL) {
+            metrics.track(Event.TopSiteOpenGoogleSearchAttribution)
+        }


Are there tests for the DefaultSessionControlController that we can add for this?

There should be

eliserichards · 2021-01-27T19:50:19Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

What questions will you answer with this data?

This probe will be used to better model how the Top Sites location contributes to high-level trends in search. This will help us understand the revenue impact of our Top Sites with regards to Search.

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
This helps us understand our users' needs and the impact it has on our Search revenue and trends.
What alternative methods did you consider to answer these questions? Why were they not sufficient?

No current alternative methods.

Can current instrumentation answer these questions?

Currently there are no events to track specifics for the Google top site.

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

All data is Category 2

How long will this data be collected?

Until 2020-08-01 with the option to renew at that point.

What populations will you measure?

All release, beta, and nightly users with telemetry enabled.

Please provide a general description of how you will analyze this data.

Glean / Amplitude

Where do you intend to share the results of your analysis?

Only on Glean, Amplitude and with mobile teams.

app/metrics.yaml

gabrielluong · 2021-01-27T20:06:23Z

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt

+        if (url == SupportUtils.GOOGLE_XX_URL) {
+            metrics.track(Event.TopSiteOpenGoogleSearchAttribution)
+        }


There should be

gabrielluong · 2021-01-27T21:20:02Z

I have asked megan from DS to take a look in case we are missing anything that is needed for the search telemetries.

st3fan · 2021-01-27T22:09:36Z

I added the pr:do-not-land label because I think this is not the correct telemetry to have. I think what we need instead is this one:

https://github.com/mozilla-mobile/fenix/blob/master/app/metrics.yaml#L903-L931

That metric will collect counts for engines and the source to identify how the search was initiated. We probbaly need to introduce a topsites source for this.

boek

Data Review Form (to be filled by Data Stewards)

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
Yes, metrics.yaml and metrics.md
Is there a control mechanism that allows the user to turn the data collection on and off?
Yes, data collection settings
If the request is for permanent data collection, is there someone who will monitor the data over time?
Has expiry
Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Type 2
Is the data collection request for default-on or default-off?
Default on
Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?
No
Is the data collection covered by the existing Firefox privacy notice? If unsure: escalate to legal if:
Yes
Does there need to be a check-in in the future to determine whether to renew the data?
Fenix team will check-in at expiry
Does the data collection use a third-party collection tool?
No

codrut-topliceanu · 2021-02-01T13:27:21Z

Thanks everyone for helping me with this patch. I really appreciate it. ❤️

@st3fan I have refactored my patch to use the search_count metric as requested. Please let me know if any other changes are needed.

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt

Addressed comments :)

gabrielluong

Can we also add a test for this?

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt

This is used to track if the `InContentTelemetry` is a result of the user using the Google Top Site. It looks for `&channel=ts` within the uri.

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt

pocmo · 2021-02-10T16:45:56Z

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt

+                        searchAccessPoint.let { sap ->
+                            MetricsUtils.createSearchEvent(searchEngine, store, sap)
+                        }
+                    }


At the end of the day this code searches the Google SearchEngine instance only, with the check above. Should we simplify this?

If we want to keep it generic for supporting more search engines in the future then:

Should we check resultUrls instead of suggestUrl since the latter one is the URL we query search suggestions from and doesn't have to be related to the search engine URL. The result URLs may be closer.

Should we also search in *.state.search.availableEngines? Otherwise you will not find a search engine if the user decided to hide it.

In appendSearchAttributionToUrlIfNeeded() we already change the URL to the one with the search attribution, so we already know the context, it seems complicated to then check this here again. 🤔 This would also allow us to directly lookup the search engine (e.g. Google) and not have to search for it here, I assume?

Very much agreed on this. I filed #17956 for this work so that we can move this patch into production sooner without another round of changes. I sent out an email to explain my thoughts behind that.

st3fan

Wonderful work - let's get it on Nightly 🤩

codrut-topliceanu · 2021-02-12T08:09:11Z

Thank you everyone who helped with this!! 🤗 You're all awesome!

* For #17418 - Adds channel "ts" to TrackKey This is used to track if the `InContentTelemetry` is a result of the user using the Google Top Site. It looks for `&channel=ts` within the uri. * For #17418 - Adds TopSite PerformedSearch back in * For #17418 - Check now looks for equality with GOOGLE_URL * For #17418 - Adds test for topSite changes

…8021)

mozilla-mobile#17637) * For mozilla-mobile#17418 - Adds channel "ts" to TrackKey This is used to track if the `InContentTelemetry` is a result of the user using the Google Top Site. It looks for `&channel=ts` within the uri. * For mozilla-mobile#17418 - Adds TopSite PerformedSearch back in * For mozilla-mobile#17418 - Check now looks for equality with GOOGLE_URL * For mozilla-mobile#17418 - Adds test for topSite changes

codrut-topliceanu added the Feature:Telemetry label Jan 26, 2021

codrut-topliceanu requested review from a team as code owners January 26, 2021 14:20

codrut-topliceanu added the needs:data-review PR is awaiting a data review label Jan 26, 2021

codrut-topliceanu force-pushed the 17418_google_top_site_telemetry branch from 9f88586 to bbe9251 Compare January 27, 2021 12:48

eliserichards previously requested changes Jan 27, 2021

View reviewed changes

eliserichards requested a review from boek January 27, 2021 19:46

gabrielluong suggested changes Jan 27, 2021

View reviewed changes

st3fan added the pr:do-not-land label Jan 27, 2021

gabrielluong self-assigned this Jan 28, 2021

boek approved these changes Jan 29, 2021

View reviewed changes

st3fan suggested changes Feb 1, 2021

View reviewed changes

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt Outdated Show resolved Hide resolved

eliserichards added needs:review PRs that need to be reviewed and removed needs:data-review PR is awaiting a data review labels Feb 2, 2021

gabrielluong reviewed Feb 4, 2021

View reviewed changes

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt Outdated Show resolved Hide resolved

gabrielluong mentioned this pull request Feb 5, 2021

For #17418 - Add event ping telemetry for the Google Top Site click #17862

Merged

3 tasks

For mozilla-mobile#17418 - Adds channel "ts" to TrackKey

b7873fa

This is used to track if the `InContentTelemetry` is a result of the user using the Google Top Site. It looks for `&channel=ts` within the uri.

codrut-topliceanu force-pushed the 17418_google_top_site_telemetry branch from d213c0a to b7873fa Compare February 10, 2021 15:10

For mozilla-mobile#17418 - Adds TopSite PerformedSearch back in

0606de8

st3fan requested a review from pocmo February 10, 2021 16:29

gabrielluong reviewed Feb 10, 2021

View reviewed changes

app/src/main/java/org/mozilla/fenix/home/sessioncontrol/SessionControlController.kt Show resolved Hide resolved

pocmo reviewed Feb 10, 2021

View reviewed changes

For mozilla-mobile#17418 - Check now looks for equality with GOOGLE_URL

2f241fd

st3fan mentioned this pull request Feb 10, 2021

Improve search telemetry #17956

Closed

For mozilla-mobile#17418 - Adds test for topSite changes

9ba6ece

st3fan requested a review from gabrielluong February 11, 2021 14:57

gabrielluong approved these changes Feb 11, 2021

View reviewed changes

st3fan removed the pr:do-not-land label Feb 11, 2021

st3fan approved these changes Feb 11, 2021

View reviewed changes

gabrielluong merged commit d56b4a2 into mozilla-mobile:master Feb 11, 2021

gabrielluong mentioned this pull request Feb 11, 2021

Fenix Google Search tile [Telemetry] #17418

Closed

7 tasks

st3fan pushed a commit that referenced this pull request Feb 16, 2021

For #17418 - Added telemetry for Google Default Top Site (#17637)

88e7819

st3fan added a commit that referenced this pull request Feb 16, 2021

For #17418 - Added telemetry for Google Default Top Site (#17637) (#1…

c46381c

…8021)

eliserichards mentioned this pull request Jul 26, 2021

For #20516: Renew product telemetry probes expiring in August #20517

Merged

3 tasks

mdboom mentioned this pull request Aug 26, 2021

For #21025: Set long term business-critical metrics as non-expiring #21038

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For #17418 - Added telemetry for Google Default Top Site #17637

For #17418 - Added telemetry for Google Default Top Site #17637

codrut-topliceanu commented Jan 26, 2021

codrut-topliceanu commented Jan 26, 2021

lwa-moz commented Jan 26, 2021

codecov-io commented Jan 27, 2021 •

edited

Loading

eliserichards Jan 27, 2021

gabrielluong Jan 27, 2021

eliserichards commented Jan 27, 2021 •

edited

Loading

gabrielluong Jan 27, 2021

gabrielluong commented Jan 27, 2021

st3fan commented Jan 27, 2021

boek left a comment

codrut-topliceanu commented Feb 1, 2021

gabrielluong left a comment

pocmo Feb 10, 2021

st3fan Feb 10, 2021 •

edited

Loading

st3fan left a comment

codrut-topliceanu commented Feb 12, 2021

For #17418 - Added telemetry for Google Default Top Site #17637

For #17418 - Added telemetry for Google Default Top Site #17637

Conversation

codrut-topliceanu commented Jan 26, 2021

Pull Request checklist

To download an APK when reviewing a PR:

codrut-topliceanu commented Jan 26, 2021

lwa-moz commented Jan 26, 2021

codecov-io commented Jan 27, 2021 • edited Loading

Codecov Report

eliserichards Jan 27, 2021

Choose a reason for hiding this comment

gabrielluong Jan 27, 2021

Choose a reason for hiding this comment

eliserichards commented Jan 27, 2021 • edited Loading

Request for data collection review form

gabrielluong Jan 27, 2021

Choose a reason for hiding this comment

gabrielluong commented Jan 27, 2021

st3fan commented Jan 27, 2021

boek left a comment

Choose a reason for hiding this comment

Data Review Form (to be filled by Data Stewards)

codrut-topliceanu commented Feb 1, 2021

gabrielluong left a comment

Choose a reason for hiding this comment

pocmo Feb 10, 2021

Choose a reason for hiding this comment

st3fan Feb 10, 2021 • edited Loading

Choose a reason for hiding this comment

st3fan left a comment

Choose a reason for hiding this comment

codrut-topliceanu commented Feb 12, 2021

codecov-io commented Jan 27, 2021 •

edited

Loading

eliserichards commented Jan 27, 2021 •

edited

Loading

st3fan Feb 10, 2021 •

edited

Loading