For #11704 Add leanplum attributes for tracking protection #11965

jhugman · 2020-06-25T16:55:27Z

Fixes #11704: added tracking_protection_enabled attribute
Fixes #11657: add LP attribute for tracking protection

Pull Request checklist

~~[ ] Tests: This PR includes thorough tests or an explanation of why it does not~~
~~[ ] Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not~~
[ ] Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

After merge

Milestone: Make sure issues finished by this pull request are added to the milestone of the version currently in development.

To download an APK when reviewing a PR:

click on Show All Checks,
click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
click on the "Fenix - assemble" task, then click "Run Artifacts".
the APK links should be on the left side of the screen, named for each CPU architecture

jhugman · 2020-06-25T17:57:29Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

What questions will you answer with this data?

To measure how often tracking protection is enabled/disabled.

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

To measure how often tracking protection is enabled/disabled, to help direct users discover ETP controls; this is in the context of a CFR like message experiment.

What alternative methods did you consider to answer these questions? Why were they not sufficient?

N/A (These are baseline metrics)

Can current instrumentation answer these questions?

No, while we are tracking some of these metrics in Glean, we need access to them in Leanplum for marketing usage.

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data c ollection categories on the found on the Mozilla wiki.

All data is Category 2.

How long will this data be collected?

This data will be present as long as the user ID exists in Leanplum.

What populations will you measure?

All release, beta, and nightly users with telemetry enabled.

Please provide a general description of how you will analyze this data.

Leanplum / Amplitude

Where do you intend to share the results of your analysis?

Only on Leanplum, Amplitude and with mobile teams.

liuche · 2020-06-26T02:11:10Z

@jhugman since there isn't an expiry for LeanPlum in the SDK, can you list who would be monitoring this data, and would be the contact for expiring/renewing if need be? I'll need that to complete the data review form.

Also wondering if it LP would do expiry for us at some point...

@cbonacuse or could you answer this question ^?

eliserichards

Looking good! Just a few questions :)

app/src/main/java/org/mozilla/fenix/components/metrics/LeanplumMetricsService.kt

docs/mma.md

cbonacuse · 2020-06-29T13:45:48Z

@liuche I'm working on getting the expiry info from Leanplum now. Stay tuned. cc: @jhugman

liuche

Data review+ only

Data Review Form (to be filled by Data Stewards)

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, this is documented in docs/mma.md

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, Leanplum is controlled by the data controls in Fenix settings

If the request is for permanent data collection, is there someone who will monitor the data over time?

@cbonacuse will monitor this, and we will also look into Leanplum expiry, or check in every 6mo

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Type 2, user interaction with tracking protection setting

Is the data collection request for default-on or default-off?

default on

Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

Is the data collection covered by the existing Firefox privacy notice?

yes

Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**

Yes, will do a review of LP probes 9/2020 @cbonacuse

Does the data collection use a third-party collection tool? If yes, escalate to legal.

Uses Leanplum, but that has been approved for use

sblatz · 2020-07-15T20:25:15Z

app/src/main/java/org/mozilla/fenix/components/metrics/LeanplumMetricsService.kt

@@ -101,6 +101,15 @@ class LeanplumMetricsService(private val application: Application) : MetricsServ

            val installedApps = MozillaProductDetector.getInstalledMozillaProducts(application)

+            val trackingProtection = application.settings().run {
+                when {
+                    !shouldUseTrackingProtection -> null


It may be fine, but I just want to make sure that Leanplum properly handles a "null" case here for when we data process this? I'm not tried adding a null value before.

Renamed to String none, and updated the docs.

codecov-commenter · 2020-07-16T15:29:53Z

Codecov Report

Merging #11965 into master will increase coverage by 0.01%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master   #11965      +/-   ##
============================================
+ Coverage     25.12%   25.14%   +0.01%     
  Complexity      845      845              
============================================
  Files           389      389              
  Lines         15695    15703       +8     
  Branches       2020     2023       +3     
============================================
+ Hits           3944     3949       +5     
- Misses        11444    11447       +3     
  Partials        307      307

Impacted Files	Coverage Δ	Complexity Δ
...fenix/components/metrics/LeanplumMetricsService.kt	`22.22% <0.00%> (-1.96%)`	`2.00 <0.00> (ø)`
...nix/components/toolbar/BrowserToolbarController.kt	`69.64% <0.00%> (+2.38%)`	`0.00% <0.00%> (ø%)`
.../fenix/browser/browsingmode/BrowsingModeManager.kt	`100.00% <0.00%> (+14.28%)`	`0.00% <0.00%> (ø%)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 667261f...1db2335. Read the comment docs.

jhugman added the needs:data-review PR is awaiting a data review label Jun 25, 2020

jhugman requested review from sblatz and eliserichards June 25, 2020 18:02

eliserichards reviewed Jun 26, 2020

View reviewed changes

app/src/main/java/org/mozilla/fenix/components/metrics/LeanplumMetricsService.kt Outdated Show resolved Hide resolved

docs/mma.md Show resolved Hide resolved

liuche approved these changes Jul 2, 2020

View reviewed changes

liuche removed the needs:data-review PR is awaiting a data review label Jul 2, 2020

jhugman requested a review from eliserichards July 15, 2020 14:04

sblatz approved these changes Jul 15, 2020

View reviewed changes

jhugman added 4 commits July 16, 2020 16:14

For 11657: add LP attribute for tracking protection

6cf4e91

For #11704: added tracking_protection_enabled attribute

d4d5fc7

Added docs for the new attributes, linking to data-review to the mma.md

1e06819

Rename null to none when no ETP is enabled

1db2335

jhugman force-pushed the jhugman/11657-add-lp-attributes branch from 0c2fb84 to 1db2335 Compare July 16, 2020 15:17

jhugman added the pr:needs-landing PRs that are ready to land [Will be merged by Mergify] label Jul 16, 2020

jhugman merged commit fb1d7e1 into master Jul 16, 2020

jhugman deleted the jhugman/11657-add-lp-attributes branch July 16, 2020 16:09

liuche mentioned this pull request Jul 20, 2020

Releng Firefox Beta 79.1.0 #12761

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For #11704 Add leanplum attributes for tracking protection #11965

For #11704 Add leanplum attributes for tracking protection #11965

jhugman commented Jun 25, 2020 •

edited

jhugman commented Jun 25, 2020 •

edited by cbonacuse

liuche commented Jun 26, 2020 •

edited

eliserichards left a comment

cbonacuse commented Jun 29, 2020

liuche left a comment

sblatz Jul 15, 2020

jhugman Jul 16, 2020

codecov-commenter commented Jul 16, 2020 •

edited

For #11704 Add leanplum attributes for tracking protection #11965

For #11704 Add leanplum attributes for tracking protection #11965

Conversation

jhugman commented Jun 25, 2020 • edited

Pull Request checklist

After merge

To download an APK when reviewing a PR:

jhugman commented Jun 25, 2020 • edited by cbonacuse

Request for data collection review form

liuche commented Jun 26, 2020 • edited

eliserichards left a comment

Choose a reason for hiding this comment

cbonacuse commented Jun 29, 2020

liuche left a comment

Choose a reason for hiding this comment

Data Review Form (to be filled by Data Stewards)

sblatz Jul 15, 2020

Choose a reason for hiding this comment

jhugman Jul 16, 2020

Choose a reason for hiding this comment

codecov-commenter commented Jul 16, 2020 • edited

Codecov Report

jhugman commented Jun 25, 2020 •

edited

jhugman commented Jun 25, 2020 •

edited by cbonacuse

liuche commented Jun 26, 2020 •

edited

codecov-commenter commented Jul 16, 2020 •

edited