Review of Telemetry

[nancyhang]

1. What questions will you answer with this data?

This will allow us to understand which features are being used and how are they used. Based on the data, we will have a better scope of shipping the right features to users and obtain actionable insights for the product. We will understand if any changes to the product (e.g. GeckoView, other features) will increase/decrease the Monthly Active Users and Daily Active Users.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

• Establish baselines or measure changes in product or platform quality or performance.
• Provide information essential for advancing a business objective such as supporting OKRs.
• Determine whether a product or platform change has an effect on user or browser behavior.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

We have access to the number of sessions, but we need to understand engagement on a more micro level.

4. Can current instrumentation answer these questions?

No.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the Mozilla Wiki.

Measurement Description	Data Collection Category	Tracking Bug #
DAU/WAU/MAU (Engagement Results)	Category 2 Interaction data	#85
Session length distribution	Category 2 Interaction data	#85
Engagement ratio	Category 2 Interaction data	#85
Number of users	Category 2 Interaction data	#85
New user retention	Category 2 Interaction data	#85
DAUs by locale	Category 2 Interaction data	#85
Distribution of Devices Over Time	Category 1 Technical data	#85
Action Counts	Category 2 Interaction data	#85 #395
Search Engine Counts	Category 2 Interaction data	#395
Search Counts	Category 2 Interaction data	#395

Sessions

Event	Category	Method	Object	Value
Start session (App is in the foreground)	Action	Foreground	App	N/A
Stop session (App is in the background)	Action	Background	App	N/A

Browsing

Event	Category	Method	Object	Value
URL entered	Action	type_url	search_bar	N/A
Search entered	Action	type_query	search_bar	bundled engine name
Voice query	Action	voice_query	voice_input	bundled engine name

6. How long will this data be collected? Choose one of the following:

I want to permanently monitor this data. (@larsbergstrom)

7. What populations will you measure?

All users opting in to telemetry

8. Which release channels?

These VR app stores: Oculus, Google Play, and Viveport.

9. Which countries?

Worldwide

10. Which locales?

Worldwide

11. If this data collection is by default on, what is the opt-out mechanism for users?

Users can go into the Developer Settings panel to toggle off.

12. Please provide a general description of how you will analyze this data.

Look at the features used over time to see if it increases with improvements to the app.

13. Where do you intend to share the results of your analysis?

Internal use only.

[larsberg]

@nancyhang I think you've got the wrong Lars Berg do you mean @larsbergstrom ? If he's okay with it it's fine with me if you'd like to permanently monitor that stuff :)

[nancyhang]

@larsberg Yep, I meant @larsbergstrom. Good catch!

[cvan]

@nancyhang: is this issue open to track the updating of https://github.com/MozillaReality/FirefoxReality/wiki/Telemetry? or something else?

[nancyhang]

@cvan it's a different, but somewhat related to the telemetry link. This needs to stay open.

[harraton]

An updated is needed to provide more detail around each individual measurement. See https://bugzilla.mozilla.org/attachment.cgi?id=9008148&action=edit for an example.

[larsbergstrom]

@harraton Do you have a pointer to the equivalent forms for Firefox Focus with GeckoView? Our results should be identical to it plus the WebVR telemetry ones we created.

[harraton]

Hi @larsbergstrom, if my sources are correct, for GV the gecko-internal stuff is grandfathered in under the data collection reviews [of which there are probably many] that added them to Gecko. Please document the new WebVR measurements. For the GV measurements, please just indicate that they are from GV.

[harraton]

https://github.com/mozilla-mobile/focus-android/wiki/Telemetry

[larsbergstrom]

Thanks for the info, @harraton! I'm a bit confused on what needs to be done here with all the different forms, some things we rely on being "grandfathered" etc. Can you please have whomever owns this process and all these corner cases reach out to me and set up a half hour to chat the week of the 29th? Thanks.

[larsbergstrom]

Moving this to backlog due to inactivity.

[cvan]

FYI: https://github.com/mozilla-mobile/focus-android/wiki/Telemetry has been moved to https://github.com/mozilla-mobile/focus-android/blob/master/docs/Telemetry.md.

And https://github.com/MozillaReality/FirefoxReality/wiki/Telemetry is FxR's Wiki page.

[cvan]

Related: see mozilla-mobile/focus-android#2328 re: about:telemetry

Screenshot of Firefox desktop's about:telemetry:

[jvonitter]

https://github.com/MozillaReality/FirefoxReality/wiki/Telemetry has been updated with the correct info. closing this!