Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

For #18836: add telemetry for start up type #19028

Merged

Conversation

mcomella
Copy link
Contributor

@mcomella mcomella commented Apr 14, 2021

@MarcLeclair For review
@mmccorks for data review

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
  • What are the most common types of start up?
  • For users in the 95th percentile of slow start ups – we assume the most likely to leave the app – what are the most common types of start ups?
  1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:
  • We want to know the most common types of start up so that we can invest perf resources into only the most common types
  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?

None: we need to understand user behavior in the wild

  1. Can current instrumentation answer these questions?

An existing probe app_opened_all_startup, tries to measure this but we found one thing wrong with it so I doubt its accuracy. So no.

  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

perf.startup.startup_type 2 - interaction #18836
  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md

  1. How long will this data be collected? Choose one of the following:
  • I want this data to be collected for 6 months initially (potentially renewable).
  1. What populations will you measure?
  • Which release channels?

All

  • Which countries?

All

  • Which locales?

All

  • Any other filters? Please describe in detail below.

N/A

  1. If this data collection is default on, what is the opt-out mechanism for users?

Telemetry opt out in app

  1. Please provide a general description of how you will analyze this data.
  • Compare different labels in GLAM
  • Write percentile query but filter for 95th percentile slow starts (probable heuristic instead: low-end devices)
  1. Where do you intend to share the results of your analysis?

Internally: GLAM, sql.tmo, gdocs

  1. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection? If so:

No

Pull Request checklist

  • Tests: This PR includes thorough tests or an explanation of why it does not
  • Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
  • Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

To download an APK when reviewing a PR:

  1. click on Show All Checks,
  2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
  3. click on the "Fenix - assemble" task, then click "Run Artifacts".
  4. the APK links should be on the left side of the screen, named for each CPU architecture

@mcomella mcomella requested review from a team as code owners April 14, 2021 15:52
@mergify
Copy link
Contributor

mergify bot commented Apr 14, 2021

This pull request has conflicts when rebasing. Could you fix it @mcomella? 🙏

@mcomella mcomella force-pushed the 18836-startup-type-telemetry-warm-hot branch from c2005f9 to cb76841 Compare April 14, 2021 16:10
@mcomella mcomella added the needs:data-review PR is awaiting a data review label Apr 14, 2021
Copy link
Contributor

@MarcLeclair MarcLeclair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me (Want to say again that I like this approach)

@mcomella mcomella changed the title For #18836: startup type telemetry warm hot For #18836: add telemetry for start up type Apr 14, 2021
@mcomella
Copy link
Contributor Author

unit test timeout while the rest passed: let's try again.

@mcomella mcomella closed this Apr 15, 2021
@mcomella mcomella reopened this Apr 15, 2021
Copy link

@mmccorks mmccorks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
    This will be documented in metrics.md

  2. Is there a control mechanism that allows the user to turn the data collection on and off?
    Yes, users can opt out of telemetry collection.

  3. If the request is for permanent data collection, is there someone who will monitor the data over time?
    Not permanent data collection.

  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
    Category 2, Interaction Data.

  5. Is the data collection request for default-on or default-off?
    Default on.

  6. Does the instrumentation include the addition of any new identifiers?
    No new identifiers.

  7. Is the data collection covered by the existing Firefox privacy notice?
    Yes.

  8. Does there need to be a check-in in the future to determine whether to renew the data?
    Yes, check in in 6 months to renew if needed (October 2021).

  9. Does the data collection use a third-party collection tool?
    No.


data-review +

@mcomella
Copy link
Contributor Author

Build failure was a network issue:

 Could not determine the dependencies of task ':app:compileDebugKotlin'.
> Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
   > Could not download leanplum-fcm-5.4.0.aar (com.leanplum:leanplum-fcm:5.4.0)
      > Could not get resource 'https://repo.maven.apache.org/maven2/com/leanplum/leanplum-fcm/5.4.0/leanplum-fcm-5.4.0.aar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/leanplum/leanplum-fcm/5.4.0/leanplum-fcm-5.4.0.aar'.
            > Connection reset

Let's rerun.

@mcomella mcomella closed this Apr 15, 2021
@mcomella mcomella reopened this Apr 15, 2021
@mcomella mcomella removed the needs:data-review PR is awaiting a data review label Apr 16, 2021
@mcomella mcomella merged commit cbc5df3 into mozilla-mobile:master Apr 16, 2021
@mcomella mcomella deleted the 18836-startup-type-telemetry-warm-hot branch April 16, 2021 22:30
@mcomella mcomella added the Feature:Performance Used for data reviews to label metrics related to performance label May 26, 2021
@mergify
Copy link
Contributor

mergify bot commented Jun 2, 2022

This pull request has conflicts when rebasing. Could you fix it @mcomella? 🙏

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Feature:Performance Used for data reviews to label metrics related to performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants