Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple window and immersive mode time measurement telemetry. #3430

Merged
merged 5 commits into from Jun 12, 2020

Conversation

daoshengmu
Copy link
Contributor

Fixes #2279.

AC 41.0.0 already provides the ability to use coarse time telemetry, so I would migrate our resting metrics from Telemetry Service.

@daoshengmu daoshengmu self-assigned this May 27, 2020
@daoshengmu daoshengmu added this to the #11 polish milestone May 27, 2020
@daoshengmu
Copy link
Contributor Author

After updating to AC 41.0.0, I will see

ANTLR Tool version 4.5.3 used for code generation does not match the current runtime version 4.7.1ANTLR Runtime version 4.5.3 used for parser compilation does not match the current runtime version 4.7.1ANTLR Tool version 4.5.3 used for code generation does not match the current runtime version 4.7.1ANTLR Runtime version 4.5.3 used for parser compilation does not match the current runtime version 4.7.1Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

in my local. This error will not block me to generate installations and even Taskcluster build still be good.

The workaround should be [1]

configurations.all() {
    resolutionStrategy.force "org.antlr:antlr4-runtime:4.5.3"
    resolutionStrategy.force "org.antlr:antlr4-tool:4.5.3"
}

However, once I force our build config to use antlr4:4.5.3. I will see another error unable to load org.antlr.v4.runtime.CharStreams.

Moreover, if choosing to jump to AC 42.0.0. the build failure would block us. It looks like our build environment is using antlr4:4.7.1. But, some components from AC still be stuck with 4.5.3.

[1] https://gitmemory.com/issue/antlr/antlr4/1782/475545852

@daoshengmu daoshengmu marked this pull request as ready for review May 28, 2020 18:54
@daoshengmu daoshengmu requested a review from keianhzo May 28, 2020 18:54
@daoshengmu
Copy link
Contributor Author

@Dexterp37 Please help review the usage of Glean. It should be the last phase of migrating telemetry service data.

app/metrics.yaml Show resolved Hide resolved
app/metrics.yaml Outdated Show resolved Hide resolved
app/metrics.yaml Outdated
bugs:
- https://github.com/MozillaReality/FirefoxReality/issues/2279
data_reviews:
- https://github.com/MozillaReality/FirefoxReality/pull/xxx

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't forget to add the data review response link here and to the other metrics.

app/metrics.yaml Outdated Show resolved Hide resolved
app/metrics.yaml Outdated Show resolved Hide resolved
app/metrics.yaml Outdated Show resolved Hide resolved
app/metrics.yaml Show resolved Hide resolved
Copy link

@Dexterp37 Dexterp37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r+ on the glean api usage. Please make sure to have this code reviewed by a FxR peer as well and to update the data-review url in the YAML files before merging.

app/metrics.yaml Show resolved Hide resolved
app/metrics.yaml Show resolved Hide resolved
Copy link
Collaborator

@keianhzo keianhzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Needs rebase and update the data_reviews when ready.

@daoshengmu
Copy link
Contributor Author

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
  • Measuring how long each page takes to load.
  • Measuring if users use WebXR APIs to enter XR immersive mode, and how long they stay in this mode.
  • Measuring how long windows are kept open.
  • Counting how many times of moving a window in a session.
  • Counting how many times of resizing a window in a session.
  • Measure how long the left/right/front side window is active in a session.
  • Measuring how long a user uses single/double/triple window in a session.
  • Measuring how long a user uses single/double/triple private window in a session.
  • Counting which multiple window mode (single, double, triple) users are using in a session.
  • Counting which multiple private window mode (single, double, triple) users are using in a session.
  1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
  • Measuring the page loading time could help us figure out the performance problem.
  • Measuring how long users stay in the immersive mode could help us understand if the existing WebXR experience are good enough for users to stay longer.
  • Measuring window opening time to realize a window's lifetime and how users like to use multiple windows.
  • To realize if users like to resize and move a window in a session.
  • To understand in multiple window mode, which placement of a window are users more likely to use.
  • To understand in general, how many windows do users like to use in a session and how long will users stay at that mode. That would help us determine how many windows would be more popular for us to provide.
  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?
  • N/A. Those metrics are migrated from #2323.
  1. Can current instrumentation answer these questions?
  • Currently no, and those metrics are migrated from #2323.
  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the found on the Mozilla wiki.
  • All data is Category 2.
  1. How long will this data be collected?

Until 11/01/2020 and will consider to expand the expire date if need.

  1. What populations will you measure?
  • All release, beta, and nightly users with telemetry enabled.
  1. Please provide a general description of how you will analyze this data.
  • Glean
  1. Where do you intend to share the results of your analysis?
  • Only on Glean and with Firefox Reality team.

@chutten, please help data review. @Dexterp37 was concerned about if we can collect the data from using private window mode. If it is unavailable, I could remove them if need.

@chutten
Copy link

chutten commented Jun 11, 2020

In future please include a full list of the data collections in the answer to Question 5.

Data Collection in Private Browsing Mode is permitted under the guidelines shared by Ehsan in September of 2018.

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Glean so is documented in its metrics documentation.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Glean so can be controlled through Firefox Reality's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

No. This collection will expire 2020-11-01.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

Yes. Daosheng Mu is responsible for renewing or removing the collection before it expires 2020-11-01.


Result: datareview+

@daoshengmu daoshengmu merged commit f9f8606 into master Jun 12, 2020
@daoshengmu daoshengmu deleted the coarseTimeTelemetry branch June 12, 2020 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate time measurement telemetry metrics to Glean
4 participants