diff --git a/datasets/streaming/error_aggregates/reference.md b/datasets/streaming/error_aggregates/reference.md index 457daea5f..f9cbca2c9 100644 --- a/datasets/streaming/error_aggregates/reference.md +++ b/datasets/streaming/error_aggregates/reference.md @@ -25,7 +25,7 @@ SELECT window_start, sum(gpu_crashes) AS gpu, sum(plugin_crashes) AS plugin, sum(gmplugin_crashes) AS gmplugin -FROM telemetry.error_aggregates +FROM telemetry.error_aggregates_v2 WHERE application = 'Firefox' AND (os_name = 'Darwin' or os_name = 'Linux' or os_name = 'Windows_NT') AND (channel = 'beta' or channel = 'release' or channel = 'nightly' or channel = 'esr') @@ -40,7 +40,7 @@ Get the number of `main_crashes` on Windows over a small interval ```sql SELECT window_start as time, sum(main_crashes) AS main_crashes -FROM telemetry.error_aggregates_v1 +FROM telemetry.error_aggregates_v2 WHERE application = 'Firefox' AND os_name = 'Windows_NT' AND channel = 'release' @@ -54,8 +54,9 @@ GROUP BY window_start ### Data sources -The aggregates in this data source are derived from main and crash [pings](../../pings.md). -Crash pings are only used to count/gather main crash events, all other errors (including all other crashes) are gathered from main pings. +The aggregates in this data source are derived from main, crash and core [pings](../../pings.md): +* crash pings are used to count/gather main and content crash events, all other errors from desktop clients (including all other crashes) are gathered from main pings +* core pings are used to count usage hours, first subsession and unique client counts. ## Scheduling @@ -63,36 +64,36 @@ The `error_aggregates` job is run continuously, using the Spark Streaming infras ## Schema -The `error_aggregates` table has the following columns which define its dimensions: +The `error_aggregates_v2` table has the following columns which define its dimensions: -* `window_start`: Beginning of interval when this sample was taken -* `window_end`: End of interval when this sample was taken (will always be 5 minutes more +* `window_start`: beginning of interval when this sample was taken +* `window_end`: end of interval when this sample was taken (will always be 5 minutes more than `window_start` for any given row) +* `submission_date_s3`: the date pings were submitted for a particular aggregate * `channel`: the channel, like `release` or `beta` * `version`: the version e.g. `57.0.1` * `display_version`: like version, but includes beta number if applicable e.g. `57.0.1b4` * `build_id`: the `YYYYMMDDhhmmss` timestamp the program was built, like `20160123180541`. This is also known as the `build ID` or `buildid` -* `application`: application name (e.g. `Firefox`) +* `application`: application name (e.g. `Firefox` or `Fennec`) * `os_name`: name of the OS (e.g. `Darwin` or `Windows_NT`) * `os_version`: version of the OS +* `architecture`: build architecture, e.g. `x86` * `country`: country code for the user (determined using geoIP), like `US` or `UK` * `experiment_id`: identifier of the experiment being participated in, such as `e10s-beta46-noapz@experiments.mozilla.org`, or null if no experiment * `experiment_branch`: the branch of the experiment being participated in, such as `control` or `experiment`, or null if no experiment -* `e10s_enabled`: whether E10s is enabled -* `gfx_compositor`: the graphics backend compositor used by the program, such as `d3d11`, `opengl` and `simple`. Null values may be reported as `none` as well -* `quantum_ready`: whether the ping was submitted from an instance of Firefox that was Quantum-ready (i.e. no old-style addons) And these are the various measures we are counting: * `usage_hours`: number of usage hours (i.e. total number of session hours reported by the pings in this aggregate, note that this might include time where people are not actively using the browser or their computer is asleep) +* `count`: number of pings processed in this aggregate * `main_crashes`: number of main process crashes (or just program crashes, in the non-e10s case) -* `content_crashes`: number of content process crashes +* `startup_crashes` : number of startup crashes +* `content_crashes`: number of content process crashes (`version => 58` only) +* `gpu_crashes`: number of GPU process crashes * `plugin_crashes`: number of plugin process crashes * `gmplugin_crashes`: number of Gecko media plugin (often abbreviated `GMPlugin`) process crashes -* `content_shutdown_crashes`: number of content process crashes that were caused by failure to shut down in a timely manner (should always be - less than the number of `content_crashes`) -* `gpu_crashes`: number of GPU process crashes +* `content_shutdown_crashes`: number of content process crashes that were caused by failure to shut down in a timely manner (`version => 58` only) * `browser_shim_usage_blocked`: number of times a CPOW shim was blocked from being created by browser code * `permissions_sql_corrupted`: number of times the permissions SQL error occurred (beta/nightly only) * `defective_permissions_sql_removed`: number of times there was a removal of defective `permissions.sqlite` (beta/nightly only)