/
metadata.yaml
75 lines (73 loc) · 3.53 KB
/
metadata.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
friendly_name: Internet Outages
# yamllint disable rule:line-length
description: |-
This contains a set aggregated metrics that correlate to internet
outages for different countries in the world.
The dataset contains the following fields:
- `country`: the Country code of the client.
- `city`: the City name (only for cities with a population >= 15000,
'unknown' otherwise).
- `datetime`: the date and the time (truncated to hour) the data was
submitted by the client.
- `proportion_undefined`: the proportion of users who failed to send
telemetry for a reason that was not listed in the other cases.
- `proportion_timeout`: the proportion of users that had their connection
timeout while uploading telemetry
([after 90s, in Firefox Desktop](https://searchfox.org/mozilla-central/rev/fa2df28a49883612bd7af4dacd80cdfedcccd2f6/toolkit/components/telemetry/app/TelemetrySend.jsm#81)).
- `proportion_abort`: the proportion of users that had their connection
terminated by the client (for example, terminating open connections before
shutting down).
- `proportion_unreachable`: the proportion of users that failed to upload
telemetry because the server was not reachable (e.g. because the host was
not reachable, proxy problems or OS waking up after a suspension).
- `proportion_terminated`: the proportion of users that had their connection
terminated internally by the networking code.
- `proportion_channel_open`: the proportion of users for which the upload
request was terminated immediately, by the client, because of a Necko
internal error.
- `avg_dns_success_time`: the average time it takes for a successful DNS
resolution, in milliseconds.
- `missing_dns_success`: counts how many sessions did not report the
`DNS_LOOKUP_TIME` histogram.
- `avg_dns_failure_time`: the average time it takes for an unsuccessful DNS
resolution, in milliseconds.
- `missing_dns_failure`: counts how many sessions did not report the
`DNS_FAILED_LOOKUP_TIME` histogram.
- `count_dns_failure`: the average count of unsuccessful DNS resolutions
reported.
- `ssl_error_prop`: the proportion of users that reported an error through
the `SSL_CERT_VERIFICATION_ERRORS` histogram.
- `avg_tls_handshake_time`: the average time after the TCP SYN to ready
for HTTP, in milliseconds.
Caveats with the data:
As with any observational data, there are many caveats and interpretation must
be done carefully. Below is a list of issues we have considered, but it is not
exhaustive.
- Firefox users are not representative of the general population in their
region.
- Users can experience multiple types of failures and so the proportions
are not summable. For example, if 2.4% of clients had a timeout and 2.6% of
clients had eUnreachable that doesn't necessarily mean that 5.0% of clients
had a timeout or a eUnreachable
- Geographical data is based on IPGeo databases. These databases are
imperfect, so some activity may be attributed to the wrong location.
Further, proxy and VPN usage can create geo-attribution errors.
- Data contains duplicates prior to mid-2022. See [this bug](https://bugzilla.mozilla.org/show_bug.cgi?id=1753470)
for further information.
owners:
- aplacitelli@mozilla.com
labels:
incremental: true
review_bugs:
- '1640204'
scheduling:
dag_name: bqetl_internet_outages
bigquery:
time_partitioning:
type: day
field: datetime
require_partition_filter: null
clustering:
fields:
- country
- city