Merge remote-tracking branch 'mozilla/master'

fzzzy · Jun 21, 2017 · 4aeafe5 · 4aeafe5
2 parents aabd228 + a4b561d
commit 4aeafe5
Show file tree

Hide file tree

Showing 8 changed files with 148 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -20,13 +20,14 @@ Test Pilot is not intended to replace trains for most features, nor is it a test
     - [Verifying deployments](docs/development/verification.md) - Verifying Test Pilot deployments.
     - [Add-on environment](docs/development/environment.md) - Configuring to which server the add-on connects.
 - Developing experiments
+    - [Experiment metrics](docs/experiments/ga.md) - The use of Google Analytics to track experiment data.
     - [Variant testing](docs/experiments/variants.md) - Creating variant (e.g. A/B) tests in experiments.
     - [Example experiments](docs/experiments/)
 - Metrics
     - [Telemetry](docs/metrics/telemetry.md) - How we use Firefox telemetry.
     - [Google Analytics](docs/metrics/ga.md) - How we use Google Analytics.
     - [New features](docs/metrics/new_features.md) - Everything needed to instrument something new.
-- [Experiment content] (docs/content/reference.md) - Management of experiment content.
+- [Experiment content](docs/content/reference.md) - Management of experiment content.
 - [Process](docs/process.md) - How we create, triage, and assign work.
 - [FAQ](docs/faq.md)
 - [Contributing to Test Pilot](CONTRIBUTING.md)

diff --git a/docs/experiments/ga.md b/docs/experiments/ga.md
@@ -0,0 +1,67 @@
+[👈 Back to README](../../README.md)
+
+# Experiment Metrics
+Test Pilot experiments use Google Analytics for metrics collection, and Google Data Studio for visualization and reporting. Each experiment should create and use a new property in Mozilla’s Google Analytics account. If you need help doing so, please talk to Wil Clouser.
+
+Events are reported through the low-level [Google Analytics Measurement Protocol](https://developers.google.com/analytics/devguides/collection/protocol/v1/). Refer to the documentation for the [developer guide](https://developers.google.com/analytics/devguides/collection/protocol/v1/devguide) and [parameter reference](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters) to understand to basics of how data is reported. The [hit builder](https://ga-dev-tools.appspot.com/hit-builder/) can help you construct and validate events before reporting.
+
+## Submission
+The following fields are used in experiment event reporting. All fields are required, unless noted. Experiments should implement additional properties of [sessions](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#session), [exceptions](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#exception), [social interactions](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#social), [traffic sources](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#trafficsources), [content information](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#content), and [system attributes](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#system), where appropriate.
+
+### General Fields
+- [`v`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#v) - this indicates the version of the measurement protocol being used. Currently always `1`.
+- [`tid`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#tid) - this indicates the ID of the Google Analytics property being used by the experiment. Should follow the form `UA-XXXX-Y`.
+- [`aip`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#aip) - this indicates that the IP address of the sender should be anonymized. Always `1`.
+- [`ds`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ds) - this indicates the context from which the event is being reported. Should be `addon` if sent from an experiment’s add-on, `web` if sent from an associated web property, or `app` if sent from an associated mobile application.
+- [`qt`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#qt) - if [batching](https://developers.google.com/analytics/devguides/collection/protocol/v1/devguide#batch) or delaying reports, this should represent the time delta between the event happening and the time it is being reported. This should not exceed 4 hours. _(Optional)_
+- [`z`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#z) - the unix timestamp of the event taking place, used for cache-busting.
+
+### User Fields
+- [`cid`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#cid) - a [UUIDv4](https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_.28random.29) that should be consistent across all events reported with this user. It should be stored with a persistent mechanism. If there is a sync component to the experiment, this should also be synced and be made consistent across clients and devices. This can be generated with [the `uuid` npm package](https://www.npmjs.com/package/uuid).
+- [`uid`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#uid) - the user’s telemetry ID, available at `toolkit.telemetry.cachedClientID`. _(Optional)_
+
+### Session
+- [`ua`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ua) - the user agent, as reported by [`navigator.userAgent`](https://developer.mozilla.org/en-US/docs/Web/API/NavigatorID/userAgent).
+
+### System Info
+- [`ul`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ul) - the user’s language, as reported by  [`navigator.language`](https://developer.mozilla.org/en-US/docs/Web/API/NavigatorLanguage/language).
+
+### Hit
+- [`t`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#t) - the type of hit. Usually `event` for Test Pilot usage.
+
+### App Tracking Fields
+- [`an`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#an) - the name of the Test Pilot experiment.
+- [`aid`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#aid) - the add-on’s ID, if one exists. _(Optional)_
+- [`av`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#av) - the add-on’s version number, if one exists. _(Optional)_
+- [`aiid`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#aiid) - an indication of the location of the experiment. For most usage, should be  `testpilot`. This should be changed if shipping with [Shield](https://wiki.mozilla.org/Firefox/Shield) or in moz-central.
+
+### Event Fields
+For more information about the components of an event and best practices for doing so, see [Google’s documentation on events](https://support.google.com/analytics/answer/1033068) .
+
+- [`ec`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ec) - the category of the event being reported.
+- [`ea`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ea) - the action of the event being reported.
+- [`el`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#el) - the event’s label. _(Optional)_
+- [`ev`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ev) - the event’s value, as an integer. _(Optional)_
+
+### Custom Dimension/Metric Fields
+Google Analytics allows you to report [custom dimensions and metrics](https://support.google.com/analytics/answer/2709828?hl=en) with each event, up to 20 of each. Care should be taken in choosing these, given their limited number and inability to repurpose them once used. They are [defined in the property’s admin](https://support.google.com/analytics/answer/2709828?hl=en#configuration) before being used, and their definition and use are documented in each experiment’s repository.
+
+- [`cd<index>`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#cd_) - the value of custom dimension number `<index>`.
+- [`cm<index>`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#cm_) - the value of custom metric number `<index>`.
+
+#### Global Custom Dimensions/Metrics
+Some custom dimensions and metrics are used consistently across Test Pilot experiments. These use the highest index numbers possible, for consistency across reports. These are all required.
+
+- `cd20` - indicates the Firefox release channel. This can be reverse-engineered by comparing the user agent to [information from `product-details`](https://product-details.mozilla.org/1.0/). Should be one of `esr`, `release`, `beta`, `developer`, or `nightly`.
+
+### Content Experiment Fields
+- [`xid`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#xid) - the ID of the current experiment._(Optional)_
+- [`xvar`](https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#xvar) - the name of the population to which the user belongs in the current experiment. _(Optional)_
+
+## Content Experiments
+By using the [content experiment fields](#content-experiment-fields), experiments are free to use Google Analytics’ [Content Experiments](https://support.google.com/analytics/answer/1745147?hl=en&ref_topic=1745207&visit_id=1-636335713647262387-4167257197&rd=1) for variant testing. Users may only belong to one experiment at a time.
+
+Eventually, this section will include more specific information about the use of content experiments. See [bug 2537](https://github.com/mozilla/testpilot/issues/2537) for details.
+
+## Visualization and Reporting
+Eventually, this section will include information about the use of Data Studio to analyze and visualize collected data. See [bug 2539](https://github.com/mozilla/testpilot/issues/2539) for details.
diff --git a/locales/es-AR/app.ftl b/locales/es-AR/app.ftl
@@ -101,6 +101,12 @@ emailOptInSuccessMessage2 = ¡Gracias!
 emailOptInConfirmationClose = Hacia los experimentos...
 
 
+// news feed updates.
+[[ updateList ]]
+
+latestUpdatesTitle = Últimas actualizaciones
+
+
 // A listing of all Test Pilot experiments.
 [[ experimentsList ]]
 
@@ -109,7 +115,8 @@ experimentListJustLaunchedTab = Recién lanzado
 experimentListJustUpdatedTab = Recién actualizado
 experimentListEndingTomorrow = Termina mañana
 experimentListEndingSoon = Termina pronto
-experimentsListCondensedHeader = ¡Elegí tus experimentos!
+experimentCondensedHeader = ¡Bienvenido a Test Pilot!
+experimentListHeader = ¡Elegí tus experimentos!
 
 
 // An individual experiment in the listing of all Test Pilot experiments.
@@ -280,3 +287,18 @@ noScriptLink = Encuentre el motivo
 
 viewPastExperiments = Ver experimentos pasados
 hidePastExperiments = Ocultar experimentos pasados
+
+
+// Text of warnings to the user if various error conditions are detected
+[[ warnings ]]
+
+warningGenericTitle = ¡Algo está mal!
+warningGenericDetail = Algo falló con Test Pilot. <a>Informá la falla</a> y mencioná este mensaje de error.
+warningUpgradeFirefoxTitle = ¡Actualizá Firefox para continuar!
+warningUpgradeFirefoxDetail = Test Pilot requiere la última versión de Firefox. <a>Actualizá Firefox</a> para comenzar.
+warningHttpsRequiredTitle = ¡HTTPS requerido!
+warningHttpsRequiredDetail = Test Pilot debe accederse  mediante HTTPS. Mirá <a>nuestra documentación</a> para más detalles.
+warningMissingPrefTitle = ¿Desarrollando Test Pilot?
+warningMissingPrefDetail = Al ejecutar Test Pilot localmente o en entornos de desarrollo, se necesita una configuración especial. Mirá <a>nuestra documentación</a> para más detalles.
+warningBadHostnameTitle = ¡Nombre de servidor no aprobado!
+warningBadHostnameDetail = El sitio de Test Pilot solamente puede ser accedido desde  testpilot.firefox.com, testpilot.stage.mozaws.net, testpilot.dev.mozaws.net, o example.com:8000.  Mirá <a>nuestra documentación</a> para más detalles.
diff --git a/locales/es-ES/app.ftl b/locales/es-ES/app.ftl
@@ -284,3 +284,18 @@ noScriptLink = Descubre por qué
 
 viewPastExperiments = Ver experimentos antiguos
 hidePastExperiments = Ocultar experimentos antiguos
+
+
+// Text of warnings to the user if various error conditions are detected
+[[ warnings ]]
+
+warningGenericTitle = ¡Algo está mal!
+warningGenericDetail = Algo ha fallado con Test Pilot. Por favor <a>informa del error</a> y menciona este mensaje de error.
+warningUpgradeFirefoxTitle = ¡Actualiza Firefox para continuar!
+warningUpgradeFirefoxDetail = Test Pilot requiere la última versión de Firefox. <a>Actualiza Firefox</a> para comenzar.
+warningHttpsRequiredTitle = ¡Se requiere HTTPS!
+warningHttpsRequiredDetail = Test Pilot debe ser accedido mediante HTTPS. Por favor mira <a>nuestra documentación</a> para más detalles.
+warningMissingPrefTitle = ¿Desarrollando Test Pilot?
+warningMissingPrefDetail = Al ejecutar Test Pilot localmente o en entornos de desarrollo, se necesita una configuración especial. Por favor mira <a>nuestra documentación</a> para más detalles.
+warningBadHostnameTitle = ¡Nombre de servidor no aprobado!
+warningBadHostnameDetail = El sitio de Test Pilot solo puede ser accedido desde testpilot.firefox.com, testpilot.stage.mozaws.net, testpilot.dev.mozaws.net, o example.com:8000. Mira <a>nuestra documentación</a> para más detalles.
diff --git a/locales/fa/app.ftl b/locales/fa/app.ftl
@@ -284,3 +284,17 @@ noScriptLink = ببینید چرا
 
 viewPastExperiments = نمایش آزمایش‌های گذشته
 hidePastExperiments = مخفی کردن آزمایش‌های گذشته
+
+
+// Text of warnings to the user if various error conditions are detected
+[[ warnings ]]
+
+warningGenericTitle = خطایی رخ داده است!
+warningGenericDetail = مشکلی در خلبان آزمایشی رخ داده است. لطفا <a>یک گزارش خطا ایجاد کنید<a/> و این پیغام خطا را ذکر نمایید.
+warningUpgradeFirefoxTitle = برای ادامه، فایرفاکس را ارتقا دهید!
+warningUpgradeFirefoxDetail = خلبان آزمایشی به آخرین نسخه فایرفاکس احتیاج دارد، برای شروع <a>فایرفاکس را ارتقا دهید</a>.
+warningHttpsRequiredTitle = HTTPS مورد نیاز است!
+warningHttpsRequiredDetail = خلبان آزمایشی باید از طریق HTTPS مورد استفاده قرار بگیرد. برای جزییات لطفا <a>مستندات ما</a> را ببینید.
+warningMissingPrefDetail = وقتی خلبان آزمایشی را محلی یا در محیط‌های توسعه اجرا می‌کنید، پیکربندی ویژه‌ای مورد نیاز است. برای جزییات لطفا <a>مستندات ما<a/> را ببینید.
+warningBadHostnameTitle = نام میزبان تایید نشده!
+warningBadHostnameDetail = ممکن است خلبان آزمایشی تنها از طریق testpilot.firefox.com، testpilot.stage.mozaws.net، testpilot.dev.mozaws.net یا example.com:8000 دردسترس باشد. برای جزییات لطفا <a>مستندات ما</a> را ببینید.
diff --git a/locales/fy-NL/app.ftl b/locales/fy-NL/app.ftl
@@ -284,3 +284,18 @@ noScriptLink = Besjoch wêrom
 
 viewPastExperiments = Alde eksperiminten besjen
 hidePastExperiments = Alde eksperiminten ferstopje
+
+
+// Text of warnings to the user if various error conditions are detected
+[[ warnings ]]
+
+warningGenericTitle = Der is wat mis!
+warningGenericDetail = Der is wat misgien mei Test Pilot. <a>Meld in bug</a> en neam dizze flatermelding.
+warningUpgradeFirefoxTitle = Fernij Firefox om troch te gean!
+warningUpgradeFirefoxDetail = Test Pilot fereasket de nijste ferzje fan Firefox. <a>Fernij Firefox</a> om te begjinnen.
+warningHttpsRequiredTitle = HTTPS fereaske!
+warningHttpsRequiredDetail = Test Pilot moat fia HTTPS benadere wurde. Besjoch <a>ús dokumintaasje</a> foar details.
+warningMissingPrefTitle = Untwikkelje jo Test Pilot?
+warningMissingPrefDetail = Foar it lokaal of yn ûntwikkelomjouwingen útfieren fan Test Pilot is spesjale konfiguraasje fereaske. Besjoch <a>ús dokumintaasje</a> foar details.
+warningBadHostnameTitle = Net-goedkarde hostnamme!
+warningBadHostnameDetail = De Test Pilot-website mei allinnich fan testpilot.firefox.com, testpilot.stage.mozaws.net, testpilot.dev.mozaws.net of example.com:8000 ôf benadere wurde. Besjoch <a>ús dokumintaasje</a> foar details.
diff --git a/locales/sl/app.ftl b/locales/sl/app.ftl
@@ -299,5 +299,6 @@ warningUpgradeFirefoxDetail = Test Pilot zahteva najnovejšo različico Firefoxa
 warningHttpsRequiredTitle = HTTPS zahtevan!
 warningHttpsRequiredDetail = Dostop do Test Pilota mora biti preko HTTPS. Za podrobnosti glejte <a>našo dokumentacijo</a>.
 warningMissingPrefTitle = Razvijate Test Pilot?
+warningMissingPrefDetail = Kadar Test Pilot poganjate lokalno ali v razvojnih okoljih, je zahtevana posebna konfiguracija. Za podrobnosti glejte <a>našo dokumentacijo</a>.
 warningBadHostnameTitle = Neodobreno ime gostitelja!
 warningBadHostnameDetail = Do strani Test Pilota je omogočen dostop samo s testpilot.firefox.com, testpilot.stage.mozaws.net, testpilot.dev.mozaws.net ali example.com:8000. Za podrobnosti glejte <a>našo dokumentacijo</a>.
diff --git a/locales/zh-TW/app.ftl b/locales/zh-TW/app.ftl
@@ -284,3 +284,14 @@ noScriptLink = 找出原因
 
 viewPastExperiments = 檢視先前的實驗
 hidePastExperiments = 隱藏先前的實驗
+
+
+// Text of warnings to the user if various error conditions are detected
+[[ warnings ]]
+
+warningGenericTitle = 有些東西不對勁！
+warningUpgradeFirefoxTitle = 升級 Firefox 後再繼續使用！
+warningUpgradeFirefoxDetail = 需要使用最新版本的 Firefox 才能使用 Test Pilot。<a>請升級 Firefox</a> 開始使用。
+warningHttpsRequiredTitle = 必須使用 HTTPS！
+warningHttpsRequiredDetail = 必須透過 HTTPS 通訊協定存取 Test Pilot。請參考<a>文件</a>當中的詳細資訊。
+warningMissingPrefTitle = 正在開發 Test Pilot 嗎？