feat: add telemetry documentation

README.md

@@ -24,7 +24,11 @@ A longer repository description.
 
 ## Documentation
 
-[Insert link to documentation]() or expand with Install, Build, Usage sections.
+### Telemetry
+
+[Telemetry collection policy](./docs/telemetry/CollectionPolicy.md)
+[Privacy Policy](./docs/telemetry/PrivacyPolicy.md)
+[FAQs](./docs/telemetry/FAQs.md)
 
 ## Lead Maintainer
 

docs/telemetry/CollectedData.template.md

@@ -0,0 +1,36 @@
+# Collected telemetry data
+
+Copied from https://github.com/ipfs-shipyard/ignite-metrics/blob/feature/metric-library/129/docs/telemetry/CollectedData.template.md
+
+<!--
+This is the default for projects using the ignite-metrics library. Each project's own `./docs/telemetry/CollectedData.md` file should contain very specific information under each of the following sections:
+
+* 'What metrics data DO we collect'
+
+-->
+## What metrics data DON'T we collect
+
+We do not collect any of the below listed items in our metric data:
+
+1. User identifiable data (names, email addresses, aliases, handles, etc.)
+2. CIDs
+3. IP addresses
+
+Countly also calls out how they protect your data at https://support.count.ly/hc/en-us/articles/360037501372-Security-Privacy-and-Access-FAQ
+
+## What metrics data DO we collect
+
+As a general rule, we collect only application data; no user data. Some examples of application data we collect are:
+
+| Metric data name | Metric feature name | What is this data used for |
+|:----------------:|---------------------|----------------------------|
+| Fill in          | Fill in             | Fill in                    |
+|                  |                     |                            |
+|                  |                     |                            |
+
+
+## Other related documents
+
+* https://github.com/ipfs-shipyard/ignite-metrics/blob/feature/metric-library/129/docs/telemetry/CollectionPolicy.md
+* https://github.com/ipfs-shipyard/ignite-metrics/blob/feature/metric-library/129/docs/telemetry/PrivacyPolicy.md
+* https://github.com/ipfs-shipyard/ignite-metrics/blob/feature/metric-library/129/docs/telemetry/FAQs.md

docs/telemetry/CollectionPolicy.md

@@ -0,0 +1,125 @@
+# Telemetry collection policy
+
+This document is the source of truth for our telemetry policies and practices across all Ignite team projects. The discussion held at https://github.com/ipfs/ipfs-gui/issues/125 is the origin and inspiration for this library and all documents under `./docs/telemetry`.
+
+The initial list of projects we're adding telemetry to was originally discussed at https://github.com/ipfs/ipfs-gui/issues/105.
+
+For an up to date list of projects we own that follow this policy, please see:
+
+* https://github.com/ipfs/ipfs-gui#primary-projects
+* https://github.com/ipfs/ipfs-gui#other-ipfs-gui--tools-owned-projects
+
+## History
+
+Before 2022Q4, telemetry was only implemented in https://github.com/ipfs/ipfs-webui and https://github.com/ipfs/ipfs-desktop, using our [self-hosted countly](https://count.ly/community-edition) server. The telemetry implemented within ipfs-webui and ipfs-desktop was opt-in only. i.e. metrics data was sent only if a user specifically opted-in to collecting metrics.
+
+Our other projects did not do telemetry or any metrics data collection and did not mention telemetry or metrics at all. The only exception is https://github.com/ipfs/ipfs-companion where we have had a [privacy policy since 2019-02-15](https://github.com/ipfs/ipfs-companion/blob/main/PRIVACY-POLICY.md). The privacy policy in ipfs-companion indicates that no data is collected whatsoever, and that will be changing.
+
+## Metrics collection expectations
+
+It's important to clarify that our intention for telemetry with our projects is solely to prioritize our work efforts in bringing us all to the web3 mecca. This includes not violating our users expectations of privacy nor anonymity.
+
+### Application data vs User data
+
+Our telemetry is solely focused on the collection of application data, not user data.
+
+#### What is Application data?
+
+Application data only contains information about events, modules, performance, and use of our applications, not the users themselves.
+
+#### What is User data?
+
+User data can vary across industries and applications, but is generally data that is used to classify or categorize users, typically for targeting classes or categories of users via campaigns or other marketing methods for various purposes. Think demographics, emails, physical addresses, personal preferences, and marketing.
+
+As a general rule, we do not collect user or personal data. See our [Privacy Policy](./PrivacyPolicy.md) for more details.
+
+### Metric features and their groupings
+
+The metric features in the first column name and their descriptions come directly from [countly's support website](https://support.count.ly/hc/en-us/articles/360037441932-Web-analytics-JavaScript-#features-for-consent).
+
+| Metric feature name | Consent method | Metric feature group name | Metric feature description                                                     | What do we use this metric for?                                                          |
+|:-------------------:|----------------|---------------------------|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|
+| sessions            | Opt-out        | Minimal                   | tracks when, how often, and how long users use your website                    | For determining unique daily/weekly/monthly active users in order to prioritize projects |
+| views               | Opt-out        | Minimal                   | allows for the views/pages accessed by a user to be tracked                    | For tracking application pages and component views only                    |
+| events              | Opt-out        | Minimal                   | allows your events to be sent to the server                                    | Timed and one-off application events such as app_start, app_close, function_used, etc. |
+| crashes             | Opt-in         | Performance               | allows JavaScript errors to be tracked                                         | Not used yet - Eventually for bug fixes and debugging |
+| apm                 | Opt-in         | Performance               | allows performance tracking of application by recording traces                 | Not used yet - Eventually for bug fixes and debugging |
+| scrolls             | Opt-in         | UX                        | allows a user’s scrolls to be tracked on the heatmap                           | Not used yet - Possibly for performance and UX optimizations |
+| clicks              | Opt-in         | UX                        | allows a user’s clicks and link clicks to be tracked on the heatmap            | Not used yet - Possibly for performance and UX optimizations |
+| forms               | Opt-in         | UX                        | allows a user’s form submissions to be tracked                                 | Not used yet - Possibly for bug fixes and debugging, performance and UX optimizations |
+| star-rating         | Opt-in         | Feedback                  | allows user rating and feedback tracking through rating widgets                | Not used yet - Possibly for getting user feedback more directly through our applications |
+| feedback            | Opt-in         | Feedback                  | allows survey, nps rating and feedback tracking through feedback widgets       | Not used yet - Possibly for getting user feedback more directly through our applications |
+| location            | Opt-in         | Location                  | allows a user’s location (country, city area) to be recorded                   | N/A - we have no current need for collecting a user's location data. |
+| attribution         | Opt-in         | N/A                       | allows the campaign from which a user came to be tracked                       | N/A - we do not have any plans to implement or utilize user campaigns |
+| users               | Opt-in         | N/A                       | allows user information, including custom properties, to be collected/provided | N/A - we will not collect user information |
+
+The code implementing these groupings can be found [here](./../../src/MetricsProvider.ts#L17-L30)
+
+### What do the consent toggling UIs look like?
+
+Consent toggling has two distinct views depending on the type of application implementing telemetry using this library: Projects without a settings page, and Projects with settings page(s).
+
+#### Projects without its own settings page(s)
+
+In this case, we would follow the patterns discussed in https://github.com/ipfs/public-gateway-checker/issues/340 and https://github.com/ipfs-shipyard/ignite-metrics/issues/36.
+
+There will be a floating icon in the bottom left of the page: ![example telemetry settings modal toggle for projects withour a settings page](./images/telemetry-modal-toggle.png)
+
+When clicked, that icon will expand into a modal that will allow you to enable/disable the collection of groups of features as defined above in the "Metric features and their groupings" section.
+
+Single toggle Example: ![example modal containing a single consent toggle](./images/single-toggle-modal.png)
+
+Multi toggle Example: ![example modal containing multiple consent toggles](./images/multi-toggle-modal.png)
+
+#### Projects with its own settings page(s)
+
+The consent should conform to the UX expectations of the library it's being implemented in.
+
+##### IPFS-companion
+
+It should look like the rest of the settings in the application.
+
+Potential UI with multiple toggles: ![example telemetry multi-toggle UI for settings page in ipfs-companion](./images/settings-page-multi-toggle.png)
+
+Example with single toggle (PR at https://github.com/ipfs/ipfs-companion/pull/1117): ![example telemetry single-toggle UI for settings page in ipfs-companion](./images/settings-page-single-toggle.png)
+
+##### IPFS-Webui & IPFS-Desktop
+
+Analytics toggling and consent notifications are already implemented in the desktop app and webui.
+
+Example: ![example telemetry toggle UI for settings page in ipfs-webui and ipfs-desktop](./images/webui-settings-page-single-toggle.png)
+
+### Informing users of changes to this policy
+
+We will not display notifications to users regarding telemetry changes, but users can follow release-notes as well as changes to this library, it's telemetry documentation (`./docs/telemetry/*`), and projects implementing this library to stay up to date on our telemetry policy changes.
+
+## SOPs (Standard Operating Procedures)
+
+### Adding telemetry collection to a project without telemetry collection
+
+1. Copy the [CollectedData.template](./CollectedData.template.md) file to your project at `./docs/telemetry/CollectedData.md`
+2. Implement the library according to your project's needs
+3. Update your `CollectedData.md` file specifying exactly which metrics your collecting, and when they're collected.
+
+### Changing what metrics data is collected by a project
+
+When you add/remove metrics data that is collected by a project, you MUST follow this checklist:
+
+1. Fist ask yourself: Are you trying to collect user-identifiable, or other user data at all? STOP.
+   * You must get legal approval
+   * You must update the following documents:
+      * `./PrivacyPolicy.md` - responsibility of legal write a new doc after meeting with you and discussing what your plans are. You would then update the `PrivacyPolicy.md` in this repo.
+      * `./CollectionPolicy.md` - responsibility of you to update according to the changes you're making.
+      * `./FAQs.md` - responsibility of you to update clarifying exactly why we're now collecting user data.
+1. Modify the project's `./docs/telemetry/CollectedData.md` file.
+   * Ensure the table under `What metrics data DO we collect` is accurate and up to date
+
+
+### Changing a metric feature's grouping or opt-in/opt-out policy
+
+When you move a metric feature to/from a particular feature group, or that group changes its consent method, you must update:
+
+1. The table above at `Metric features and their groupings`
+2. The code at https://github.com/ipfs-shipyard/ignite-metrics/blob/f4431f59ce363c898affd4171eed979ede24e514/src/CountlyMetrics.ts#L10-L16
+3. Possibly the code at https://github.com/ipfs-shipyard/ignite-metrics/blob/f4431f59ce363c898affd4171eed979ede24e514/src/CountlyMetrics.ts#L27-L36
+

docs/telemetry/FAQs.md

@@ -0,0 +1,53 @@
+# FAQs - Frequently asked questions
+
+## Why are you collecting metrics?
+
+We are collecting metrics for all of our projects to assist with prioritizing work and provide proof that our work is valuable. Our prior state had the following issues:
+
+1. Opt-in only metrics for webui and desktop did not send metrics data to our countly server
+    1. We have no insight into the total number of users in our applications
+    2. We had no insight into the percentage of users who declined or accepted metrics
+    3. If a user did not accept or decline metrics, usage was not collected
+2. Lack of singular metrics collection library
+    1. Consent and telemetry patterns were non-existent
+    2. Privacy policy and metrics collection would have needed to be implemented for each library separately.
+    3. Analytics across projects does not allow for normalized comparisons of usage.
+3. Missing telemetry
+    1. We have no insight into the usage of a majority of our projects
+        1. It is understood (not confirmed with data) that two of our top three projects are ipfs-webui & ipfs-desktop, and they had ****some**** telemetry in place.
+    2. We have three separate usecases for ipfs-webui, and are only tracking two of them:
+        1. [webui.ipfs.io](http://webui.ipfs.io) & local - tracked
+        2. webui usage within ipfs-desktop - tracked as desktop usage
+        3. webui bundled with kubo - not tracked
+
+## What data is collected?
+
+In order to standardize the documentation of metrics collection, we will keep a file in each project's repository that
+specifies all metrics data collected by that project. The file will be located at `./docs/telemetry/CollectedData.md`. These documents will provide answers to the following bullet-points:
+
+* This is what will be collected
+* This is what won't be collected
+* Why we are collecting each piece of metric data.
+
+You can read our [Privacy Policy](./PrivacyPolicy.md) and [Collection Policy](./CollectionPolicy.md) for more generic details.
+
+## As an existing user, how am I notified about changes to the data that is collected?
+
+You can follow updates to the metrics we collect for each repo in that repo's release notes. You can find changes to our collection policy in our [Collection Policy](./CollectionPolicy.md) document. The document at `./docs/telemetry/CollectedData.md` in each relevant repo will be updated when any metrics collection changes are made.
+
+## How do I opt-out of metrics?
+
+For apps with settings pages, such as ipfs-webui, ipfs-desktop, and ipfs-companion, you can opt out in each of the applications' setting pages. For apps without settings pages (or single page applications where settings pages don't make sense), we will be following the [patterns set by the public-gateway-checker project](https://github.com/ipfs/public-gateway-checker/issues/340#issuecomment-1371410214).
+
+The basic implementation is a floating icon on the bottom left of the page that displays a modal when clicked. The displayed modal allows the enabling/disabling of metrics collection for our different groupings.
+
+For public-gateway-checker changes, see the following PRs and issues:
+
+* https://github.com/ipfs/public-gateway-checker/pull/309
+* https://github.com/ipfs/public-gateway-checker/issues/340
+* https://github.com/ipfs/public-gateway-checker/issues/341
+* https://github.com/ipfs/public-gateway-checker/issues/342
+
+## Can I turn telemetry completely off?
+
+Yes. If you toggle all telemetry groups off (i.e. opt-out), no telemetry will be collected. The one caveat is that we send a single request when you opt-out of metrics collection.

docs/telemetry/PrivacyPolicy.md

@@ -0,0 +1,5 @@
+# Privacy Policy
+
+To be filled in by legal team based off of meeting documented at https://pl-strflt.notion.site/2023-01-05-Privacy-Policy-Updates-9bef84008b52449496ad5b806d529dfe
+
+Until then, for all project's other than ipfs-companion (which is the only project we own with a current [privacy policy](https://github.com/ipfs/ipfs-companion/blob/main/PRIVACY-POLICY.md)), you can reference [ipns://protocol.ai/legal/#privacy-policy](ipns://protocol.ai/legal/#privacy-policy)

src/MetricsProvider.ts

@@ -1,5 +1,5 @@
 import { COUNTLY_API_URL } from './config'
-import type { consentTypes, eventTypes } from 'countly-sdk-web'
+import type { consentTypes, metricFeatures } from 'countly-sdk-web'
 import Countly from 'countly-sdk-web'
 
 interface MetricsProviderConstructorOptions {
@@ -9,21 +9,24 @@ interface MetricsProviderConstructorOptions {
 }
 
 export default class MetricsProvider {
-  private readonly minimalEvents: eventTypes[] = ['sessions', 'views']
-  private readonly marketingEvents: eventTypes[] = ['attribution', 'users', 'location']
-  private readonly trackingEvents: eventTypes[] = ['events', 'crashes', 'apm']
-  private readonly performanceEvents: eventTypes[] = ['scrolls', 'clicks', 'forms', 'star-rating', 'feedback']
-  private readonly groupedFeatures: Record<consentTypes, eventTypes[]> = {
+  private readonly minimalFeatures: metricFeatures[] = ['sessions', 'views', 'events']
+  private readonly performanceFeatures: metricFeatures[] = ['crashes', 'apm']
+  private readonly uxFeatures: metricFeatures[] = ['scrolls', 'clicks', 'forms']
+  private readonly feedbackFeatures: metricFeatures[] = ['star-rating', 'feedback']
+  private readonly locationFeatures: metricFeatures[] = ['location']
+  private readonly groupedFeatures: Record<consentTypes, metricFeatures[]> = {
     all: [
-      ...this.marketingEvents,
-      ...this.minimalEvents,
-      ...this.performanceEvents,
-      ...this.trackingEvents
+      ...this.minimalFeatures,
+      ...this.locationFeatures,
+      ...this.performanceFeatures,
+      ...this.uxFeatures,
+      ...this.feedbackFeatures
     ],
-    marketing: this.marketingEvents,
-    minimal: this.minimalEvents,
-    performance: this.performanceEvents,
-    tracking: this.trackingEvents
+    minimal: this.minimalFeatures,
+    performance: this.performanceFeatures,
+    ux: this.uxFeatures,
+    feedback: this.feedbackFeatures,
+    location: this.locationFeatures
   }
 
   private sessionStarted: boolean = false
@@ -78,7 +81,7 @@ export default class MetricsProvider {
     this.metricsService.remove_consent(consent, true)
   }
 
-  checkConsent (consent: consentTypes | eventTypes) {
+  checkConsent (consent: consentTypes | metricFeatures) {
     if (consent in this.groupedFeatures) {
       return this.groupedFeatures[consent as consentTypes].every(this.metricsService.check_consent)
     }

src/types/countly.d.ts

@@ -18,15 +18,15 @@ declare module 'countly-sdk-web' {
     segmentation?: Segments
   }
 
-  export type eventTypes = 'apm' | 'attribution' | 'clicks' | 'crashes' | 'events' | 'feedback' | 'forms' |
+  export type metricFeatures = 'apm' | 'attribution' | 'clicks' | 'crashes' | 'events' | 'feedback' | 'forms' |
   'location' | 'scrolls' | 'sessions' | 'star-rating' | 'users' | 'views'
-  export type consentTypes = 'all' | 'minimal' | 'marketing' | 'tracking' | 'performance'
+  export type consentTypes = 'all' | 'minimal' | 'performance' | 'ux' | 'feedback' | 'location'
   type Segments = Record<string, string>
   type IgnoreList = Array<string | RegExp>
   type CountlyEventQueueItem = [string, CountlyEventData] | [eventName: string, key: string] | [eventName: string]
   interface CountlyWebSdk {
-    group_features: (arg0: Record<consentTypes, eventTypes[]>) => unknown
-    check_consent: (consentFeature: eventTypes | consentTypes) => boolean
+    group_features: (arg0: Record<consentTypes, metricFeatures[]>) => unknown
+    check_consent: (consentFeature: metricFeatures | consentTypes) => boolean
     add_consent: (consentFeature: consentTypes | consentTypes[]) => void
     remove_consent: (consentFeature: consentTypes | consentTypes[], enforceConsentUpdate?: boolean) => void
     require_consent: boolean