Official Metrics

[jdorn]

Overview

There should be a distinction between "official" metrics (version controlled, vetted by data team, tested, etc.) and "ad-hoc" metrics which anyone can create.

Official metrics are locked down and cannot be edited from the GrowthBook UI. Changes must be made wherever they are being version controlled instead (e.g. a git repo). Official metrics should also be visually distinct in the UI (e.g. with a badge) so people know they can trust them.

Screenshots

"Official" badge with tooltip added everywhere we render the metric name.

Managed By

There are 3 different places people manage their GrowthBook metrics:

1. In a config.yml file (self-hosted only)
2. In the GrowthBook UI
3. Via the REST API

This PR adds a new managedBy property to metrics, fact tables, fact filters, and fact metrics. It has 3 possible values:

• "" (empty string) - can be managed via both the UI and REST API
• "api" - can only be managed via the REST API
• "config" - can only be managed via a config.yml file. This is only for regular metrics, not fact tables/filters/metrics for now.

By default, we will infer this value. Anything defined in config.yml will get a value of "config". Everything else will get a value of "" (empty string). When creating/updating resources via the REST API, you can opt-in to setting this field to "api" to lock down the UI.

For now, we have simple logic to determine official status. If the managedBy property is empty, it is "ad-hoc", otherwise it is considered "official". We may make this more explicit in the future.

Config.yml Changes

There is a new environment variable ALLOW_CREATE_METRICS. When set to true, new metrics can be created via the UI even while config.yml is being used. This creates 2 classes of metrics - Official ones that can only be changed via config.yml and "ad-hoc" metrics that can be managed via the UI.

GitHub Metrics Guide

New guide in docs about how to store fact table and metric definitions in a GitHub repo and sync them to GrowthBook automatically via the /bulk-import/facts REST endpoint.

Testing

• Use bulk endpoint to create Official fact tables, filters, and metrics, verify they cannot be edited in the UI
• Creating new fact tables, filters, and metrics and updating them in the UI continues to work
• Read access filters for projects are working for fact tables, fact metrics, and regular metrics
• Create new regular metrics and updating them in the UI continues to work
• Create a metric via REST API and mark it as Official, verify it cannot be edited in the UI
• When using config.yml, regular metrics marked as Official and cannot be edited in the UI
• When using config.yml, you can create new metrics and edit them in the UI (only when ALLOW_CREATE_METRICS env var is true)
• Official badge shows up throughout the UI
• The steps in the GitHub integration guide work correctly

Future Work

• Allow the same mixing and matching between config.yml, the API, and the UI for Segments and Dimensions
• Relax config.yml strictness for organization settings. For example, if 3 org settings are defined in config.yml, block those 3 from being edited in the UI, but still allow editing other settings.
• Helper scripts for transforming from dbt semantic layer to the GrowthBook bulk import format for use in a GitHub Action
• Easy way to clean up old "Official" metrics that are no longer being synced
• Support additional metadata so we can link to the git repo/file from within GrowthBook

[github-actions]

Your preview environment pr-2007-bttf has been deployed.

Preview environment endpoints are available at:

• https://growthbook-3000-pr-2007-bttf.growthbook.okteto.dev
• https://growthbook-3100-pr-2007-bttf.growthbook.okteto.dev

[github-actions]

Deploy preview for docs ready!

✅ Preview
https://docs-2glp5lpqh-growthbook.vercel.app

Built with commit a314dd9.
This pull request is being automatically deployed with vercel-action

[itsgrimetime]

just noticed this - happy to help test it out whenever it's ready!