Add support for auto generating metrics

[mknowlton89]

Features and Changes

This PR introduces logic to automatically generate metrics for our various event trackers. In it's current state, this only supports Segment & Rudderstack. Once I get some initial feedback on the overall structure of the PR, I'll expand to include additional event trackers, specifically GA4, Firebase, and Amplitude, with more to follow in separate PRs.

To do this, when an organization adds a datasource via an event tracker supported above, they'll see an option on the NewDataSourceForm to look up what metrics we can generate for them automatically. I built a method within SqlIntegration that adds a supportsAutoGeneratedMetrics property to the integration's getDataSourceProperties so we can easily determine on the front and back end if the particular data source supports auto generating metrics.

isAutoGeneratingMetricsSupported(): boolean {
    const supportedEventTrackers: SchemaFormat[] = ["segment", "rudderstack"];

    if (
      this.settings.schemaFormat &&
      supportedEventTrackers.includes(this.settings.schemaFormat)
    ) {
      return true;
    }
    return false;
  }

This hits a new endpoint /datasource/:datasourceId/auto-metrics - This endpoint looks up the unique events that are tracked by the event tracker. This is possible since the event trackers all have a table that lists all of the events that they track. In the case of Segment & Rudderstack this lives on the tracks table.

The endpoint returns a list of events that we think we can generate, along with sql queries for count and binomial metrics derived from the tracked events.

Here, the user can preview the underlying SQL that would eventually power these metrics, and opt to create binomial and count metrics for each tracked event. I've added some tooltips throughout the table to provide some insight.

Then, when the user clicks "Save" if there are any metrics they've indicated they want us to create for them, (via the toggle state) we'll kick off an async job (createAutoGeneratedMetrics) that creates those metrics.

There has also been some logic defined to pluralize the event names (E.G. for an Order Placed event via Segment, the count metric associated with that would be Orders Placed.)

This is currently handled via a pretty simple map. Eventually, we could leverage ChatGPT for this. If the map is unable to map an event to a pluralized version, it will simply return a displayName of Count of {event}.

The seed for the pluralization map came from Segment's documentation here - https://segment.com/docs/connections/spec/ecommerce/v2/. This map will likely be a bit of a living document as we expand support.

Testing

• Set up a Segment destination with BigQuery and add that data source to GrowthBook and confirm all tracked events are displayed and created correctly, with the correct underlying sql query.
• Ensure the above works for revenue, count. and binomial metrics.

Screenshots

[github-actions]

Your preview environment pr-1283-bttf has been deployed.

Preview environment endpoints are available at:

• https://growthbook-3000-pr-1283-bttf.growthbook.okteto.dev
• https://growthbook-3100-pr-1283-bttf.growthbook.okteto.dev

[jdorn]

This data structure is a little weird. It's mixing form state (e.g. createBinomialFromEvent) with data from the query (e.g. lastTrackedAt). Seems like that should be 2 separate data structures.

[mknowlton89]

Updated this to be a bit more future proof and to separate the concerns.

Currently, this data takes the following shape:

export type TrackedEventData = {
  event: string;
  displayName: string;
  count: number;
  hasUserId: boolean;
  lastTrackedAt: Date;
  metricsToCreate: {
    name: string;
    sql: string;
    type: MetricType;
    shouldCreate?: boolean;
  }[];
};

[jdorn]

This method also feels duplicative with the new generateTableName method.

[mknowlton89]

@jdorn Just pushed up a big refactor of generateTableName that handles the logic here as well, so we've now gotten rid of the getInformationSchemaTableFromClause method.

All of the changes were done in this commit.