Fix unnecessary activation fact metric data retrieval (and two minor FE bugs)

[lukesonnet]

Features and Changes

Fact metrics currently have hidden logic returning conversion windows in a few places where they shouldn't if a conversion window has not been selected.

Back end fix (SqlIntegration.ts):

• Pass ignoreConversionWindow through to get metric end filter for activation metric (Closes [Bug] Conversion Delay settings for Fact Tables Metrics working incorrectly #1882). Without this change we inefficiently get (by default), 3 extra days of data that we later correctly filter out in the __distinctUsers CTE. Now with this PR we just never get those additional 3 extra days of data.

FE fixes:

• Update the mouseover in the results table to not show a window for a metric when one doesn't exist
• We should update the old view in the config tab showing the overrides to have "none" when there is no conversion window!

Testing

Some in-app testing with fact metrics that do and do not have conversion windows, showing the correct behavior and generated SQL.

Screenshots

Metric name mouse over before, even though this metric has conversion window == off

Metric name mouse over after:

old page config tab before, even though number of purchases has no conversion window:

old page config tab after:

Back-end SQL before:

__experimentExposures AS (
    -- Viewed Experiment
    SELECT
      e.user_id as user_id,
      cast(e.variation_id as varchar) as variation,
      e.timestamp as timestamp
    FROM
      __rawExperiment e
    WHERE
      e.experiment_id = 'checkout-layout'
      AND e.timestamp >= '2021-11-22 20:23:00'
      AND e.timestamp <= '2023-10-17 17:04:00'
  ),
  __activationMetric as ( -- Metric (any order)
    SELECT
      user_id as user_id,
      1 as value,
      m.timestamp as timestamp
    FROM
      (
        SELECT
          userid as user_id,
          timestamp as timestamp,
          qty,
          amount
        FROM
          orders
      ) m
    WHERE
      m.timestamp >= '2021-11-22 20:23:00'
      AND m.timestamp <= '2023-10-20 17:04:00'
  ),
  __experimentUnits AS (
    -- One row per user
    SELECT
      e.user_id AS user_id,
      (
        CASE
          WHEN count(distinct e.variation) > 1 THEN '__multiple__'
          ELSE max(e.variation)
        END
      ) AS variation,
      MIN(e.timestamp) AS first_exposure_timestamp,
      MIN(
        (
          CASE
            WHEN a.timestamp >= e.timestamp THEN a.timestamp
            ELSE NULL
          END
        )
      ) AS first_activation_timestamp
    FROM
      __experimentExposures e
      LEFT JOIN __activationMetric a ON (a.user_id = e.user_id)
    GROUP BY
      e.user_id
  ),
  __distinctUsers AS (
    SELECT
      user_id,
      cast('All' as varchar) AS dimension,
      variation,
      first_activation_timestamp AS timestamp,
      date_trunc('day', first_exposure_timestamp) AS first_exposure_date,
      first_exposure_timestamp AS preexposure_end,
      first_exposure_timestamp - INTERVAL '336 hours' AS preexposure_start
    FROM
      __experimentUnits
    WHERE
      first_activation_timestamp IS NOT NULL
      AND first_activation_timestamp <= '2023-10-17 17:04:00'
  ),

Notice how we filter out first_activation_timestamp <= '2023-10-17 17:04:00' but we're nonsensically getting data in the activation metric CTE with m.timestamp <= '2023-10-20 17:04:00'

Now after this change this same SQL snippet is:

__experimentExposures AS (
  -- Viewed Experiment
  SELECT
    e.user_id as user_id,
    cast(e.variation_id as varchar) as variation,
    e.timestamp as timestamp
  FROM
    __rawExperiment e
  WHERE
    e.experiment_id = 'checkout-layout'
    AND e.timestamp >= '2021-11-22 20:23:00'
    AND e.timestamp <= '2023-10-17 17:04:00'
),
__activationMetric as ( -- Metric (any order)
  SELECT
    user_id as user_id,
    1 as value,
    m.timestamp as timestamp
  FROM
    (
      SELECT
        userid as user_id,
        timestamp as timestamp,
        qty,
        amount
      FROM
        orders
    ) m
  WHERE
    m.timestamp >= '2021-11-22 20:23:00'
    AND m.timestamp <= '2023-10-17 17:04:00'
),
__experimentUnits AS (
  -- One row per user
  SELECT
    e.user_id AS user_id,
    (
      CASE
        WHEN count(distinct e.variation) > 1 THEN '__multiple__'
        ELSE max(e.variation)
      END
    ) AS variation,
    MIN(e.timestamp) AS first_exposure_timestamp,
    MIN(
      (
        CASE
          WHEN a.timestamp >= e.timestamp THEN a.timestamp
          ELSE NULL
        END
      )
    ) AS first_activation_timestamp
  FROM
    __experimentExposures e
    LEFT JOIN __activationMetric a ON (a.user_id = e.user_id)
  GROUP BY
    e.user_id
),
__distinctUsers AS (
  SELECT
    user_id,
    cast('All' as varchar) AS dimension,
    variation,
    first_activation_timestamp AS timestamp,
    date_trunc('day', first_exposure_timestamp) AS first_exposure_date,
    first_exposure_timestamp AS preexposure_end,
    first_exposure_timestamp - INTERVAL '336 hours' AS preexposure_start
  FROM
    __experimentUnits
  WHERE
    first_activation_timestamp IS NOT NULL
    AND first_activation_timestamp <= '2023-10-17 17:04:00'
),

[github-actions]

Your preview environment pr-1922-bttf has been deployed.

Preview environment endpoints are available at:

• https://growthbook-3000-pr-1922-bttf.growthbook.okteto.dev
• https://growthbook-3100-pr-1922-bttf.growthbook.okteto.dev