Modify cascading window logic for excluding in-progress conversions #1921
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Your preview environment pr-1921-bttf has been deployed. Preview environment endpoints are available at: |
jdorn
approved these changes
Dec 6, 2023
bryce-fitzsimons
pushed a commit
that referenced
this pull request
Dec 12, 2023
…1921) * Modify cascading window logic for excluding in-progress conversions * Move fn * Test
bryce-fitzsimons
added a commit
that referenced
this pull request
Jan 12, 2024
* initial, updating existing types and methods with new bucket stuff * moar types * simplify targeting ui, remove stickyBucketing var * add minBucketVersion, add reseed option for new phases * org setting for sticky bucketing, premium feature, organize settings page * lock SB switch based if !hasCommercialFeature * working on rollout recommendation logic * variation blocking ui, wip * tweak logic kinda * tweak * re-add disableStickyBucketing * ui wip * ui wip * ui wip * ui wip * ui wip * ui wip * ui wip * ui wip * Support ability to set role with SCIM request. (#1779) * Support ability to set role with SCIM request. * Modify cascading window logic for excluding in-progress conversions (#1921) * Modify cascading window logic for excluding in-progress conversions * Move fn * Test * Fix unnecessary activation fact metric data retrieval (and two minor FE bugs) (#1922) * Update conversion window logic for fact metrics * CLean up language * More proximate fix * Pre-compute safe dimension slices (#1873) * Add reliable dimensions query * Modal mostly working * Get save working * lint and such * merge origin main * Wire auto dimensions through to units and traffic query * remove dimensions for traffic * Fix SQL * Bryce comments * Move to kebab & add context * rename && remove ff * remove FF * Table version * ui/code tweaks * Add running language * remove console * Remove metadata field * Additional renaming * Hide entry with no dimensions * Move storing specified values directly to exposure query * TS * Small updates * Fix type issue * Rename && add lookback selector * Cleanup * push pixels * Add tracking * Blank out button for automatic dimensions * Ensure cast to string before string comparison --------- Co-authored-by: Bryce <bryce1@gmail.com> * Add new rollback workflow (#1867) * Fix bug - metric capping cannot be updated on old metrics (#1930) * Add Random On-Screen Celebration Service (#1826) * Add on-screen confetti celebration service to app. Launch experiment consumes it and will fire 20% of the time. * Allow orgs to define role when creating and updating groups via SCIM (#1909) * Adds support for setting team/group global role when creating a team or updating a team via SCIM endpoints * SDK compatibility versioning (#1893) * initial: new sdkVersion field, sdkid/form UI * sdk-versioning package, getCurrentVersion method * getCapabilities * payload scrubbing. mostly feature complete * BE resolveJsonModule * update scrubbing logic to use pick * refactor json * isOutdated check * add all sdks, fill in versions/capabilities * tweaks * get minimal safe capabilities when multi-language, add tests * basic gating in SDK connection form * more retrofitting * more retrofitting * address feedback * address feedback * lint * use today's capabilities as a snapshot for default sdk version * small fixes * clean up scrubbing, add tests * fix sdkconnection post endpoint * fix model option version * api types * fix test * multi-stage scrubbing, update legacy connection logic, tests * update php for looseUnmarshalling * address feedback * log payload deltas instead of scrubbing until confident * use logger * log capabilities * use isEqual for better payload scrub comparison (#1935) * ui wip * resize release plan summary. probably redo this later... * fix hook deps * ui wip * block save when no changes, cleanup old code * more clarity around SB SDK support * refactor models based on discussions (no blockedVariations, new bool) * wip multi-stage modal * wip multi-stage modal * wip multi-stage modal * wip multi-stage modal * wip multi-stage modal * wip multi-stage modal * wip multi-stage modal * wip multi-stage modal * confirm check * add warnings while making changes * cleanup * cleanup * fine tune ui * remove blocked from variation (add later) * clean up modal ui * payload scrubbing * remove reseed from phases modals, fix any type * attempt saved group change detection * lint * move warning to release plan page, don't warn when using recommended, ui tweaks * change targeting language, ui tweaks * fix saved group comparison, fix release plan selector, adjust ui * wip SB onboarding / toggle * sticky bucketing onboarding and toggle gating + warnings * phases: looking for targeting? -> make changes * refactor * retool warnings * address some feedback * adjust settings SB toggle * fix saved group restrictive ANY calc * fix saved group restrictive ANY calc again * make changes to choices * other fix * tweaks * calculate risk level per option * phase and warning stuff * ui tweak, edit phases show/hide button logic * adjust warning language * move same-phase-everyone into default release plan options * add SB keys to payload scrubber * implement some suggestions * fix legacy webhook capabilities * fix help text * fix help text again * update types * ui tweaks * remove onboarding CTAs when sdk issues detected, point to docs * gate fallback attributes by org setting * UI tweaks for sticky bucket onboarding and Make Changes modal * minor ui tweaks * Sticky bucketing docs (#2017) * talk about new stuff in experiment-configuration, fix proxy docs (unrelated) * sb doc stub * sb doc stub * sb doc wip * change slug * sb doc wip * sb doc wip * sb doc wip * sdk docs * Language * de-emphasize fallbackAttribute in examples * Update sticky bucket docs page with fallback attribute example and helper image for SDK connections --------- Co-authored-by: Luke Sonnet <luke.sonnet@gmail.com> Co-authored-by: Jeremy Dorn <jeremy@jeremydorn.com> --------- Co-authored-by: Michael Knowlton <michael@growthbook.io> Co-authored-by: Luke Sonnet <lukesonnet@users.noreply.github.com> Co-authored-by: Adnan Chowdhury <2374625+bttf@users.noreply.github.com> Co-authored-by: Jeremy Dorn <jeremy@jeremydorn.com> Co-authored-by: Luke Sonnet <luke.sonnet@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Features and Changes
Users can currently choose to 'Exclude in-progress conversions' to drop users that have not had the full time window to convert. However, we don't properly calculate that "full time window". Currently, we just use the metric (or denominator metric if a funnel or ratio metric) conversion window, but we need to take:
We also need to always add the activation metric (if specified) window to the value of the above since activation metrics start their own conversion windows. So if a user has 72 hours to activate and a ratio with (24 hours numerator, 1 hours denominator), we need to give them a total of MAX(24, 1) + 72 hours to convert.
Closes #1844
Also, this followed work in #1880, which, while addressing some issues, didn't capture the full complexity of the above flow.
Testing
Added unit tests
Tested in-app:
Exclude In-Progress ConversionsResults as expected:
Adding the activation metric adds 72 hours to all of those above.\
Note: the metric filter in
getMetricEndremains a little inefficient because it doesn't know that it only needs the max for ratio metrics rather than the sum for funnel metrics, but it isn't dropping relevant data, just is collecting slightly too much data.