ref(experiments): Default useExperiment reportExposure to false#114217
Merged
ref(experiments): Default useExperiment reportExposure to false#114217
Conversation
The backend org serializer auto-logs exposure for every experiment-mode flagpole flag whenever an org is loaded, so the FE default of true was producing redundant exposure POSTs from any call site that just wanted to read the assignment. Flip the default to false and require call sites to opt in at the single point that represents the user actually seeing the experiment. The synchronous browser-side Amplitude group property write still fires from that opt-in site to win the race against events on the same mount. Onboarding flips onboarding.tsx to reportExposure: true; the other SCM onboarding consumers fall through to the new false default.
github-actions
Bot
added
the
Scope: Frontend
Drop the reference to the backend serializer auto-logging exposure; that path is going away. Frame the FE as the source of truth so each call site has to opt in intentionally.
Replace the double-counting framing with the underlying concern: shared components that consume useExperiment can render neither the control nor the active variant, and reporting exposure there misattributes the user as having seen the experiment when they have not.
Drop the TestComponent default of true, which existed only to keep existing exposure tests passing after the hook default flipped to false. The default hid the new behavior from the test file. Each test that exercises exposure now passes the prop explicitly so the spec reflects how consumers must use the hook.
Drop the shared-component aside; the rule about opting in at the variant-rendering site already covers it.
ryan953
reviewed
Apr 30, 2026
|
|
||
| export function useExperiment(options: UseExperimentOptions): UseExperimentResult { | ||
| const {feature, reportExposure = true} = options; | ||
| const {feature, reportExposure = false} = options; |
Member
There was a problem hiding this comment.
What if there was no default value: so people are forced to learn about this. I've found in previous experiences that exposure is not something people intuitively think about or add even when it's needed.
ryan953
approved these changes
Apr 30, 2026
Drop the default value entirely so every call site has to opt in or out explicitly. Pairs with the FE-as-source-of-truth shift: exposure is easy to miss when reading code, and a required flag forces the decision at the call site. The two onboarding hooks that previously relied on the default (useWelcomeAnalyticsEffect, useWelcomeHandleComplete) now pass reportExposure: false explicitly, matching the intent already described in the PR body.
cleptric
pushed a commit
that referenced
this pull request
May 5, 2026
) ## TL;DR Flip `useExperiment`'s `reportExposure` default from `true` to `false`. Pairs with an upcoming BE change that stops the org serializer from auto-logging exposure on every page load. With BE auto-logging gone, the FE becomes the source of truth, so call sites have to be intentional about whether the user is actually being rendered one of the experiment variants. ## Why a true default is wrong now `useExperiment` can be consumed inside shared components whose surrounding flow can render something other than the control or active variant. When that happens the user has not seen the experiment, but a default-true call site logs them as exposed anyway. Flipping to `false` makes the opt-in visible at the call site, so exposure is only recorded where a variant is actually rendered. ## Effect on `onboarding-scm-experiment` - `onboarding.tsx:184` opts in with `reportExposure: true`. Single point that fires the FE exposure POST and the synchronous browser `Amplitude.groupIdentify` for the parent SCM experiment. - `useWelcomeAnalyticsEffect` and `useWelcomeHandleComplete` previously relied on the default and now fall through to `false`. Both run in flows where the user may not have rendered either variant. - `onboarding.tsx:190` (`onboarding-scm-project-details-experiment`) keeps its `reportExposure: hasScmOnboarding` gate and is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Flip
useExperiment'sreportExposuredefault fromtruetofalse. Pairs with an upcoming BE change that stops the org serializer from auto-logging exposure on every page load. With BE auto-logging gone, the FE becomes the source of truth, so call sites have to be intentional about whether the user is actually being rendered one of the experiment variants.Why a true default is wrong now
useExperimentcan be consumed inside shared components whose surrounding flow can render something other than the control or active variant. When that happens the user has not seen the experiment, but a default-true call site logs them as exposed anyway. Flipping tofalsemakes the opt-in visible at the call site, so exposure is only recorded where a variant is actually rendered.Effect on
onboarding-scm-experimentonboarding.tsx:184opts in withreportExposure: true. Single point that fires the FE exposure POST and the synchronous browserAmplitude.groupIdentifyfor the parent SCM experiment.useWelcomeAnalyticsEffectanduseWelcomeHandleCompletepreviously relied on the default and now fall through tofalse. Both run in flows where the user may not have rendered either variant.onboarding.tsx:190(onboarding-scm-project-details-experiment) keeps itsreportExposure: hasScmOnboardinggate and is unchanged.