Skip to content

Conversation

@JoshFerge
Copy link
Member

@JoshFerge JoshFerge commented Nov 14, 2025

When the organizations:gen-ai-features feature flag is disabled, the "gen ai" setting is now disabled and set to "off" regardless of the org's setting value.
Screenshot 2025-11-19 at 3 51 34 PM

When the organizations:gen-ai-features feature flag is disabled:
- "Show Generative AI Features" toggle is disabled (grayed out) and set to off
- "Enable AI Code Review" toggle is also disabled (grayed out)
- Appropriate disabled reason messages are shown to users

Changes:
- Updated aiFeatureSettings.tsx to check feature flag and disable toggle
- Updated preventAiSettings.tsx to check feature flag and show appropriate disabled reason
- Added tests to verify behavior when feature flag is disabled
- Updated existing tests to include feature flag where needed
The feature flag should only control whether the setting is editable,
not what value is currently displayed. The defaultValue should always
reflect the organization's actual hideAiFeatures setting.
The frontend receives feature flags without the 'organizations:' prefix.
Changed from 'organizations:gen-ai-features' to 'gen-ai-features'.
- When feature flag is off, toggle shows as unchecked (off) regardless of org setting
- Removed 'This feature is not available' tooltip - fields just gray out silently
- Disabled tag only shows for specific reasons (self-hosted, non-US region)
- Feature flag acts as master switch that overrides org settings in UI
Removes the async keyword from the test function that doesn't use any
await expressions, fixing the eslint @typescript-eslint/require-await
error.
@JoshFerge JoshFerge force-pushed the jferg/disable-ai-settings-when-feature-flag-off branch from 066b0d5 to 2f562f0 Compare November 19, 2025 20:28
When the gen-ai-features flag is disabled but the organization's
enablePrReviewTestGeneration setting is true, the UI was incorrectly
showing the checkbox as checked. This fix uses setValue to transform
the initial display value to false when the feature flag is off,
ensuring the UI accurately reflects that the feature is disabled.

Also adds test coverage for this scenario.
… flag is off

When the gen-ai-features flag is disabled, the "Show Generative AI Features"
checkbox now displays as unchecked regardless of the organization's stored
hideAiFeatures value. This provides a clearer UX indicating that AI features
are not available.

Changes:
- Added setValue to hideAiFeatures field to force display value to false when flag is off
- Updated preventAiSettings visibility logic to check initialData when flag is off
  to prevent setValue transformation from hiding the field
- When flag is on, visibility uses current model value for dynamic toggling
…sabled

When the gen-ai-features flag is disabled, force the "Show Generative AI
Features" checkbox to display as unchecked, even if the organization's
hideAiFeatures value in the database is true.

This ensures the UI accurately reflects that AI features are disabled
when the feature flag is off, preventing confusion where the toggle
appears enabled but is actually disabled.

Implementation uses setValue to transform the initial display value to
false when the feature flag is off, while keeping the API submission
logic unchanged via getValue.
@JoshFerge JoshFerge marked this pull request as ready for review November 19, 2025 21:20
@JoshFerge JoshFerge requested review from a team and Dhrumil-Sentry November 19, 2025 21:20
Comment on lines +33 to +38
setValue: value => {
if (!hasFeatureFlag) {
return false;
}
return value;
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is necessary if it's already disabled.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the whole set value fn?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no i think it is, otherwise the value shows as true
Screenshot 2025-11-19 at 4 24 24 PM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then it should be done in getValue I think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm that doesn't seem to work?

return false;
}
return value;
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: setValue doesn't invert hideAiFeatures value correctly

The setValue function returns the raw value when the feature flag is enabled, but it should invert it to match the UI semantics. The field stores hideAiFeatures from the backend, but the UI displays "Show Generative AI Features" (the opposite). When hasFeatureFlag is true, setValue should return !value to invert the backend value to the UI value, just like defaultValue does with !organization.hideAiFeatures. Without this inversion, a backend value of hideAiFeatures: false (AI enabled) would incorrectly display as unchecked in the UI.

Fix in Cursor Fix in Web

@codecov
Copy link

codecov bot commented Nov 19, 2025

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
12592 3 12589 10
View the top 3 failed test(s) by shortest run time
OrganizationSettingsForm shows PR Review and Test Generation field when AI features are enabled
Stack Traces | 0.146s run time
TestingLibraryElementError: Unable to find an element with the text: Enable AI Code Review. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

Ignored nodes: comments, script, style
...
    at Object.getElementError (.../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/config.js:37:19)
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:76:38
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:52:17
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:95:19
    at Object.getByText (.../settings/organizationGeneralSettings/organizationSettingsForm.spec.tsx:331:19)
    at Promise.finally.completed (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1559:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1499:10)
    at _callCircusTest (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1009:40)
    at _runTest (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:949:3)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:839:13)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:829:11)
    at run (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:757:3)
    at runAndTransformResultsToJestFormat (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1920:21)
    at jestAdapter (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/runner.js:101:19)
    at runTestInternal (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:272:16)
    at runTest (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:340:7)
    at Object.worker (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:494:12)
OrganizationSettingsForm renders AI Code Review field
Stack Traces | 0.168s run time
TestingLibraryElementError: Unable to find an element with the text: Enable AI Code Review. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

Ignored nodes: comments, script, style
...
    at Object.getElementError (.../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/config.js:37:19)
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:76:38
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:52:17
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:95:19
    at Object.getByText (.../settings/organizationGeneralSettings/organizationSettingsForm.spec.tsx:288:19)
    at Promise.finally.completed (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1559:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1499:10)
    at _callCircusTest (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1009:40)
    at _runTest (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:949:3)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:839:13)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:829:11)
    at run (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:757:3)
    at runAndTransformResultsToJestFormat (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1920:21)
    at jestAdapter (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/runner.js:101:19)
    at runTestInternal (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:272:16)
    at runTest (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:340:7)
    at Object.worker (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:494:12)
OrganizationSettingsForm AI Code Review field is disabled when feature flag is off
Stack Traces | 0.229s run time
TestingLibraryElementError: Unable to find an accessible element with the role "checkbox" and name `/Enable AI Code Review/i`

Here are the accessible roles:

  textbox:

  Name "Organization Slug":
  ...

  Name "Display Name":
  ...

  Name "Organization ID":
  ...

  Name "Default Role":
  ...

  Name "Debug Files Access":
  ...

  --------------------------------------------------
  link:

  Name "new features":
  ...

  Name "generative AI features":
  ...

  --------------------------------------------------
  checkbox:

  Name "Early Adopter":
  ...

  Name "Show Generative AI Features":
  ...

  Name "Enable Code Coverage Insights disabled powered by Codecov":
  ...

  Name "Open Team Membership":
  ...

  Name "Let Members Invite Others":
  ...

  Name "Let Members Create Projects":
  ...

  Name "Let Members Delete Events":
  ...

  Name "Let Members Create and Edit Alerts":
  ...

  --------------------------------------------------
  status:

  Name "":
  ...

  --------------------------------------------------
  img:

  Name "":
  ...

  Name "":
  ...

  --------------------------------------------------
  radiogroup:

  Name "Avatar Type":
  ...

  --------------------------------------------------
  radio:

  Name "Use initials":
  ...

  Name "Upload an image":
  ...

  --------------------------------------------------
  button:

  Name "Save Avatar":
  ...

  --------------------------------------------------
  presentation:

  Name "":
  ...

  --------------------------------------------------

Ignored nodes: comments, script, style
...
    at Object.getElementError (.../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/config.js:37:19)
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:76:38
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:52:17
    at .../sentry/node_modules/.pnpm/@testing-library+dom@10.4.0/node_modules/@.../dom/dist/query-helpers.js:95:19
    at Object.getByRole (.../settings/organizationGeneralSettings/organizationSettingsForm.spec.tsx:423:37)
    at Promise.finally.completed (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1559:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1499:10)
    at _callCircusTest (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1009:40)
    at _runTest (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:949:3)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:839:13)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:829:11)
    at _runTestsForDescribeBlock (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:829:11)
    at run (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:757:3)
    at runAndTransformResultsToJestFormat (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/jestAdapterInit.js:1920:21)
    at jestAdapter (.../sentry/node_modules/.pnpm/jest-circus@30.0.4_babel-plugin-macros@3.1..../jest-circus/build/runner.js:101:19)
    at runTestInternal (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:272:16)
    at runTest (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:340:7)
    at Object.worker (.../sentry/node_modules/.pnpm/jest-runner@30.0..../jest-runner/build/testWorker.js:494:12)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@JoshFerge JoshFerge merged commit 2e75243 into master Nov 19, 2025
47 checks passed
@JoshFerge JoshFerge deleted the jferg/disable-ai-settings-when-feature-flag-off branch November 19, 2025 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants