Skip to content

ci: Run flagsmith-private test suites inline on 0.9.0#7667

Draft
matthewelwell wants to merge 2 commits into
mainfrom
feat/consolidate-private-tests-flagsmith-private-0.9.0
Draft

ci: Run flagsmith-private test suites inline on 0.9.0#7667
matthewelwell wants to merge 2 commits into
mainfrom
feat/consolidate-private-tests-flagsmith-private-0.9.0

Conversation

@matthewelwell
Copy link
Copy Markdown
Contributor

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Contributes to Flagsmith/flagsmith-private#177, supersedes #7665.

Follows on from #7656, which consolidated rbac and workflows_logic into flagsmith-private and runs their suites inline via make test. This extends the same pattern to flagsmith-private 0.9.0, which also bundles auth_controller, licensing and flagsmith-ldap so the private suites run inline against Core API again.

  • Bump flagsmith-private to >=0.9.0. It now ships auth_controller, licensing and flagsmith-ldap so drop the extras and git source references; integrate-private-tests no longer clones them separately.
  • integrate-private-tests pulls each app's tests from flagsmith-private at the installed wheel's tag and drops them under tests/ so they inherit Core API's tests/conftest and run inline with make test.
  • .gitignore the cloned repo and the staged tests/*_tests directories.

scim and release_pipelines_logic are deliberately left out for now and continue to run in flagsmith-private's own CI:

  • scim: its compliance tests require an SCIM_SERVICE_PROVIDER config (notably dropping the request-based BASE_LOCATION_GETTER) that directly conflicts with Core API's own tests/integration/scim tests, and django-scim2 reads that setting once at import — so both suites cannot share one pytest session.
  • release_pipelines_logic: one test asserts a non-deterministic email ordering that only holds in isolation; it fails when collected alongside the full Core API suite.

Both gaps are tracked in Flagsmith/flagsmith-private#177.

How did you test this code?

Locally, against the Compose Postgres, with --extra dev --extra private installed:

  1. make integrate-private-tests to stage the suites.

  2. Ran the staged suites under Core API's pytest config (--ds=app.settings.test, --import-mode=importlib):

    uv run pytest tests/rbac_tests tests/workflow_tests tests/auth_controller_tests \
      tests/saml_integration_tests tests/saml_unit_tests
    

    Result: 343 passed.

khvn26 and others added 2 commits June 1, 2026 15:49
Bump flagsmith-private to 0.8.0, which now bundles auth_controller
alongside rbac and workflows_logic, and pull its rbac, workflows_logic,
auth_controller and saml suites into the inline `make test` run so they
run again like before the repos were split out.

- Drop the auth-controller extra and git source; it ships in
  flagsmith-private 0.8.0, so integrate-private-tests no longer clones
  flagsmith-auth-controller separately.
- integrate-private-tests pulls each app's tests from flagsmith-private
  at the installed wheel's tag and drops them under tests/ so they
  inherit Core API's tests/conftest.
- scim and release_pipelines are not pulled: scim's compliance tests need
  an SCIM_SERVICE_PROVIDER config that conflicts with Core API's own scim
  tests (django-scim2 caches it globally), and release_pipelines has an
  order-dependent assertion that only holds in isolation. Both run in
  flagsmith-private's CI.

beep boop
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Jun 1, 2026 4:11pm
flagsmith-frontend-preview Ignored Ignored Jun 1, 2026 4:11pm
flagsmith-frontend-staging Ignored Ignored Jun 1, 2026 4:11pm

Request Review

@github-actions github-actions Bot added api Issue related to the REST API ci-cd Build, test and deployment related labels Jun 1, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 1, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.37%. Comparing base (1ee9b7b) to head (7ecd04e).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7667      +/-   ##
==========================================
- Coverage   98.52%   98.37%   -0.16%     
==========================================
  Files        1444     1444              
  Lines       55083    54964     -119     
==========================================
- Hits        54273    54072     -201     
- Misses        810      892      +82     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API ci-cd Build, test and deployment related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants