Test fixture cleanup by galvana · Pull Request #6008 · ethyca/fides

galvana · 2025-04-08T05:36:36Z

Description Of Changes

General cleanup of tests and fixtures. Tests no longer have implied dependencies or required execution order. This means tests can run independently and more importantly in parallel.

Removed most autouse fixtures except for a few essential ones. Each test now explicitly defines its required fixtures.
Refactored tests to run independently without requiring a specific test order to pass.
A standard clear_db_tables function-scoped fixture to reset the DB state at the end of each test.
Removed ctl and ops specific db fixtures in favor of single db and async_session fixtures defined at the root level conftest.py.
Refactored migrate_db to only handle DB migration. Side effects of creating seed and sample data were moved to separate functions.
Added pytest-xdist and enabled parallel test execution with 4 workers
- ops-unit-api from 14 minutes to 4.5 minutes
- ops-unit-non-api from 15 minutes to 4.5 minutes

Steps to Confirm

All tests should pass

Pre-Merge Checklist

vercel · 2025-04-08T05:36:37Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
fides-plus-nightly	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 23, 2025 11:34pm
fides-privacy-center	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 23, 2025 11:34pm

codecov · 2025-04-14T08:10:54Z

Codecov Report

Attention: Patch coverage is 80.00000% with 16 lines in your changes missing coverage. Please review.

Project coverage is 86.85%. Comparing base (72532df) to head (eb2e09a).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/fides/api/db/seed.py	84.61%	7 Missing and 1 partial ⚠️
src/fides/api/api/deps.py	60.00%	4 Missing ⚠️
src/fides/api/app_setup.py	66.66%	2 Missing and 1 partial ⚠️
src/fides/api/api/v1/endpoints/admin.py	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6008      +/-   ##
==========================================
- Coverage   86.87%   86.85%   -0.03%     
==========================================
  Files         419      419              
  Lines       25980    26000      +20     
  Branches     2827     2829       +2     
==========================================
+ Hits        22570    22581      +11     
- Misses       2789     2806      +17     
+ Partials      621      613       -8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

galvana · 2025-04-21T22:00:18Z

-            "timeout",
-            "--signal=INT",
-            "360",


We don't need to timeout anymore, tests don't get stuck

galvana · 2025-04-21T22:00:35Z

+                "-n",
+                "4",


This flag runs the tests with 4 workers

galvana · 2025-04-21T22:01:44Z



-def load_default_dsr_policies() -> None:
+def load_default_dsr_policies(session: Session) -> None:


Made this synchronous to avoid DB locking in the FastAPI lifespan function

galvana · 2025-04-21T22:03:02Z


+@pytest.fixture(scope="function")
+@pytest.mark.asyncio
+async def fideslang_resources(


This fixture creates the data needed for some CLI tests. This makes it so we're not dependent on the side-effects of other tests

galvana · 2025-04-21T22:03:52Z

    ApplicationConfig.update_config_set(db, CONFIG)


-@pytest.fixture(autouse=True, scope="function")


Removed autouse to make it easier to see which fixtures a test is dependent on

galvana · 2025-04-21T22:05:25Z

+@pytest.fixture(scope="function")
+def default_data_categories(db: Session):
+    for data_category in DEFAULT_TAXONOMY.data_category:
+        if (
+            DataCategoryDbModel.get_by(db, field="name", value=data_category.name)
+            is None
+        ):
+            DataCategoryDbModel.create(
+                db=db, data=data_category.model_dump(mode="json")
+            )
+
+
+@pytest.fixture(scope="function")
+def default_data_uses(db: Session):
+    for data_use in DEFAULT_TAXONOMY.data_use:
+        if DataUse.get_by(db, field="name", value=data_use.name) is None:
+            DataUse.create(db=db, data=data_use.model_dump(mode="json"))
+
+
+@pytest.fixture(scope="function")
+def default_organization(db: Session):
+    load_default_organization(db)
+
+
+@pytest.fixture(scope="function")
+def default_taxonomy(db: Session):
+    load_default_taxonomy(db)


Saving some time by not creating these fixtures unless they're needed by a test

galvana · 2025-04-21T22:06:38Z

+
+
+@pytest.fixture(scope="function", autouse=True)
+async def clear_db_tables(db, async_session):


Clears all database tables after each test run. This also closes any open DB connections (sync or async) that may linger after a test

galvana · 2025-04-21T22:08:36Z

+def pytest_configure_node(node):
+    """Pytest hook automatically called for each xdist worker node configuration."""
+    if hasattr(node, "workerinput") and node.workerinput:
+        worker_id = node.workerinput["workerid"]
+        print(
+            f"[Configure Node] Configuring database and config for worker {worker_id}..."
+        )
+
+        os.environ["FIDES__DATABASE__TEST_DB"] = f"fides_test_{worker_id}"
+
+        get_config.cache_clear()
+        fides_config = get_config()
+        sync_db_uri = fides_config.database.sqlalchemy_test_database_uri
+        async_db_uri = fides_config.database.async_database_uri
+
+        # Log connection strings
+        print(
+            f"[Configure Node] Sync DB URI: {sync_db_uri} Async DB URI: {async_db_uri}"
+        )
+    else:
+        print(
+            "[Configure Node] Skipping DB setup/config update on single node or non-xdist run."
+        )


This is what allows for parallel test execution. This runs before any of the tests, and allows each worker to specify the env var for its own test DB. Each worker then creates its own database at the beginning of the test run that it uses for all of its tests.

galvana · 2025-04-21T22:11:12Z

 pytest-env==0.7.0
 pytest-mock==3.14.0
 pytest-rerunfailures==14.0
+pytest-xdist==3.6.1


For parallel test execution ⚡

JadeCara

Really nice work! Very exciting changes!

JadeCara · 2025-04-22T18:20:05Z

+            configure_db(CONFIG.database.sync_database_uri)
+            with get_autoclose_db_session() as session:
+                seed_db(session)
+            if CONFIG.database.load_samples:
+                async with get_async_autoclose_db_session() as async_session:
+                    await seed.load_samples(async_session)


JadeCara · 2025-04-22T18:23:36Z

    return FidesDataCategory


+# TODO: Move away from using this, it conflicts with the DataCategory model


Do we have a story to follow up on this?

JadeCara · 2025-04-22T18:34:09Z


 @pytest.mark.unit
 class TestFilterDataCategories:
-    @pytest.mark.skip("this times out on CI")


JadeCara · 2025-04-22T18:37:46Z



 class TestDB:
-    @pytest.mark.skip(


JadeCara · 2025-04-22T18:39:21Z



 @pytest.fixture(scope="session")
-def monkeysession():


cypress · 2025-04-24T00:02:57Z

fides Run #12866

Run Properties: Passed #12866 • bf97f58e8e: Test fixture cleanup (#6008)

Project	`fides`
Branch Review	`main`
Run status	`Passed #12866`
Run duration	`00m 53s`
Commit	`bf97f58e8e: Test fixture cleanup (#6008)`
Committer	`Adrian Galvan`
View all properties for this run ↗︎

Test results
Failures	`0`
Flaky	`0`
Pending	`0`
Skipped	`0`
Passing	`5`
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

Test fixture cleanup

6534a8c

vercel Bot deployed to Preview – fides-privacy-center April 8, 2025 05:37 View deployment

Restoring some autouse fixtures

0bd433d

vercel Bot deployed to Preview – fides-privacy-center April 8, 2025 06:34 View deployment

Fixing DRP endpoint tests

b4df483

vercel Bot deployed to Preview – fides-privacy-center April 9, 2025 06:41 View deployment

Fixing test_policy_endpoints

5718e06

vercel Bot deployed to Preview – fides-privacy-center April 9, 2025 07:01 View deployment

Marking test_config_endpoints as needs investigation

b5de4eb

vercel Bot deployed to Preview – fides-privacy-center April 9, 2025 07:30 View deployment

Merge branch 'main' into test-cleanup

3e1964f

vercel Bot deployed to Preview – fides-plus-nightly April 14, 2025 01:06 View deployment

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 01:06 View deployment

Fixing more tests

0f1371d

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 02:27 View deployment

Static check fixes

f9dfbc9

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 02:35 View deployment

Removing autouse=True from seed_data

a914afa

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 05:04 View deployment

Creating default_data_category fixture

3167875

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 05:39 View deployment

Adding default_data_uses fixture

29c250b

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 05:49 View deployment

Fixing more tests

7ea1abc

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 06:41 View deployment

Fixin default_access_policy fixture

58b1671

vercel Bot deployed to Preview – fides-privacy-center April 14, 2025 07:41 View deployment

Merge branch 'main' into test-cleanup

00ea021

vercel Bot deployed to Preview – fides-privacy-center April 21, 2025 05:20 View deployment

Adding mssql health check

f0b41c3

vercel Bot deployed to Preview – fides-privacy-center April 21, 2025 16:15 View deployment

Cleaning up fixtures

60c8937

vercel Bot deployed to Preview – fides-privacy-center April 21, 2025 18:44 View deployment

Merge branch 'main' into test-cleanup

0d44358

vercel Bot deployed to Preview – fides-privacy-center April 21, 2025 19:40 View deployment

vercel Bot deployed to Preview – fides-plus-nightly April 21, 2025 19:44 View deployment

Adding missing fixtures

baa5a84

vercel Bot deployed to Preview – fides-privacy-center April 21, 2025 19:59 View deployment

galvana commented Apr 21, 2025

View reviewed changes

Merge branch 'main' into test-cleanup

e83d6c6

vercel Bot deployed to Preview – fides-plus-nightly April 22, 2025 04:52 View deployment

vercel Bot deployed to Preview – fides-privacy-center April 22, 2025 04:52 View deployment

galvana requested a review from JadeCara April 22, 2025 15:15

Merge branch 'main' into test-cleanup

9f90a0e

vercel Bot deployed to Preview – fides-privacy-center April 22, 2025 15:15 View deployment

Merge branch 'main' into test-cleanup

9017a7a

vercel Bot deployed to Preview – fides-privacy-center April 22, 2025 17:45 View deployment

JadeCara approved these changes Apr 22, 2025

View reviewed changes

galvana mentioned this pull request Apr 22, 2025

DB fixture cleanup #5831

Closed

15 tasks

Merge branch 'main' into test-cleanup

12364ee

vercel Bot deployed to Preview – fides-plus-nightly April 23, 2025 15:31 View deployment

vercel Bot deployed to Preview – fides-privacy-center April 23, 2025 15:32 View deployment

galvana added 4 commits April 23, 2025 11:54

Merge branch 'main' into test-cleanup

e320714

Merge branch 'main' into test-cleanup

a8f90d2

Updating change log

53f34c3

Merge branch 'main' into test-cleanup

eb2e09a



		def load_default_dsr_policies() -> None:
		def load_default_dsr_policies(session: Session) -> None:

		ApplicationConfig.update_config_set(db, CONFIG)


		@pytest.fixture(autouse=True, scope="function")



		@pytest.fixture(scope="function", autouse=True)
		async def clear_db_tables(db, async_session):

		return FidesDataCategory


		# TODO: Move away from using this, it conflicts with the DataCategory model

Conversation

galvana commented Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description Of Changes

Steps to Confirm

Pre-Merge Checklist

Uh oh!

vercel Bot commented Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Apr 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

galvana Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JadeCara left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cypress Bot commented Apr 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

galvana commented Apr 8, 2025 •

edited

Loading

vercel Bot commented Apr 8, 2025 •

edited

Loading

codecov Bot commented Apr 14, 2025 •

edited

Loading

galvana Apr 21, 2025 •

edited

Loading