Skip to content

fix: close tmp fs and evict udf modules on ExecutorManager.close()#5046

Merged
Yicong-Huang merged 6 commits into
apache:mainfrom
Ma77Ball:fix/fixtureLeaks
May 13, 2026
Merged

fix: close tmp fs and evict udf modules on ExecutorManager.close()#5046
Yicong-Huang merged 6 commits into
apache:mainfrom
Ma77Ball:fix/fixtureLeaks

Conversation

@Ma77Ball
Copy link
Copy Markdown
Contributor

@Ma77Ball Ma77Ball commented May 13, 2026

What changes were proposed in this PR?

The executor_manager fixture in test_executor_manager.py guarded its cleanup with hasattr(manager, "_fs"), but the @cached_property attribute is named fs, so the guard was always false and manager.close() never ran. This leaked TempFS directories into /var/tmp on every test run. Even when close() did run from the sibling fixture, it only closed the filesystem and left stale udf-vN entries in sys.modules plus the tmp directory in sys.path. This PR drops the broken guard and extends ExecutorManager.close() to also remove the module from sys.modules and the path from sys.path, so a single close() call fully reverses every global side effect of fs and load_executor_definition.

Any related issues, documentation, or discussions?

Closes: #4730

How was this PR tested?

  • All 20 tests in test_executor_manager.py pass.
  • Verified /var/tmp no longer accumulates TempFS directories after a test run.
  • Verified no leaked udf-v* files anywhere under /tmp or /var/tmp after a run.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.7 in compliance with ASF

@Ma77Ball
Copy link
Copy Markdown
Contributor Author

@chenlica please assign a reviewer.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.37%. Comparing base (bb795a0) to head (94770c6).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5046      +/-   ##
============================================
+ Coverage     42.81%   43.37%   +0.55%     
+ Complexity     2206     2172      -34     
============================================
  Files          1045     1035      -10     
  Lines         40119    38750    -1369     
  Branches       4239     4051     -188     
============================================
- Hits          17178    16806     -372     
+ Misses        21875    20892     -983     
+ Partials       1066     1052      -14     
Flag Coverage Δ *Carryforward flag
access-control-service 39.53% <ø> (ø) Carriedforward from c93c8f7
agent-service 33.72% <ø> (ø) Carriedforward from c93c8f7
amber 43.17% <ø> (-0.52%) ⬇️ Carriedforward from c93c8f7
computing-unit-managing-service 0.00% <ø> (ø) Carriedforward from c93c8f7
config-service 0.00% <ø> (ø) Carriedforward from c93c8f7
file-service 32.10% <ø> (-0.09%) ⬇️ Carriedforward from c93c8f7
frontend 35.17% <ø> (+1.24%) ⬆️ Carriedforward from c93c8f7
python 88.94% <100.00%> (-0.03%) ⬇️
workflow-compiling-service 47.72% <ø> (ø) Carriedforward from c93c8f7

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chenlica chenlica requested review from bobbai00 and mengw15 May 13, 2026 06:10
@chenlica
Copy link
Copy Markdown
Contributor

@mengw15 @bobbai00 Please review it.

@Yicong-Huang
Copy link
Copy Markdown
Contributor

Also please make coverage happy ;) thanks

@Ma77Ball Ma77Ball requested a review from Yicong-Huang May 13, 2026 08:46
Copy link
Copy Markdown
Contributor

@Yicong-Huang Yicong-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Yicong-Huang Yicong-Huang merged commit c58941e into apache:main May 13, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ExecutorManager test fixture leaks tmp dirs and modules due to hasattr(_fs) typo

4 participants