Skip to content

Use AR_WORK_SPACE/offload/ as base dir for offload temp files#1552

Merged
yiliu30 merged 4 commits intomainfrom
offload-use-ar-workspace-dir
Mar 17, 2026
Merged

Use AR_WORK_SPACE/offload/ as base dir for offload temp files#1552
yiliu30 merged 4 commits intomainfrom
offload-use-ar-workspace-dir

Conversation

@yiliu30
Copy link
Copy Markdown
Contributor

@yiliu30 yiliu30 commented Mar 16, 2026

Summary

  • Redirect OffloadManager temp files from /tmp/ to AR_WORK_SPACE/offload/ so they live alongside other workspace artifacts and respect the user-configured workspace path.
  • When low_cpu_mem_usage=True, large .pt offload files now go to ar_work_space/offload/<prefix>_xxxx/ by default (or a custom path if AR_WORK_SPACE is set via environment variable).

Test plan

  • pytest test/test_cpu/core/test_autoround.py -v — all relevant tests pass; log output confirms tempdir = .../ar_work_space/offload/compressor_...
  • Verify with custom AR_WORK_SPACE env var that offload files land in the expected location

OffloadManager previously wrote temp .pt files to /tmp. Redirect them
to AR_WORK_SPACE/offload/ so they live alongside other workspace
artifacts and respect the user-configured workspace path.

Signed-off-by: yiliu7 <yi4.liu@intel.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Copilot AI review requested due to automatic review settings March 16, 2026 10:34
@yiliu30 yiliu30 requested a review from lvliang-intel March 16, 2026 10:36
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Redirects OffloadManager’s temporary offload files from the system temp directory to a workspace-local directory so offload artifacts live alongside other AR_WORK_SPACE outputs.

Changes:

  • Create <AR_WORK_SPACE>/offload/ and place OffloadManager temp directories under it via tempfile.mkdtemp(..., dir=base_dir).
  • Log the resulting workspace-based tempdir path for easier debugging/validation.

@wenhuach21
Copy link
Copy Markdown
Contributor

Why not use output_dir and delete the offload directory after quantization?

After removing the per-run temp subdirectory, also remove the parent
ar_work_space/offload/ directory if it is empty. Uses os.rmdir which
only succeeds on empty directories, so concurrent runs are safe.

Signed-off-by: yiliu7 <yi4.liu@intel.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
@yiliu30
Copy link
Copy Markdown
Contributor Author

yiliu30 commented Mar 16, 2026

Why not use output_dir

output_dir is meant for final output, mixing temporary offload files into the user's output directory would be messy.

and delete the offload directory after quantization?

The cleanup cleanup does happen through the final reload().

@yiliu30 yiliu30 requested a review from wenhuach21 March 17, 2026 02:42
@yiliu30 yiliu30 added this to the 0.12.0 milestone Mar 17, 2026
@yiliu30 yiliu30 merged commit 803912a into main Mar 17, 2026
29 checks passed
@yiliu30 yiliu30 deleted the offload-use-ar-workspace-dir branch March 17, 2026 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants