Skip to content

feat: further vram optimizations#765

Merged
avinash2692 merged 2 commits intomainfrom
feat/further-vram-optimizations
Mar 30, 2026
Merged

feat: further vram optimizations#765
avinash2692 merged 2 commits intomainfrom
feat/further-vram-optimizations

Conversation

@avinash2692
Copy link
Copy Markdown
Member

@avinash2692 avinash2692 commented Mar 30, 2026

Further optimizations to the GPU VRAM when running tests.

Type of PR

  • Bug Fix
  • New Feature
  • Documentation
  • Other

Description

###F ix GPU OOM in test runs: ollama lifecycle ownership + VRAM timing**

  • run_tests_with_ollama.sh starts and warms up all three ollama models unconditionally, even when an external process has already started the server — fragmenting VRAM before HF tests run and triggering OOM
  • Add OLLAMA_EXTERNAL and OLLAMA_SKIP_WARMUP env guards to the shell script so an external orchestrator can own the lifecycle (both default to 0 so standalone use is unaffected)
  • conftest.py handles VRAM timing — loading models right before the ollama test group and evicting after via keep_alive
  • Fix _check_ollama_available() hardcoded port to respect OLLAMA_HOST/OLLAMA_PORT env vars
  • Add extra GPU cleanup in test_alora_train_integration.py for all tests.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code as added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

@avinash2692 avinash2692 requested a review from a team as a code owner March 30, 2026 00:39
@github-actions github-actions bot added the enhancement New feature or request label Mar 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

The PR description has been updated. Please fill out the template for your PR to be reviewed.

@avinash2692 avinash2692 added this pull request to the merge queue Mar 30, 2026
Merged via the queue into main with commit 243a161 Mar 30, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants