Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support loading custom models (trust_remote_code=True) in offline mode from local #36808

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

VladKha
Copy link

@VladKha VladKha commented Mar 19, 2025

What does this PR do?

Fixes #34855
And should fix UKPLab/sentence-transformers#2613 (will close separately)

  • Now custom models (models that require trust_remote_code=True) can be fully saved with save_pretrained() and able to be loaded with from_pretrained() in offline mode
  • All required code files are now saved in the local dir during save_pretrained(). And during loading with from_pretrained() in offline mode (even on a new fresh machine) it won't try to reach out to HF hub since all files will be self-contained in local dir
  • Adds a couple of tests for corresponding behavior

Before submitting

Who can review?

@Rocketknight1
@ArthurZucker

@github-actions github-actions bot marked this pull request as draft March 19, 2025 02:08
Copy link

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the Ready for review button (at the bottom of the PR page).

@VladKha VladKha changed the title Add full support for loading dynamic models (`trust_remote_code=True) in offline mode [WIP] Add full support for loading dynamic models (`trust_remote_code=True) in offline mode Mar 19, 2025
@VladKha VladKha changed the title [WIP] Add full support for loading dynamic models (`trust_remote_code=True) in offline mode [WIP] Add full support for loading custom dynamic models (`trust_remote_code=True) in offline mode Mar 19, 2025
@VladKha VladKha changed the title [WIP] Add full support for loading custom dynamic models (`trust_remote_code=True) in offline mode [WIP] Add full support for loading custom models (`trust_remote_code=True) in offline mode Mar 19, 2025
@VladKha VladKha changed the title [WIP] Add full support for loading custom models (`trust_remote_code=True) in offline mode [WIP] Add full support for loading custom models (trust_remote_code=True) in offline mode Mar 19, 2025
@VladKha VladKha force-pushed the offline-mode-remote-code-full-support branch from ea80e31 to d5fe4d0 Compare March 19, 2025 04:17
@VladKha VladKha changed the title [WIP] Add full support for loading custom models (trust_remote_code=True) in offline mode Add full support for loading custom models (trust_remote_code=True) in offline mode Mar 19, 2025
@VladKha VladKha changed the title Add full support for loading custom models (trust_remote_code=True) in offline mode Support loading custom models (trust_remote_code=True) in offline mode from local Mar 20, 2025
@VladKha
Copy link
Author

VladKha commented Mar 24, 2025

@Rocketknight1, run_tests has been running for 5 days now. Are you aware if this is expected behavior?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant