I always keep a local copy of every model I work with regularly in a models directory on my server. I noticed that unsloth will write input_embeddings.pt and output_embeddings.pt to the source model directory, then in some cases will delete the source. This means that I now have to re-download the model. This is far from ideal.
I've tried in vain to do various workarounds such as:
- making the directory read-only (breaks unsloth due to permissions, obviously)
- making the directory and base model contents owned by root but the directory having 1777 permissions (all users get rwx permissions + sticky, same as your standard /tmp directory, but unsloth wants to modify the tokenizer)
But those tricks always fail.
Can you instead create a different directory under huggingface's cache directory instead? For example, maybe for Mistral-7B-v0.3 you might create a tmp directory called ~/.cache/unsloth/path-to-using-dashes-instead-of-slashes-Mistral-7B-v0.3-unsloth-work and write your diffs there. You could even be really fancy and make it attempt a symlink to the original files, then write your updated files to that directory in lieu of the symlinked version.
I'm getting to the point where I will have to switch to using ZFS copy-on-write or do an overlay mount or similar since I literally keep having to place the original model back in place on an almost daily basis
I always keep a local copy of every model I work with regularly in a models directory on my server. I noticed that unsloth will write
input_embeddings.ptandoutput_embeddings.ptto the source model directory, then in some cases will delete the source. This means that I now have to re-download the model. This is far from ideal.I've tried in vain to do various workarounds such as:
But those tricks always fail.
Can you instead create a different directory under huggingface's cache directory instead? For example, maybe for
Mistral-7B-v0.3you might create a tmp directory called~/.cache/unsloth/path-to-using-dashes-instead-of-slashes-Mistral-7B-v0.3-unsloth-workand write your diffs there. You could even be really fancy and make it attempt a symlink to the original files, then write your updated files to that directory in lieu of the symlinked version.I'm getting to the point where I will have to switch to using ZFS copy-on-write or do an overlay mount or similar since I literally keep having to place the original model back in place on an almost daily basis