Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
method
argument toControlVector.train
. This defaults to the old method,method="pca_diff"
, but it can be switched tomethod="pca_center"
for more stability. This seems to be necessary for training vectors on larger models, such as llama3-70b. Thanks to @r3nddControlVector.import_gguf
as a counterpart toControlVector.export_gguf
. GGUF is now the preferred way to save control vectors, for better compatibility with llama.cpp (however, the old pickle method will continue to work)ControlVector
Co-Authored-With: r3ndd@users.noreply.github.com