[fixed] multi loras in one ggml file!

[FNsi]

Merged
Now 7B & 13B so🤷
Really fast!!!!!

base on Wizard-uncensored.

2 version

1.7b + multi languages only.

2.13b + Starcoder lora.

feel free to check

click this guy ->🤷

[FNsi]

That part was only proved that intelligence is illustration

[pugzly]

“Your scientists were so preoccupied with whether they could, they didn’t stop to think if they should.”

On more serious note, have you tested it?
Are you seeing any improvement (or degradation) in any of the aspects?

[FNsi]

Since I haven't use 7b before... but there's a model card in readme

[FNsi]

After some test like help me list 21 type of blood test, works fine.

Meanwhile that model help me understand the build.zig in llama.cpp. It's rust!!!!

[BetaDoggo]

For anyone wondering, this is a meme. The model is actually just WizardLM-uncensored.

[FNsi]

Oh? How?

[FNsi]

I have to say you are right. I test it with same seed same input sequence in 3 different merged steps. And the output is the same.

[FNsi]

To be clear, I uploaded it with new script merged.

For anyone wondering it's a meme, try speak Finnish to the last update.

🥲after I tested with same seed same ctx same prompt sequence, results are same, so it is not working.

Sorry for anyone downloaded it.

Now it works fine.

[Castaa]

Cool, I'll try it out. What Llama.cpp parameters do you recommend for it. Thanks.

[FNsi]

No clue man, using interactive mode with -c 2048 -n -1 is fine to test everything.

[Castaa]

No clue man, using interactive mode with -c 2048 -n -1 is fine to test everything.

using ./main -m models/ggml-model-35-q51.bin -c 2048 -n -1 -i

And it just starts dumping readable but random text?

[FNsi]

Emm, I think you need use with -f prompt/Dan.txt or somthing. Find the prompt directory and pick up what you like.

Also need add -r "User" or "any name you want" with -i

[FNsi]

Done it again with new peft method.

[BetaDoggo]

There's probably an issue what whatever script you're using to merge the loras. The hash of the models matches the base model so it seems like nothing was modified.

[FNsi]

I am trying to fix it.

Script used merge_weights but it's been removed during peft update.

Done. Wrote a new script

Dict broken.

Woo, finally, done.