Skip to content

Commit

Permalink
support model Mixtral-8x7B-Instruct-v0.1-GGUF
Browse files Browse the repository at this point in the history
  • Loading branch information
smalltong02 committed May 2, 2024
1 parent 69a525e commit d5b9b9a
Show file tree
Hide file tree
Showing 4 changed files with 91,178 additions and 0 deletions.
10 changes: 10 additions & 0 deletions WebUI/configs/webuiconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -1233,6 +1233,16 @@
"preset": "Mistral Instruct",
"load_type": "llamacpp",
"Huggingface": "TheBloke/Mixtral-8x7B-v0.1-GGUF"
},
"Mixtral-8x7B-Instruct-v0.1-GGUF": {
"path": "models/llm/Mixtral-8x7B-Instruct-v0.1-GGUF",
"device": "cpu",
"maxmemory": 50,
"cputhreads": 4,
"loadbits": 16,
"preset": "Mistral Instruct",
"load_type": "llamacpp",
"Huggingface": "TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF"
}
},
"70B Model": {
Expand Down
3 changes: 3 additions & 0 deletions models/llm/Mixtral-8x7B-Instruct-v0.1-GGUF/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"model_type": "mixtral"
}
Loading

0 comments on commit d5b9b9a

Please sign in to comment.