exllama(v2): fix exllamav1, add exllamav2 #1384
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
✅ Deploy Preview for localai canceled.
|
98 tasks
truecharts-admin
referenced
this pull request
in truecharts/public
Dec 18, 2023
…1.0 by renovate (#16284) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [quay.io/go-skynet/local-ai](https://togithub.com/mudler/LocalAI) | minor | `v2.0.0-cublas-cuda11` -> `v2.1.0-cublas-cuda11` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>mudler/LocalAI (quay.io/go-skynet/local-ai)</summary> ### [`v2.1.0`](https://togithub.com/mudler/LocalAI/releases/tag/v2.1.0) [Compare Source](https://togithub.com/mudler/LocalAI/compare/v2.0.0...v2.1.0) <!-- Release notes generated using configuration in .github/release.yml at master --> ##### What's Changed ##### Breaking Changes 🛠 - feat(alias): alias llama to llama-cpp, update docs by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1448](https://togithub.com/mudler/LocalAI/pull/1448) ##### Bug fixes 🐛 - fix(piper): pin petals, phonemize and espeak by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1393](https://togithub.com/mudler/LocalAI/pull/1393) - update(llama.cpp): update server, correctly propagate LLAMA_VERSION by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1440](https://togithub.com/mudler/LocalAI/pull/1440) ##### Exciting New Features 🎉 - Added Check API KEYs file to API.go by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1381](https://togithub.com/mudler/LocalAI/pull/1381) - exllama(v2): fix exllamav1, add exllamav2 by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1384](https://togithub.com/mudler/LocalAI/pull/1384) - Fix: API Key / JSON Fast Follow [#​1](https://togithub.com/mudler/LocalAI/issues/1) by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1388](https://togithub.com/mudler/LocalAI/pull/1388) - feat: add transformers-musicgen backend by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1387](https://togithub.com/mudler/LocalAI/pull/1387) - feat(diffusers): update, add autopipeline, controlnet by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1432](https://togithub.com/mudler/LocalAI/pull/1432) - feat(img2vid,txt2vid): Initial support for img2vid,txt2vid by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1442](https://togithub.com/mudler/LocalAI/pull/1442) ##### 👒 Dependencies - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1378](https://togithub.com/mudler/LocalAI/pull/1378) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1379](https://togithub.com/mudler/LocalAI/pull/1379) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1430](https://togithub.com/mudler/LocalAI/pull/1430) - ⬆️ Update mudler/go-piper by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1441](https://togithub.com/mudler/LocalAI/pull/1441) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1434](https://togithub.com/mudler/LocalAI/pull/1434) ##### Other Changes - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1385](https://togithub.com/mudler/LocalAI/pull/1385) - docs: site update fixing old image text / How To update updating GPU and CPU docker pages by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1399](https://togithub.com/mudler/LocalAI/pull/1399) - feat: cuda transformers by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1401](https://togithub.com/mudler/LocalAI/pull/1401) - feat(entrypoint): optionally prepare extra endpoints by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1405](https://togithub.com/mudler/LocalAI/pull/1405) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1390](https://togithub.com/mudler/LocalAI/pull/1390) - ⬆️ Update mudler/go-piper by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1400](https://togithub.com/mudler/LocalAI/pull/1400) - tests: add diffusers tests by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1419](https://togithub.com/mudler/LocalAI/pull/1419) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1418](https://togithub.com/mudler/LocalAI/pull/1418) - How To Updates / Model Used Switched / Removed "docker-compose" (RIP) by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1417](https://togithub.com/mudler/LocalAI/pull/1417) - fix(transformers\*): add sentence-transformers and transformers-musicgen tests, fix musicgen wrapper by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1420](https://togithub.com/mudler/LocalAI/pull/1420) - extras: add vllm,bark,vall-e-x tests, bump diffusers by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1422](https://togithub.com/mudler/LocalAI/pull/1422) - Documentation for Hipblas by [@​sfxworks](https://togithub.com/sfxworks) in [https://github.com/mudler/LocalAI/pull/1425](https://togithub.com/mudler/LocalAI/pull/1425) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1391](https://togithub.com/mudler/LocalAI/pull/1391) - docs: add aikit to integrations by [@​sozercan](https://togithub.com/sozercan) in [https://github.com/mudler/LocalAI/pull/1412](https://togithub.com/mudler/LocalAI/pull/1412) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1429](https://togithub.com/mudler/LocalAI/pull/1429) - docs(mixtral): add mixtral example by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1449](https://togithub.com/mudler/LocalAI/pull/1449) ##### New Contributors - [@​sozercan](https://togithub.com/sozercan) made their first contribution in [https://github.com/mudler/LocalAI/pull/1412](https://togithub.com/mudler/LocalAI/pull/1412) **Full Changelog**: mudler/LocalAI@v2.0.0...v2.1.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 10pm on monday" in timezone Europe/Amsterdam, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMDIuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEwMi4wIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIn0=-->
truecharts-admin
referenced
this pull request
in truecharts/public
Dec 18, 2023
…1.0@f0b3afa by renovate (#16304) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [quay.io/go-skynet/local-ai](https://togithub.com/mudler/LocalAI) | minor | `v2.0.0-cublas-cuda12-ffmpeg` -> `v2.1.0-cublas-cuda12-ffmpeg` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>mudler/LocalAI (quay.io/go-skynet/local-ai)</summary> ### [`v2.1.0`](https://togithub.com/mudler/LocalAI/releases/tag/v2.1.0) [Compare Source](https://togithub.com/mudler/LocalAI/compare/v2.0.0...v2.1.0) <!-- Release notes generated using configuration in .github/release.yml at master --> #### What's Changed ##### Breaking Changes 🛠 - feat(alias): alias llama to llama-cpp, update docs by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1448](https://togithub.com/mudler/LocalAI/pull/1448) ##### Bug fixes 🐛 - fix(piper): pin petals, phonemize and espeak by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1393](https://togithub.com/mudler/LocalAI/pull/1393) - update(llama.cpp): update server, correctly propagate LLAMA_VERSION by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1440](https://togithub.com/mudler/LocalAI/pull/1440) ##### Exciting New Features 🎉 - Added Check API KEYs file to API.go by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1381](https://togithub.com/mudler/LocalAI/pull/1381) - exllama(v2): fix exllamav1, add exllamav2 by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1384](https://togithub.com/mudler/LocalAI/pull/1384) - Fix: API Key / JSON Fast Follow [#​1](https://togithub.com/mudler/LocalAI/issues/1) by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1388](https://togithub.com/mudler/LocalAI/pull/1388) - feat: add transformers-musicgen backend by [@​dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1387](https://togithub.com/mudler/LocalAI/pull/1387) - feat(diffusers): update, add autopipeline, controlnet by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1432](https://togithub.com/mudler/LocalAI/pull/1432) - feat(img2vid,txt2vid): Initial support for img2vid,txt2vid by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1442](https://togithub.com/mudler/LocalAI/pull/1442) ##### 👒 Dependencies - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1378](https://togithub.com/mudler/LocalAI/pull/1378) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1379](https://togithub.com/mudler/LocalAI/pull/1379) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1430](https://togithub.com/mudler/LocalAI/pull/1430) - ⬆️ Update mudler/go-piper by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1441](https://togithub.com/mudler/LocalAI/pull/1441) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1434](https://togithub.com/mudler/LocalAI/pull/1434) ##### Other Changes - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1385](https://togithub.com/mudler/LocalAI/pull/1385) - docs: site update fixing old image text / How To update updating GPU and CPU docker pages by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1399](https://togithub.com/mudler/LocalAI/pull/1399) - feat: cuda transformers by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1401](https://togithub.com/mudler/LocalAI/pull/1401) - feat(entrypoint): optionally prepare extra endpoints by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1405](https://togithub.com/mudler/LocalAI/pull/1405) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1390](https://togithub.com/mudler/LocalAI/pull/1390) - ⬆️ Update mudler/go-piper by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1400](https://togithub.com/mudler/LocalAI/pull/1400) - tests: add diffusers tests by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1419](https://togithub.com/mudler/LocalAI/pull/1419) - ⬆️ Update ggerganov/whisper.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1418](https://togithub.com/mudler/LocalAI/pull/1418) - How To Updates / Model Used Switched / Removed "docker-compose" (RIP) by [@​lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1417](https://togithub.com/mudler/LocalAI/pull/1417) - fix(transformers\*): add sentence-transformers and transformers-musicgen tests, fix musicgen wrapper by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1420](https://togithub.com/mudler/LocalAI/pull/1420) - extras: add vllm,bark,vall-e-x tests, bump diffusers by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1422](https://togithub.com/mudler/LocalAI/pull/1422) - Documentation for Hipblas by [@​sfxworks](https://togithub.com/sfxworks) in [https://github.com/mudler/LocalAI/pull/1425](https://togithub.com/mudler/LocalAI/pull/1425) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1391](https://togithub.com/mudler/LocalAI/pull/1391) - docs: add aikit to integrations by [@​sozercan](https://togithub.com/sozercan) in [https://github.com/mudler/LocalAI/pull/1412](https://togithub.com/mudler/LocalAI/pull/1412) - ⬆️ Update ggerganov/llama.cpp by [@​localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1429](https://togithub.com/mudler/LocalAI/pull/1429) - docs(mixtral): add mixtral example by [@​mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1449](https://togithub.com/mudler/LocalAI/pull/1449) #### New Contributors - [@​sozercan](https://togithub.com/sozercan) made their first contribution in [https://github.com/mudler/LocalAI/pull/1412](https://togithub.com/mudler/LocalAI/pull/1412) **Full Changelog**: mudler/LocalAI@v2.0.0...v2.1.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 10pm on monday" in timezone Europe/Amsterdam, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMDIuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEwMi4wIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIn0=-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes #1053 . It is a first stab at defining an exllamav2 backend
Notes for Reviewers
Signed commits