Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(autogpt/transformers): consume trust_remote_code #1799

Merged
merged 1 commit into from
Mar 5, 2024

Conversation

dave-gray101
Copy link
Collaborator

Discussion in our help channel pointed me at the trust_remote_code parameter used in Transformers backends.

Currently, this is enabled on default... while convenient, I do not feel that it is wise to expose our users to pickle supply chain attack s, especially since many of them are trying out AI generation for the first time and may not fully understand the risks this option exposes them to.

Down the line, it will make sense to detect the error raised by these backends when they encounter a model that simply doesn't work without custom code so that we can give a clear error message to the user explaining both the solution and the risks involved, but I wanted to get a "quick fix" out stopping the problem while we investigate that.

Luckily, vLLM was already doing the right thing - so we even have a parameter already plumbed through gRPC and it's a simple matter of modifying the python gRPC servers to actually read it

@dave-gray101 dave-gray101 requested a review from mudler March 5, 2024 17:29
Copy link

netlify bot commented Mar 5, 2024

Deploy Preview for localai canceled.

Name Link
🔨 Latest commit cd8b73b
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/65e7565f4c6b7200088cb8b3

Copy link
Owner

@mudler mudler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

@mudler mudler enabled auto-merge (squash) March 5, 2024 17:32
@mudler mudler added enhancement New feature or request breaking-change labels Mar 5, 2024
@mudler mudler changed the title Security Fix / Defaults Change: Disable External Code by default! feat(autogpt/transformers): consume trust_remote_code Mar 5, 2024
@mudler
Copy link
Owner

mudler commented Mar 5, 2024

@dave-gray101 can you pls use conventional commit message? It's ok for now as editing the PR title will get into the history with that, but it really helps to have a clean history when looking back at the changes in the code

@mudler mudler disabled auto-merge March 5, 2024 18:47
@mudler mudler merged commit 5c69dd1 into mudler:master Mar 5, 2024
19 checks passed
truecharts-admin added a commit to truecharts/charts that referenced this pull request Mar 17, 2024
…0.0@5cd0285 by renovate (#19391)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) |
minor | `v2.9.0` -> `v2.10.0` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>mudler/LocalAI (docker.io/localai/localai)</summary>

###
[`v2.10.0`](https://togithub.com/mudler/LocalAI/releases/tag/v2.10.0)

[Compare
Source](https://togithub.com/mudler/LocalAI/compare/v2.9.0...v2.10.0)

##### LocalAI v2.10.0 Release Notes

Excited to announce the release of LocalAI v2.10.0! This version
introduces significant changes, including breaking changes, numerous bug
fixes, exciting new features, dependency updates, and more. Here's a
summary of what's new:

##### Breaking Changes 🛠

- The `trust_remote_code` setting in the YAML config file of the model
are now consumed for enhanced security measures also for the AutoGPTQ
and transformers backend, thanks to
[@&#8203;dave-gray101](https://togithub.com/dave-gray101)'s contribution
([#&#8203;1799](https://togithub.com/mudler/LocalAI/pull/1799)). If your
model relied on the old behavior and you are sure of what you are doing,
set `trust_remote_code: true` in the YAML config file.

##### Bug Fixes 🐛

- Various fixes have been implemented to enhance the stability and
performance of LocalAI:
- SSE no longer omits empty `finish_reason` fields for better
compatibility with the OpenAI API, fixed by
[@&#8203;mudler](https://togithub.com/mudler)
([#&#8203;1745](https://togithub.com/mudler/LocalAI/pull/1745)).
- Functions now correctly handle scenarios with no results, also
addressed by [@&#8203;mudler](https://togithub.com/mudler)
([#&#8203;1758](https://togithub.com/mudler/LocalAI/pull/1758)).
- A Command Injection Vulnerability has been fixed by
[@&#8203;ouxs-19](https://togithub.com/ouxs-19)
([#&#8203;1778](https://togithub.com/mudler/LocalAI/pull/1778)).
- OpenCL-based builds for llama.cpp have been restored, thanks to
[@&#8203;cryptk](https://togithub.com/cryptk)'s efforts
([#&#8203;1828](https://togithub.com/mudler/LocalAI/pull/1828),
[#&#8203;1830](https://togithub.com/mudler/LocalAI/pull/1830)).
- An issue with OSX build `default.metallib` has been resolved, which
should now allow running the llama-cpp backend on Apple arm64, fixed by
[@&#8203;dave-gray101](https://togithub.com/dave-gray101)
([#&#8203;1837](https://togithub.com/mudler/LocalAI/pull/1837)).

##### Exciting New Features 🎉

-   LocalAI continues to evolve with several new features:
- Ongoing implementation of the assistants API, making great progress
thanks to community contributions, including an initial implementation
by [@&#8203;christ66](https://togithub.com/christ66)
([#&#8203;1761](https://togithub.com/mudler/LocalAI/pull/1761)).
- Addition of diffusers/transformers support for Intel GPU - now you can
generate images and use the `transformer` backend also on Intel GPUs,
implemented by [@&#8203;mudler](https://togithub.com/mudler)
([#&#8203;1746](https://togithub.com/mudler/LocalAI/pull/1746)).
- Introduction of Bitsandbytes quantization for transformer backend
enhancement and a fix for transformer backend error on CUDA by
[@&#8203;fakezeta](https://togithub.com/fakezeta)
([#&#8203;1823](https://togithub.com/mudler/LocalAI/pull/1823)).
- Compatibility layers for Elevenlabs and OpenAI TTS, enhancing
text-to-speech capabilities: Now LocalAI is compatible with Elevenlabs
and OpenAI TTS, thanks to [@&#8203;mudler](https://togithub.com/mudler)
([#&#8203;1834](https://togithub.com/mudler/LocalAI/pull/1834)).
- vLLM now supports `stream: true`! This feature was introduced by
[@&#8203;golgeek](https://togithub.com/golgeek)
([#&#8203;1749](https://togithub.com/mudler/LocalAI/pull/1749)).

##### Dependency Updates 👒

- Our continuous effort to keep dependencies up-to-date includes
multiple updates to `ggerganov/llama.cpp`, `donomii/go-rwkv.cpp`,
`mudler/go-stable-diffusion`, and others, ensuring that LocalAI is built
on the latest and most secure libraries.

##### Other Changes

- Several internal changes have been made to improve the development
process and documentation, including updates to integration guides,
stress reduction on self-hosted runners, and more.

#### Details of What's Changed

##### Breaking Changes 🛠

- feat(autogpt/transformers): consume `trust_remote_code` by
[@&#8203;dave-gray101](https://togithub.com/dave-gray101) in
[mudler/LocalAI#1799

##### Bug fixes 🐛

- fix(sse): do not omit empty finish_reason by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1745
- fix(functions): handle correctly when there are no results by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1758
- fix(tests): re-enable tests after code move by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1764
- Fix Command Injection Vulnerability by
[@&#8203;ouxs-19](https://togithub.com/ouxs-19) in
[mudler/LocalAI#1778
- fix: the correct BUILD_TYPE for OpenCL is clblas (with no t) by
[@&#8203;cryptk](https://togithub.com/cryptk) in
[mudler/LocalAI#1828
- fix: missing OpenCL libraries from docker containers during clblas
docker build by [@&#8203;cryptk](https://togithub.com/cryptk) in
[mudler/LocalAI#1830
- fix: osx build default.metallib by
[@&#8203;dave-gray101](https://togithub.com/dave-gray101) in
[mudler/LocalAI#1837

##### Exciting New Features 🎉

- fix: vllm - use AsyncLLMEngine to allow true streaming mode by
[@&#8203;golgeek](https://togithub.com/golgeek) in
[mudler/LocalAI#1749
- refactor: move remaining api packages to core by
[@&#8203;dave-gray101](https://togithub.com/dave-gray101) in
[mudler/LocalAI#1731
- Bump vLLM version + more options when loading models in vLLM by
[@&#8203;golgeek](https://togithub.com/golgeek) in
[mudler/LocalAI#1782
- feat(assistant): Initial implementation of assistants api by
[@&#8203;christ66](https://togithub.com/christ66) in
[mudler/LocalAI#1761
- feat(intel): add diffusers/transformers support by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1746
- fix(config): set better defaults for inferencing by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1822
- fix(docker-compose): update docker compose file by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1824
- feat(model-help): display help text in markdown by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1825
- feat: Add Bitsandbytes quantization for transformer backend
enhancement
[#&#8203;1775](https://togithub.com/mudler/LocalAI/issues/1775) and fix:
Transformer backend error on CUDA
[#&#8203;1774](https://togithub.com/mudler/LocalAI/issues/1774) by
[@&#8203;fakezeta](https://togithub.com/fakezeta) in
[mudler/LocalAI#1823
- feat(tts): add Elevenlabs and OpenAI TTS compatibility layer by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1834
- feat(embeddings): do not require to be configured by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1842

##### 👒 Dependencies

- ⬆️ Update docs version mudler/LocalAI by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1752
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1753
- deps(llama.cpp): update by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1759
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1756
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1767
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1772
- ⬆️ Update donomii/go-rwkv.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1771
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1779
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1789
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1791
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1794
- depedencies(sentencentranformers): update dependencies by
[@&#8203;TwinFinz](https://togithub.com/TwinFinz) in
[mudler/LocalAI#1797
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1801
- ⬆️ Update mudler/go-stable-diffusion by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1802
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1805
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1811
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1827

##### Other Changes

- ci: add stablediffusion to release by
[@&#8203;sozercan](https://togithub.com/sozercan) in
[mudler/LocalAI#1757
- Update integrations.md by
[@&#8203;Joshhua5](https://togithub.com/Joshhua5) in
[mudler/LocalAI#1765
- ci: reduce stress on self-hosted runners by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1776
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1785
- Revert "feat(assistant): Initial implementation of assistants api" by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1790
- Edit links in readme and integrations page by
[@&#8203;lunamidori5](https://togithub.com/lunamidori5) in
[mudler/LocalAI#1796
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1813
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1816
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1818
- fix(doc/examples): set defaults to mirostat by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1820
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1821
- fix: OSX Build Files for llama.cpp by
[@&#8203;dave-gray101](https://togithub.com/dave-gray101) in
[mudler/LocalAI#1836
- ⬆️ Update go-skynet/go-llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1835
- docs(transformers): add docs section about transformers by
[@&#8203;mudler](https://togithub.com/mudler) in
[mudler/LocalAI#1841
- ⬆️ Update mudler/go-piper by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1844
- ⬆️ Update ggerganov/llama.cpp by
[@&#8203;localai-bot](https://togithub.com/localai-bot) in
[mudler/LocalAI#1840

#### New Contributors

- [@&#8203;golgeek](https://togithub.com/golgeek) made their first
contribution in
[mudler/LocalAI#1749
- [@&#8203;Joshhua5](https://togithub.com/Joshhua5) made their first
contribution in
[mudler/LocalAI#1765
- [@&#8203;ouxs-19](https://togithub.com/ouxs-19) made their first
contribution in
[mudler/LocalAI#1778
- [@&#8203;TwinFinz](https://togithub.com/TwinFinz) made their first
contribution in
[mudler/LocalAI#1797
- [@&#8203;cryptk](https://togithub.com/cryptk) made their first
contribution in
[mudler/LocalAI#1828
- [@&#8203;fakezeta](https://togithub.com/fakezeta) made their first
contribution in
[mudler/LocalAI#1823

Thank you to all contributors and users for your continued support and
feedback, making LocalAI better with each release!

**Full Changelog**:
mudler/LocalAI@v2.9.0...v2.10.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNTAuMSIsInVwZGF0ZWRJblZlciI6IjM3LjI1MC4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIn0=-->
@dave-gray101 dave-gray101 deleted the sec-disable-trust-remote-code branch April 6, 2024 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants