Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Crash: Support old MPT GGUF conversions with duplicated output tensor #2329

Open
dlippold opened this issue May 9, 2024 · 2 comments
Labels
chat gpt4all-chat issues enhancement New feature or request

Comments

@dlippold
Copy link

dlippold commented May 9, 2024

Bug Report

The fine-tuned MPT model from https://huggingface.co/maddes8cht/mosaicml-mpt-7b-instruct-gguf/ in quantization Q4_1 was usabel in release 2.7.2 but not longer in 2.7.3 and later. In particular it is currently not usable.

When I try to load the model file I get the following error message:

Could not load model due to invalid model file for mosaicml-mpt-7b-instruct-Q4_1.gguf

The reason of the problem may have to do with #2006

Steps to Reproduce

  1. Download the model file from the specified URL
  2. Start GPT4all
  3. Choose the downloaded model file

Expected Behavior

The model file should be loaded.

Your Environment

  • GPT4All version: 2.7.2, 2.7.3, 2.7.5
  • Operating System: Ubuntu Linux 22.04.
  • Chat model used (if applicable): see above
@dlippold dlippold added bug-unconfirmed chat gpt4all-chat issues labels May 9, 2024
@cebtenzzre
Copy link
Member

I fixed this upstream in ggerganov/llama.cpp#6139 which should make it into the next release of GPT4All (already included in #2310).

@cebtenzzre cebtenzzre added enhancement New feature or request and removed bug-unconfirmed labels May 9, 2024
@cebtenzzre cebtenzzre changed the title Certain version of MPT GGUF model not usable anymore [Feature] Support old MPT GGUF conversions with duplicated output tensor May 9, 2024
@dlippold
Copy link
Author

Version 2.8.0 crashes when loading the model named above.

@dlippold dlippold changed the title [Feature] Support old MPT GGUF conversions with duplicated output tensor [Feature] Crash: Support old MPT GGUF conversions with duplicated output tensor Jun 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chat gpt4all-chat issues enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants