Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

i get the exit code -1073741819 (0xC0000005) while trying to run the basic gpt4all script #2128

Closed
Fabbbr0 opened this issue Mar 14, 2024 · 5 comments
Labels
bindings gpt4all-binding issues bug Something isn't working python-bindings gpt4all-bindings Python specific issues

Comments

@Fabbbr0
Copy link

Fabbbr0 commented Mar 14, 2024

Example Code

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

and this is what i get

Process finished with exit code -1073741819 (0xC0000005)

Your Environment

  • Bindings version : 2.2.1 post1
  • Operating System: Windows
  • Chat model used: orca-mini-3b-gguf2-q4_0.gguf
@Fabbbr0 Fabbbr0 added bindings gpt4all-binding issues bug-unconfirmed labels Mar 14, 2024
@cebtenzzre
Copy link
Member

I'm going to need more information to debug this.

If you download this file and install it with pip install gpt4all-2.2.1.post1-py3-none-win_amd64.whl, you will have a debug build of the GPT4All bindings.

You should also install procdump.

You'll need to procdump -accepteula first.

Then you should be able to run procdump -e -x . python myscript.py in the same directory as your script.

If your script crashes, procdump should generate a minidump with a .DMP extension in your current directory. If you ZIP that up and send it to me via e-mail (jared@nomic.ai) or Discord (@cebtenzzre), I can inspect the backtrace to see what went wrong.

@cebtenzzre cebtenzzre added python-bindings gpt4all-bindings Python specific issues windows-wontstart need-info Further information from issue author is requested labels Mar 15, 2024
@Fabbbr0
Copy link
Author

Fabbbr0 commented Mar 18, 2024

I'm going to need more information to debug this.

If you download this file and install it with pip install gpt4all-2.2.1.post1-py3-none-win_amd64.whl, you will have a debug build of the GPT4All bindings.

You should also install procdump.

You'll need to procdump -accepteula first.

Then you should be able to run procdump -e -x . python myscript.py in the same directory as your script.

If your script crashes, procdump should generate a minidump with a .DMP extension in your current directory. If you ZIP that up and send it to me via e-mail (jared@nomic.ai) or Discord (@cebtenzzre), I can inspect the backtrace to see what went wrong.

i tried to run procdump -e -x . python main.py but i got this:

PS D:\kock\python\GtpforAll> .\procdump.exe -e -x . python main.py

ProcDump v11.0 - Sysinternals process dump utility
Copyright (C) 2009-2022 Mark Russinovich and Andrew Richards
Sysinternals - www.sysinternals.com

Process:               python.exe (14728)
CPU threshold:         n/a
Performance counter:   n/a
Commit threshold:      n/a
Threshold seconds:     n/a
Hung window check:     Disabled
Log debug strings:     Disabled
Exception monitor:     Unhandled
Exception filter:      [Includes]
                       *
                       [Excludes]
Terminate monitor:     Disabled
Cloning type:          Disabled
Concurrent limit:      n/a
Avoid outage:          n/a
Number of dumps:       1
Dump folder:           .\
Dump filename/mask:    PROCESSNAME_YYMMDD_HHMMSS
Queue to WER:          Disabled
Kill after dump:       Disabled


Press Ctrl-C to end monitoring without terminating the process.

[14:32:55] Process Exit: PID 14728, Exit Code 0xc0000005
[14:32:56] The process has exited.
[14:32:56] Dump count not reached.

P.S. (im running procdump using .\procdump.exe)

@cebtenzzre cebtenzzre added bug Something isn't working and removed need-info Further information from issue author is requested windows-wontstart bug-unconfirmed labels Mar 18, 2024
@cebtenzzre
Copy link
Member

The problem is that this is the OP's copy of orca-mini-3b-gguf2-q4_0.gguf. It's way too small (22 MiB instead of 1.8 GiB) and is crashing llama.cpp. I can reproduce the crash on Linux as well.

The bug is twofold:

  • We are keeping this partial file instead of detecting the incomplete download, either by verifying the filesize, the file hash, or by checking for an interrupted connection
  • llama.cpp is segfaulting internally instead of rejecting this file, or at least failing an assertion

@cebtenzzre
Copy link
Member

cebtenzzre commented Mar 21, 2024

As I discovered in #2154, this class of problem is stupidly easy to reproduce with an empty cache and gpt4all 2.2.1.post1:

>>> x = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')
^C
<snip>
KeyboardInterrupt
>>> x = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')
llama_model_load: error loading model: failed to allocate buffer
llama_load_model_from_file_gpt4all: failed to load model
LLAMA ERROR: failed to load model from /Users/jared/.cache/gpt4all/orca-mini-3b-gguf2-q4_0.gguf

The segfault will still be possible if you pass in a bad model, but writing an incomplete model under the final name should be robustly avoided now.

@cebtenzzre
Copy link
Member

The underlying segfault is most likely fixed by ggerganov/llama.cpp#6885. We will update our llama.cpp dependency at some point, and this issue should be fixed at all levels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bindings gpt4all-binding issues bug Something isn't working python-bindings gpt4all-bindings Python specific issues
Projects
None yet
Development

No branches or pull requests

2 participants