i get the exit code -1073741819 (0xC0000005) while trying to run the basic gpt4all script

[Fabbbr0]

Example Code

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

and this is what i get

Process finished with exit code -1073741819 (0xC0000005)

Your Environment

• Bindings version : 2.2.1 post1
• Operating System: Windows
• Chat model used: orca-mini-3b-gguf2-q4_0.gguf

[cebtenzzre]

I'm going to need more information to debug this.

If you download this file and install it with pip install gpt4all-2.2.1.post1-py3-none-win_amd64.whl, you will have a debug build of the GPT4All bindings.

You should also install procdump.

You'll need to procdump -accepteula first.

Then you should be able to run procdump -e -x . python myscript.py in the same directory as your script.

If your script crashes, procdump should generate a minidump with a .DMP extension in your current directory. If you ZIP that up and send it to me via e-mail (jared@nomic.ai) or Discord (@cebtenzzre), I can inspect the backtrace to see what went wrong.

[Fabbbr0]

I'm going to need more information to debug this.

If you download this file and install it with pip install gpt4all-2.2.1.post1-py3-none-win_amd64.whl, you will have a debug build of the GPT4All bindings.

You should also install procdump.

You'll need to procdump -accepteula first.

Then you should be able to run procdump -e -x . python myscript.py in the same directory as your script.

If your script crashes, procdump should generate a minidump with a .DMP extension in your current directory. If you ZIP that up and send it to me via e-mail (jared@nomic.ai) or Discord (@cebtenzzre), I can inspect the backtrace to see what went wrong.

i tried to run procdump -e -x . python main.py but i got this:

PS D:\kock\python\GtpforAll> .\procdump.exe -e -x . python main.py

ProcDump v11.0 - Sysinternals process dump utility
Copyright (C) 2009-2022 Mark Russinovich and Andrew Richards
Sysinternals - www.sysinternals.com

Process:               python.exe (14728)
CPU threshold:         n/a
Performance counter:   n/a
Commit threshold:      n/a
Threshold seconds:     n/a
Hung window check:     Disabled
Log debug strings:     Disabled
Exception monitor:     Unhandled
Exception filter:      [Includes]
                       *
                       [Excludes]
Terminate monitor:     Disabled
Cloning type:          Disabled
Concurrent limit:      n/a
Avoid outage:          n/a
Number of dumps:       1
Dump folder:           .\
Dump filename/mask:    PROCESSNAME_YYMMDD_HHMMSS
Queue to WER:          Disabled
Kill after dump:       Disabled


Press Ctrl-C to end monitoring without terminating the process.

[14:32:55] Process Exit: PID 14728, Exit Code 0xc0000005
[14:32:56] The process has exited.
[14:32:56] Dump count not reached.

P.S. (im running procdump using .\procdump.exe)

[cebtenzzre]

The problem is that this is the OP's copy of orca-mini-3b-gguf2-q4_0.gguf. It's way too small (22 MiB instead of 1.8 GiB) and is crashing llama.cpp. I can reproduce the crash on Linux as well.

The bug is twofold:

• We are keeping this partial file instead of detecting the incomplete download, either by verifying the filesize, the file hash, or by checking for an interrupted connection
• llama.cpp is segfaulting internally instead of rejecting this file, or at least failing an assertion

[cebtenzzre]

As I discovered in #2154, this class of problem is stupidly easy to reproduce with an empty cache and gpt4all 2.2.1.post1:

>>> x = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')
^C
<snip>
KeyboardInterrupt
>>> x = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')
llama_model_load: error loading model: failed to allocate buffer
llama_load_model_from_file_gpt4all: failed to load model
LLAMA ERROR: failed to load model from /Users/jared/.cache/gpt4all/orca-mini-3b-gguf2-q4_0.gguf

The segfault will still be possible if you pass in a bad model, but writing an incomplete model under the final name should be robustly avoided now.

[cebtenzzre]

The underlying segfault is most likely fixed by ggerganov/llama.cpp#6885. We will update our llama.cpp dependency at some point, and this issue should be fixed at all levels.