Segmentation fault (core dumped)

[anamariaUIC]

Hello,

I installed private GPT via:

rm -rf .local .cache /scratch/network/$USER/privateGPT /scratch/network/$USER/tmp
mkdir /scratch/network/$USER/tmp
export TMPDIR=/scratch/network/$USER/tmp
module load Python/3.11.3-GCCcore-12.3.0
pip install --user poetry
module load git
module load CUDA
export PATH=$PATH:/home/$USER/.local/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/$USER/.local/lib
cd /scratch/network/$USER
git clone https://github.com/imartinez/privateGPT.git
cd privateGPT/
export PYTHONPATH=$PYTHONPATH:$PWD
poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"
poetry run python scripts/setup
CMAKE_ARGS='-DLLAMA_CUBLAS=on' poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
PGPT_PROFILES=local make run

When I query a 3 pages long .pdf file I am getting: Segmentation fault (core dumped) error.
Please see in attach details.

How this can be resolved?

[beaconai]

i think your GPU utilization increased

[beaconai]

i'm facing the problem "Initial token count exceed token limit" can anyone help me out, im upload a CVS file and im getting this error.

[anamariaUIC]

I only queried a tiny .txt file one question and attempted to ask a 3 page .pdf one question when I received this segmentation error… Does privateGPT has the run on a GPU? Sent from Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: beaconai ***@***.***> Sent: Monday, April 1, 2024 10:03:26 PM To: zylon-ai/private-gpt ***@***.***> Cc: Sokovic, Ana Marija ***@***.***>; Author ***@***.***> Subject: Re: [zylon-ai/private-gpt] Segmentation fault (core dumped) (Issue #1817) i think your GPU utilization increased — Reply to this email directly, view it on GitHub<#1817 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUHBH5AUACV6HUGRZCXEV4LY3INX5AVCNFSM6AAAAABFSHNKG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQHE4DQOBVGE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[boufaka]

I solve the same issue (segementation fault) changing a file in version 0.5.0:

privateGPT/private_gpt/components/llm/llm_component.py

The setting "offload_kqv": True to "offload_kqv": False

I hope this helps