Minimum RAM?

[ghost]

I have 16GB of RAM, and I'm running Ubuntu 22.04.2.
I've ingested only the included state_of_the_union.txt document.
I've even taken context tokens down to 500, 100, and 10.

Starting up requires about 10.5GB of RAM to get to the prompt. But asking any query results in running out of memory and a crash. This is with absolutely everything shutdown aside from PrivateGPT running in terminal.

What is the minimum RAM required for the default installation instructions listed in the readme?

[thekit]

I didn't change my context token settings and was pegging out at 15gb of ram used and a lot of disk activity. I could get answers some of he time with this set up using state of the union. There were a lot of warnings.

[Therealkorris]

I ran some tests today with 16GB I was at 98% would also love to know the requirements, also if anyone has any tips on how to make it answer faster, that would be great.

[ghost]

I increased my swap file size from 500MB to 4GB for a total of 20GB of memory and I managed to get a response to my initial 8 token question "Why was the NATO Alliance created?" with a context of 1000 tokens. In total this used 19.9GB of data including 2GB for the operating system.

A second question triggers the typical segmentation fault and core dump.

[quantumalchemy]

fyi .. wanted to test this on my old laptop carbon x1 thinkpad 6 gen i5 with just 8gb ram running Debian variant (mx linux)
with swap increased: vm.swappiness=60 and was pretty good response with ingest taking just 10sec and privateGPT.py responce at 21sec
upping swap was the cure . -- This is the Coolest llm project!