Too slow responding...

[ivanbodurov]

I copy pasted this:

git clone https://github.com/ohmplatform/FreedomGPT.git
cd FreedomGPT
yarn install
yarn start:prod

and got an endless loading.

[cooperdk]

try executing the chat app in resources/models/(OS) and see what happens...

[ivanbodurov]

Where is for Linux?
Also I did it on Windows 10 and nothing hapened? Can you explain me in details? Maybe I'm missing something.

[cooperdk]

It is a CLI application, so execute it in terminal or command prompt. You will have to direct it to the model location and run it in interactive mode, something like:

chat.exe -m %USERPROFILE%\FreedomGPT\ggml-alpaca-7b-q4.bin -i --color

You will get a prompt once the model is loaded. You should tell it what to do there.

For Linux, I don't know where it is located. You should have installed the chat compiled from alpaca.cpp in a bin directory somewhere...
probably in /usr/bin/chat but who knows.

[ivanbodurov]

C:\Users\IvanB\AppData\Local\FreedomGPT\app-1.1.2\resources\models\windows> chat.exe -m C:\Users\IvanB\FreedomGPT\ggml-alpaca-7b-q4.bin -i --color

Still nothing..

[ivanbodurov]

Can you help me? Why I got nothing?

[cooperdk]

Can you help me? Why I got nothing?

You don't get a response at all?
Do you get the opportunity back to write a new command, or does it hang?

You may need to tell us your computer specifications.
Maybe your pc cannot handle this.

[ivanbodurov]

That is:
CPU: Intel(R) Pentium(R) Gold G5400 CPU @ 3.70GHz 3.70 GHz
GPU: nVidia Geforce GTX 1650 4GB
RAM: 8,00 GB

System type: 64-bit, processor x64

[cooperdk]

Yeah, then you should hold your breath. For a long time.
With those specs it will take a lot of time for the model to be ready and it will be so slow I am almost going to say, don't bother.
Your CPU is a 2 core with 4 threads. And you have 8 GB of ram which is almost used by just running Windows.
Even in Linux, your system will struggle with a 4 GB model af it also needs ram to do the work besides the 4 GB the smallest model requires.

My suggestion would be to get one of the last two generations of i7 or i9. And at least 32 GB ram, at the bare minimum 16. These models will run ok with those specifications, it's what I do.

But your specifications are so low that this won't be funny if it even works.

[cooperdk]

Yeah, then you should hold your breath. For a long time.
With those specs it will take a lot of time for the model to be ready and it will be so slow I am almost going to say, don't bother.
Your CPU is a 2 core with 4 threads. And you have 8 GB of ram which is almost used by just running Windows.
Even in Linux, your system will struggle with a 4 GB model af it also needs ram to do the work besides the 4 GB the smallest model requires.

The GPU wouldn't even be able to handle this model if GPI was supported by the alpaca program.

My suggestion would be to get one of the last two generations of i7 or i9. And at least 32 GB ram, at the bare minimum 16. These models will run ok with those specifications, it's what I do.

But your specifications are so low that this won't be funny if it even works.

[Aporie]

I get the exact same problem with a i7 8th gen, GeForce GTX 1050 Ti, 16Gb ram.
I don't see any error under any tab on localhost:9000, but I'm running it on linux.
Will let it run for a while to see if something happens, but from what I've already tried, 5-10min still no replies.
Under http://localhost:8889/ and http://localhost:3000 I get:
Cannot GET /
But under renderers I have: webpack-dev-server - 404s will fallback to '/index.html'

Dunno if there is any issue there.

[zouqi1987]

Same issue. It can respond, but at a super laggy pace.....
2.6 GHz 6-Core Intel Core i7
AMD Radeon Pro 5300M 4 GB
Intel UHD Graphics 630 1536 MB
16 GB 2667 MHz DDR4

Start the program by yarn start:prod

[ivanbodurov]

Yes and therefore when FreedomGPT.com will be fixed??

[VerifiedStoic]

GTX 3070 8GB with Ryzen 7 and 16GB RAM. Rarely, it will respond quickly. But the majority of time, it never answers. I've left it overnight and still no response. Need fixed please.

[cooperdk]

GTX 3070 8GB with Ryzen 7 and 16GB RAM. Rarely, it will respond quickly. But the majority of time, it never answers. I've left it overnight and still no response. Need fixed please.

Why fix when it is likely your computer specifications that are not good enough? 16 GB ram is not much for AI. And Alpaca does NOT support GPU. I would get at least 16 more.
That said, the problem likely lies elsewhere anyway. The FreedomGPT is a simple graphic interface for Alpaca, a console program to process the AI model.

I would advise you to test that program (it is in the resources/models subdir).

[ItsPi3141]

That is:
CPU: Intel(R) Pentium(R) Gold G5400 CPU @ 3.70GHz 3.70 GHz
GPU: nVidia Geforce GTX 1650 4GB
RAM: 8,00 GB

System type: 64-bit, processor x64

Based on that spec and comparing it to my laptop (100 words in 45 minutes), your computer is probably gonna take like an hour to generate 100 words.

[aristotle999]

Sorry, it's unusably slow. I have an Intel i7-12700F, 32 GB RAM, GeForce GTX 1080Ti. I'm not going to leave it on overnight to wait for an answer. Disappointed.

[ItsPi3141]

Sorry, it's unusably slow. I have an Intel i7-12700F, 32 GB RAM, GeForce GTX 1080Ti. I'm not going to leave it on overnight to wait for an answer. Disappointed.

Then that's just your CPU and storage. The app itself isn't slow. Do you have an HDD or an SSD for your OS drive?

[Amalreji111]

Sorry, it's unusably slow. I have an Intel i7-12700F, 32 GB RAM, GeForce GTX 1080Ti. I'm not going to leave it on overnight to wait for an answer. Disappointed.

Can you check the cpu amd gpu utilisation?. For me it's 100% hence it's very slow.

[ryarasi]

Intel Core I5-12600k Desktop Processor
Crucial Ballistix MAX 32GB Kit (2 x 16GB) DDR4 4000 Memory
WD_BLACK 1TB SN750 SE NVMe Internal Gaming SSD Solid State Drive - Gen4 PCIe, M.2 2280, Up to 3,600 MB/s -
GPU | ZOTAC | ZOTAC NVIDIA GeForce RTX 3090 Trinity OC

Still facing an endless loading on the desktop version on Windows 11