Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GUI SUPPORT #21

Merged
merged 13 commits into from
May 14, 2023
Merged

GUI SUPPORT #21

merged 13 commits into from
May 14, 2023

Conversation

alxspiker
Copy link
Contributor

First working edition!
Ill continue adding to this PR to make it more complete.
Commit if you want right away, I tested and ran multiple queries.

I want to see if I can get an ingest page to ingest using the GUI too.

Run streamlit run .\gui.py to use it.

image

@alxspiker
GUI SUPPORT
8921deb 
First working edition!
Ill continue adding to this PR to make it more complete.

I want to see if I can get an ingest page to ingest using the GUI too.

Run `streamlit run .\gui.py` to use it.
@alxspiker
Merge branch 'su77ungr:main' into main
9f6b89e
@alxspiker
Copy link
Contributor Author

I know, I am awesome :P

@su77ungr
Copy link
Owner

su77ungr commented May 13, 2023 •
edited

Your development speed is incredible. Give me a break already 🥇

I like the approach without spawning a server.
Edit: should dive into the streamlit module, looks decent.

@alxspiker
Copy link
Contributor Author

Its looking great, just diving right in.

@alxspiker alxspiker mentioned this pull request May 13, 2023
Closed
@su77ungr
Copy link
Owner

You could use the dev branch for your GUI implementation + feel free to bootsrap it into another Dockerfile

@alxspiker
Copy link
Contributor Author

alxspiker commented May 13, 2023 •
edited by su77ungr

I don't know much about docker, so I dont understand the failing check.

@su77ungr su77ungr closed this May 13, 2023
@su77ungr su77ungr reopened this May 13, 2023
@su77ungr
Copy link
Owner

su77ungr commented May 13, 2023 •
edited

This is caused by timeouts when fetching the ggml. HuggingFace does not have stable connections. Idk where GitHub actions are running from. I likely have to remove the wget of the binaries:

Dockerfile should be fixed soon. Just ignore it for now

should be fixed now.

Ah nice we can use this as a dependency check: streamlit is missing

@su77ungr su77ungr closed this May 13, 2023
@su77ungr su77ungr reopened this May 13, 2023
@alxspiker
Copy link
Contributor Author

Works as intended now for chatting with the documents AI. Also learned a lot about steamlit.
P.S. Still needs code cleanup, working on that right now.

@alxspiker
Copy link
Contributor Author

Getting an error on second query:

RuntimeError: Storage folder db is already accessed by another instance of Qdrant client. If you require concurrent access, use Qdrant server instead.

@su77ungr
Copy link
Owner

when running main or inside your implementation?

@alxspiker
Copy link
Contributor Author

Testing both to make sure, got a fix for the GUI, just got to test it.

@alxspiker
Copy link
Contributor Author

Ah, got a small fix coming for startLLM. Seems like I was caching the DB and LLM when I didnt need to, also gui check for user input was incorrect.

@alxspiker
Copy link
Contributor Author

And done!
image

@alxspiker
Copy link
Contributor Author

Tested both console and gui, works as intended.

@alxspiker
Copy link
Contributor Author

alxspiker commented May 13, 2023 •
edited by su77ungr

Adding controls now to change .env on the fly
image

@su77ungr : very neat

@alxspiker
Copy link
Contributor Author

image

@alxspiker
Copy link
Contributor Author

All done for now, edit it as you wish, ill work on the ingest page tonight or tomorrow.

@alxspiker
Copy link
Contributor Author

I still randomly get RuntimeError: Storage folder db is already accessed by another instance of Qdrant client. If you require concurrent access, use Qdrant server instead.

It seems lik 1/5 runs

@su77ungr
Copy link
Owner

su77ungr commented May 14, 2023 •
edited

Perfect. Thanks for the awesome workflow.

I'm setting up a dependencies check and docker workflow so we can prototype without any issues.

Noticed some similiar behaviour, so might switch out both models and run with default arguments. Also I'd like to take a look at the LlamaCpp compiler

see #26 as a first demo.

@su77ungr su77ungr merged commit 14e258f into su77ungr:main May 14, 2023
1 check failed
@imSrbh
Copy link

imSrbh commented May 15, 2023

ModuleNotFoundError: No module named 'streamlit.proto.BackMsg_pb2'
looks like streamlit version issue

su77ungr added a commit that referenced this pull request May 15, 2023
@hippalectryon-0 @alxspiker
@su77ungr @mlynar-czyk @pgebert
Merge main in dev (#51)
038d226 
* [IDEA] Auto convert MODEL_PATH model in .env (#19)

* Create convert.py

Has to be actively run 

Checks model type of MODEL_PATH's model from .env and if is old ggml type, converts to ggjt and closes. If already ggjt, it just exits.

The code to check the model type is on line 863 "def lazy_load_file(path: Path) -> ModelPlus:"

Could be interesting if you could call it automatically from startllm and then after conversion change the name of your old file to oldmodel.bin and the new file to what you have in .env and then use it.

* Update README.md

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Better .env with more settings (#20)

* Improve .env with introducing model_stop & better documentation 

* Update example.env

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Create Dockerfile

* Update Dockerfile

* add docker support

* Update README.md

* Update README.md

* Update README.md

* fixed parsing error for literat linebreak+ added missing .bin suffix

* Update Dockerfile

* Update Dockerfile

* Update Dockerfile

* Update README.md

* simplify+change default LLM 

refering to #24

* added .epub, .html support 

see #24 ; might be missing unstructured module

* Update requirements.txt

see #24

* Create docker-image.yml

* Update README.md

* Update README.md

* Delete docker-image.yml

* Create docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update Dockerfile

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update example.env

* Update docker-image.yml

* Update README.md

* Update docker-image.yml

* Update README.md

* Update README.md

* Update docker-image.yml

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update Dockerfile

* Update Dockerfile

* Update README.md

* Update README.md

* Update Dockerfile

* Simplify Dockerfile

* Update Dockerfile

* Update README.md

* Update README.md

* Update README.md

* Update docker-image.yml

* GUI SUPPORT (#21)

* GUI SUPPORT

Run `streamlit run .\gui.py` to use it.

* Preload models only once instead of every query.

Text input doesnt disable like I tried to do after a query.

* Preload models + better interface

Still needs some work.

* specified version of streamlit else None

* Working version: Added loading message and improved UI

Still need to do cleanup, but should perform properly.

* Code cleanup for commit

* Faster responses + GUI | Run: streamlit run .\gui.py

* Edit .env directly in GUI

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Stable GUI (#29)

+ Way better qa_system initialization.
+ Tweaked UI

* Add poetry conf (#32)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* clean for merge

* cleanup

* Add pre-config, fix convert.py, (#35)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* fix ingest.py (#37)

* added list to .loader -> multiple files support

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* fix Dockerfile and README.md for streamlit (#40)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* fix README.md

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Update README.md

* changed workflow to stable

* Update README.md

* Merge hippa branch (#41)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* fix README.md

* cleaner document handling in ingest.py
remove CI

* add support for ptt, docx

* add sample documents
clean gitignore
bump package version

* load env variables in centralized file
load more variables from env
lint

* remove CI on merge

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* removed suppress code analysis warnings

this lets github flag the repo as HTML codebase. which alters discoverability

* fix the flag as html

* html error already fixed

* Allow appending ingest to existing db (#43)

*Don't recreate new db each time we run ingest

*Print progress of ingestion
*Add USE_MLOCK as env arg
*Skip prompt if empty
*lint
closes #42  

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* README cleanup (#44)

modified:   README.md

Edit the README .env to match the new example.env format
Edit typos

* Add HF embedings, add custom tuned prompts, add GPU accel (#45)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* fix README.md

* cleaner document handling in ingest.py
remove CI

* add support for ptt, docx

* add sample documents
clean gitignore
bump package version

* load env variables in centralized file
load more variables from env
lint

* remove CI on merge

* check for empty query

* print embedding progress
allow reusing collection
add env USE_MLOCK

* fix model_stop

* fix model_stop

* several minor improvements to startLLM.py
fix empty MODEL_STOP
add CHAIN_TYPE in env

* pre-commit formatting

* Add support for HuggingFace embeddings

* - add custom prompt templates tailored for vic7b-5, and better than the default ones
- update vic7b to 5.1
- add instructions for GPU support

* update example.env

* update prompts

* fix typo

* fix typo

* update example.env

* re-add strip

* Add N_GPU_LAYERS to .env
fix GPU instructions

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Added document loaders for mail file types. (#48)

Co-authored-by: PATRICK  GEBERT <patrick.gebert@etecture.de>

---------

Co-authored-by: alxspiker <alxtheprogrammer@gmail.com>
Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>
Co-authored-by: mlynar-czyk <64000170+mlynar-czyk@users.noreply.github.com>
Co-authored-by: Patrick Gebert <patrick.gebert@web.de>
Co-authored-by: PATRICK  GEBERT <patrick.gebert@etecture.de>
su77ungr added a commit that referenced this pull request May 18, 2023
@hippalectryon-0 @alxspiker
@su77ungr @mlynar-czyk @pgebert
Merge main in dev (#94)
fd103b3 
* [IDEA] Auto convert MODEL_PATH model in .env (#19)

* Create convert.py

Has to be actively run 

Checks model type of MODEL_PATH's model from .env and if is old ggml type, converts to ggjt and closes. If already ggjt, it just exits.

The code to check the model type is on line 863 "def lazy_load_file(path: Path) -> ModelPlus:"

Could be interesting if you could call it automatically from startllm and then after conversion change the name of your old file to oldmodel.bin and the new file to what you have in .env and then use it.

* Update README.md

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Better .env with more settings (#20)

* Improve .env with introducing model_stop & better documentation 

* Update example.env

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Create Dockerfile

* Update Dockerfile

* add docker support

* Update README.md

* Update README.md

* Update README.md

* fixed parsing error for literat linebreak+ added missing .bin suffix

* Update Dockerfile

* Update Dockerfile

* Update Dockerfile

* Update README.md

* simplify+change default LLM 

refering to #24

* added .epub, .html support 

see #24 ; might be missing unstructured module

* Update requirements.txt

see #24

* Create docker-image.yml

* Update README.md

* Update README.md

* Delete docker-image.yml

* Create docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update Dockerfile

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* Update example.env

* Update docker-image.yml

* Update README.md

* Update docker-image.yml

* Update README.md

* Update README.md

* Update docker-image.yml

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update Dockerfile

* Update Dockerfile

* Update README.md

* Update README.md

* Update Dockerfile

* Simplify Dockerfile

* Update Dockerfile

* Update README.md

* Update README.md

* Update README.md

* Update docker-image.yml

* GUI SUPPORT (#21)

* GUI SUPPORT

Run `streamlit run .\gui.py` to use it.

* Preload models only once instead of every query.

Text input doesnt disable like I tried to do after a query.

* Preload models + better interface

Still needs some work.

* specified version of streamlit else None

* Working version: Added loading message and improved UI

Still need to do cleanup, but should perform properly.

* Code cleanup for commit

* Faster responses + GUI | Run: streamlit run .\gui.py

* Edit .env directly in GUI

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Stable GUI (#29)

+ Way better qa_system initialization.
+ Tweaked UI

* Add poetry conf (#32)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* clean for merge

* cleanup

* Add pre-config, fix convert.py, (#35)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* fix ingest.py (#37)

* added list to .loader -> multiple files support

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* fix Dockerfile and README.md for streamlit (#40)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* fix README.md

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Update README.md

* changed workflow to stable

* Update README.md

* Merge hippa branch (#41)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* fix README.md

* cleaner document handling in ingest.py
remove CI

* add support for ptt, docx

* add sample documents
clean gitignore
bump package version

* load env variables in centralized file
load more variables from env
lint

* remove CI on merge

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* removed suppress code analysis warnings

this lets github flag the repo as HTML codebase. which alters discoverability

* fix the flag as html

* html error already fixed

* Allow appending ingest to existing db (#43)

*Don't recreate new db each time we run ingest

*Print progress of ingestion
*Add USE_MLOCK as env arg
*Skip prompt if empty
*lint
closes #42  

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* README cleanup (#44)

modified:   README.md

Edit the README .env to match the new example.env format
Edit typos

* Add HF embedings, add custom tuned prompts, add GPU accel (#45)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* clean for merge

* fix README.md

* update README.md for new convert

* redirect to main repo

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* fix README.md

* cleaner document handling in ingest.py
remove CI

* add support for ptt, docx

* add sample documents
clean gitignore
bump package version

* load env variables in centralized file
load more variables from env
lint

* remove CI on merge

* check for empty query

* print embedding progress
allow reusing collection
add env USE_MLOCK

* fix model_stop

* fix model_stop

* several minor improvements to startLLM.py
fix empty MODEL_STOP
add CHAIN_TYPE in env

* pre-commit formatting

* Add support for HuggingFace embeddings

* - add custom prompt templates tailored for vic7b-5, and better than the default ones
- update vic7b to 5.1
- add instructions for GPU support

* update example.env

* update prompts

* fix typo

* fix typo

* update example.env

* re-add strip

* Add N_GPU_LAYERS to .env
fix GPU instructions

---------

Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>

* Added document loaders for mail file types. (#48)

Co-authored-by: PATRICK  GEBERT <patrick.gebert@etecture.de>

* Adding missing dependency (#53)

* Added document loaders for mail file types.

* Updated dependencies for mail document loader.

---------

Co-authored-by: PATRICK  GEBERT <patrick.gebert@etecture.de>

* Revert "Adding missing dependency (#53)" (#54)

This reverts commit cc8cda6.

* Move to module, better terminal output, libgen script (#50)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* cleaner document handling in ingest.py
remove CI

* add support for ptt, docx

* add sample documents
clean gitignore
bump package version

* load env variables in centralized file
load more variables from env
lint

* check for empty query

* print embedding progress
allow reusing collection
add env USE_MLOCK

* fix model_stop

* several minor improvements to startLLM.py
fix empty MODEL_STOP
add CHAIN_TYPE in env

* pre-commit formatting

* Add support for HuggingFace embeddings

* - add custom prompt templates tailored for vic7b-5, and better than the default ones
- update vic7b to 5.1
- add instructions for GPU support

* update example.env

* update prompts

* fix typo

* fix typo

* update example.env

* re-add strip

* Add N_GPU_LAYERS to .env
fix GPU instructions

* move to package

* refactor ingest.py as class, add saving on the fly

* refactor ingest.py as class, add saving on the fly

* refactor ingest.py as class, add saving on the fly

* add script to ask questions to libgen

* format prompts and text in startLLM.py

* add formatting to ask_libgen.py

* add formatting to ingest.py

* handle errors

* escape strings

* fix logger level

* merge main

* update README.md

* clean for merge

* fix link

* fixed-extract-msg (#55)

* fixed extract-msg revert

* sorry for inconvenience i'm remote

* added runtime :stable badge

* Fix gui (#58)

* update requirements.txt

* remove state_of_the_union.txt
update gitignore
add multithreading

* add poetry config

* update streamlit version

* update Dockerfile

* update Dockerfile

* fix Dockerfile

* update Dockerfile

* update README.md

* update README.md

* update convert.py & pyproject.toml

* add tokenizer model

* update README & lint

* add pre-commit

* run pre-commit

* fix README.md

* fix (?) convert.py

* fix (?) convert.py

* fix package versions

* fix ingest.py

* rollback README.md

* fix Dockerfile and README.md for streamlit

* cleaner document handling in ingest.py
remove CI

* add support for ptt, docx

* add sample documents
clean gitignore
bump package version

* load env variables in centralized file
load more variables from env
lint

* check for empty query

* print embedding progress
allow reusing collection
add env USE_MLOCK

* fix model_stop

* several minor improvements to startLLM.py
fix empty MODEL_STOP
add CHAIN_TYPE in env

* pre-commit formatting

* Add support for HuggingFace embeddings

* - add custom prompt templates tailored for vic7b-5, and better than the default ones
- update vic7b to 5.1
- add instructions for GPU support

* update example.env

* update prompts

* fix typo

* fix typo

* update example.env

* re-add strip

* Add N_GPU_LAYERS to .env
fix GPU instructions

* move to package

* refactor ingest.py as class, add saving on the fly

* refactor ingest.py as class, add saving on the fly

* refactor ingest.py as class, add saving on the fly

* add script to ask questions to libgen

* format prompts and text in startLLM.py

* add formatting to ask_libgen.py

* add formatting to ingest.py

* handle errors

* escape strings

* fix logger level

* merge main

* update README.md

* clean for merge

* fix link

* Refactor the GUI

* add pypandoc_binary dep (#60)

* update readme

* fix pypandoc

* revert README.md

* automatically download models from HF (#61)

* automatically download models from HF
split load_env into load_env+utils

* add parameter for number of docs to get

* update env

* fix formatting

* fix default download

* put downloaded model in local

* fix downloading from datasets

* fix symlinks

* remove q4 model from readme, remove obsolete file

* fixed link to samples

* Revert "automatically download models from HF (#61)" (#64)

This reverts commit ccdf849.

* Fixed HF download + gui stable  (#65)

* Update load_env.py

* Create utils.py

* Update gui.py

* Update startLLM.py

* Update ask_libgen.py

* Update ingest.py

* Delete meta.json

* Update README.md

* broken link

* Fix gui+dataset (#67)

* fix gui import

* fix dataset import

* missed updating example.env

* Better chain (#70)

* put n_gpu_layers in args thanks to new langchain version

* basic chain
+ fix get_num_tokens on llama
+ fix text formatting in HTML error

* add chain to startLLM
+ add MODEL_MAX_TOKENS parameter

* fix formatting issue in print_HTML

* tweak prompt

* add betterrefine

* small fix

* fix default chain

* fix embedding with llama + model download path (#72)

* fix embedding with llama

* fix downloaded model path

* Add issue templates (#73)

* add bug template from langchain- wip

* update bug-report.yml

* add config.yml

* update bug-report.yml

* add feature-request.yml

* add documentation.yml other.yml

* update bug-report.yml (#78)

* add update note

* Better docker file (#80)

* update Dockerfile

* python >=3.10

* update lockfile

* update Dockerfile
update project version

* update Dockerfile

* fix gui in docker

* fix for main

* move files around (#81)

* fix empty doc (#82)

* Parallelize ingestion (#85)

* fix empty doc

* ingest multithreading

* better multithreading

* change default val

* change default val

* add .doc and .ppt support (#86)

* add .doc and .ppt support

* update lock

* Update README.md

* fix mp warning (#88)

* added escape characters

* Fix html formatting (#90)

* fix formatting

* fix formatting

* Update README.md

* Fix docker (#92)

* Better docker image, update readme instructions

* improved docker images

* Update README.md

---------

Co-authored-by: alxspiker <alxtheprogrammer@gmail.com>
Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com>
Co-authored-by: mlynar-czyk <64000170+mlynar-czyk@users.noreply.github.com>
Co-authored-by: Patrick Gebert <patrick.gebert@web.de>
Co-authored-by: PATRICK  GEBERT <patrick.gebert@etecture.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@alxspiker @su77ungr @imSrbh