Add HF embedings, add custom tuned prompts, add GPU accel #45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
update gitignore add multithreading
# Conflicts: # startLLM.py
# Conflicts: # README.md
fix empty MODEL_STOP add CHAIN_TYPE in env
…he default ones - update vic7b to 5.1 - add instructions for GPU support
# Conflicts: # startLLM.py
hippalectryon-0
changed the title
fix GPU instructions
hippalectryon-0
changed the title
su77ungr
approved these changes
May 15, 2023
There was a problem hiding this comment.
I edited the prompt to tell the model to say it
There was a problem hiding this comment.
Nice, did not know this even exists. Gonna ping me docs for later dev
su77ungr
added a commit
that referenced
this pull request
May 15, 2023
* [IDEA] Auto convert MODEL_PATH model in .env (#19) * Create convert.py Has to be actively run Checks model type of MODEL_PATH's model from .env and if is old ggml type, converts to ggjt and closes. If already ggjt, it just exits. The code to check the model type is on line 863 "def lazy_load_file(path: Path) -> ModelPlus:" Could be interesting if you could call it automatically from startllm and then after conversion change the name of your old file to oldmodel.bin and the new file to what you have in .env and then use it. * Update README.md --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Better .env with more settings (#20) * Improve .env with introducing model_stop & better documentation * Update example.env --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Create Dockerfile * Update Dockerfile * add docker support * Update README.md * Update README.md * Update README.md * fixed parsing error for literat linebreak+ added missing .bin suffix * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update README.md * simplify+change default LLM refering to #24 * added .epub, .html support see #24 ; might be missing unstructured module * Update requirements.txt see #24 * Create docker-image.yml * Update README.md * Update README.md * Delete docker-image.yml * Create docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update Dockerfile * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update example.env * Update docker-image.yml * Update README.md * Update docker-image.yml * Update README.md * Update README.md * Update docker-image.yml * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update Dockerfile * Update Dockerfile * Update README.md * Update README.md * Update Dockerfile * Simplify Dockerfile * Update Dockerfile * Update README.md * Update README.md * Update README.md * Update docker-image.yml * GUI SUPPORT (#21) * GUI SUPPORT Run `streamlit run .\gui.py` to use it. * Preload models only once instead of every query. Text input doesnt disable like I tried to do after a query. * Preload models + better interface Still needs some work. * specified version of streamlit else None * Working version: Added loading message and improved UI Still need to do cleanup, but should perform properly. * Code cleanup for commit * Faster responses + GUI | Run: streamlit run .\gui.py * Edit .env directly in GUI --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Update README.md * Update README.md * Update README.md * Update README.md * Stable GUI (#29) + Way better qa_system initialization. + Tweaked UI * Add poetry conf (#32) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * clean for merge * cleanup * Add pre-config, fix convert.py, (#35) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * fix ingest.py (#37) * added list to .loader -> multiple files support --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * fix Dockerfile and README.md for streamlit (#40) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * fix README.md --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Update README.md * changed workflow to stable * Update README.md * Merge hippa branch (#41) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * fix README.md * cleaner document handling in ingest.py remove CI * add support for ptt, docx * add sample documents clean gitignore bump package version * load env variables in centralized file load more variables from env lint * remove CI on merge --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * removed suppress code analysis warnings this lets github flag the repo as HTML codebase. which alters discoverability * fix the flag as html * html error already fixed * Allow appending ingest to existing db (#43) *Don't recreate new db each time we run ingest *Print progress of ingestion *Add USE_MLOCK as env arg *Skip prompt if empty *lint closes #42 --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * README cleanup (#44) modified: README.md Edit the README .env to match the new example.env format Edit typos * Add HF embedings, add custom tuned prompts, add GPU accel (#45) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * fix README.md * cleaner document handling in ingest.py remove CI * add support for ptt, docx * add sample documents clean gitignore bump package version * load env variables in centralized file load more variables from env lint * remove CI on merge * check for empty query * print embedding progress allow reusing collection add env USE_MLOCK * fix model_stop * fix model_stop * several minor improvements to startLLM.py fix empty MODEL_STOP add CHAIN_TYPE in env * pre-commit formatting * Add support for HuggingFace embeddings * - add custom prompt templates tailored for vic7b-5, and better than the default ones - update vic7b to 5.1 - add instructions for GPU support * update example.env * update prompts * fix typo * fix typo * update example.env * re-add strip * Add N_GPU_LAYERS to .env fix GPU instructions --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Added document loaders for mail file types. (#48) Co-authored-by: PATRICK GEBERT <patrick.gebert@etecture.de> --------- Co-authored-by: alxspiker <alxtheprogrammer@gmail.com> Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> Co-authored-by: mlynar-czyk <64000170+mlynar-czyk@users.noreply.github.com> Co-authored-by: Patrick Gebert <patrick.gebert@web.de> Co-authored-by: PATRICK GEBERT <patrick.gebert@etecture.de>
su77ungr
added a commit
that referenced
this pull request
May 18, 2023
* [IDEA] Auto convert MODEL_PATH model in .env (#19) * Create convert.py Has to be actively run Checks model type of MODEL_PATH's model from .env and if is old ggml type, converts to ggjt and closes. If already ggjt, it just exits. The code to check the model type is on line 863 "def lazy_load_file(path: Path) -> ModelPlus:" Could be interesting if you could call it automatically from startllm and then after conversion change the name of your old file to oldmodel.bin and the new file to what you have in .env and then use it. * Update README.md --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Better .env with more settings (#20) * Improve .env with introducing model_stop & better documentation * Update example.env --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Create Dockerfile * Update Dockerfile * add docker support * Update README.md * Update README.md * Update README.md * fixed parsing error for literat linebreak+ added missing .bin suffix * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update README.md * simplify+change default LLM refering to #24 * added .epub, .html support see #24 ; might be missing unstructured module * Update requirements.txt see #24 * Create docker-image.yml * Update README.md * Update README.md * Delete docker-image.yml * Create docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update Dockerfile * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update docker-image.yml * Update example.env * Update docker-image.yml * Update README.md * Update docker-image.yml * Update README.md * Update README.md * Update docker-image.yml * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update Dockerfile * Update Dockerfile * Update README.md * Update README.md * Update Dockerfile * Simplify Dockerfile * Update Dockerfile * Update README.md * Update README.md * Update README.md * Update docker-image.yml * GUI SUPPORT (#21) * GUI SUPPORT Run `streamlit run .\gui.py` to use it. * Preload models only once instead of every query. Text input doesnt disable like I tried to do after a query. * Preload models + better interface Still needs some work. * specified version of streamlit else None * Working version: Added loading message and improved UI Still need to do cleanup, but should perform properly. * Code cleanup for commit * Faster responses + GUI | Run: streamlit run .\gui.py * Edit .env directly in GUI --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Update README.md * Update README.md * Update README.md * Update README.md * Stable GUI (#29) + Way better qa_system initialization. + Tweaked UI * Add poetry conf (#32) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * clean for merge * cleanup * Add pre-config, fix convert.py, (#35) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * fix ingest.py (#37) * added list to .loader -> multiple files support --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * fix Dockerfile and README.md for streamlit (#40) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * fix README.md --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Update README.md * changed workflow to stable * Update README.md * Merge hippa branch (#41) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * fix README.md * cleaner document handling in ingest.py remove CI * add support for ptt, docx * add sample documents clean gitignore bump package version * load env variables in centralized file load more variables from env lint * remove CI on merge --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * removed suppress code analysis warnings this lets github flag the repo as HTML codebase. which alters discoverability * fix the flag as html * html error already fixed * Allow appending ingest to existing db (#43) *Don't recreate new db each time we run ingest *Print progress of ingestion *Add USE_MLOCK as env arg *Skip prompt if empty *lint closes #42 --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * README cleanup (#44) modified: README.md Edit the README .env to match the new example.env format Edit typos * Add HF embedings, add custom tuned prompts, add GPU accel (#45) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * clean for merge * fix README.md * update README.md for new convert * redirect to main repo * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * fix README.md * cleaner document handling in ingest.py remove CI * add support for ptt, docx * add sample documents clean gitignore bump package version * load env variables in centralized file load more variables from env lint * remove CI on merge * check for empty query * print embedding progress allow reusing collection add env USE_MLOCK * fix model_stop * fix model_stop * several minor improvements to startLLM.py fix empty MODEL_STOP add CHAIN_TYPE in env * pre-commit formatting * Add support for HuggingFace embeddings * - add custom prompt templates tailored for vic7b-5, and better than the default ones - update vic7b to 5.1 - add instructions for GPU support * update example.env * update prompts * fix typo * fix typo * update example.env * re-add strip * Add N_GPU_LAYERS to .env fix GPU instructions --------- Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> * Added document loaders for mail file types. (#48) Co-authored-by: PATRICK GEBERT <patrick.gebert@etecture.de> * Adding missing dependency (#53) * Added document loaders for mail file types. * Updated dependencies for mail document loader. --------- Co-authored-by: PATRICK GEBERT <patrick.gebert@etecture.de> * Revert "Adding missing dependency (#53)" (#54) This reverts commit cc8cda6. * Move to module, better terminal output, libgen script (#50) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * cleaner document handling in ingest.py remove CI * add support for ptt, docx * add sample documents clean gitignore bump package version * load env variables in centralized file load more variables from env lint * check for empty query * print embedding progress allow reusing collection add env USE_MLOCK * fix model_stop * several minor improvements to startLLM.py fix empty MODEL_STOP add CHAIN_TYPE in env * pre-commit formatting * Add support for HuggingFace embeddings * - add custom prompt templates tailored for vic7b-5, and better than the default ones - update vic7b to 5.1 - add instructions for GPU support * update example.env * update prompts * fix typo * fix typo * update example.env * re-add strip * Add N_GPU_LAYERS to .env fix GPU instructions * move to package * refactor ingest.py as class, add saving on the fly * refactor ingest.py as class, add saving on the fly * refactor ingest.py as class, add saving on the fly * add script to ask questions to libgen * format prompts and text in startLLM.py * add formatting to ask_libgen.py * add formatting to ingest.py * handle errors * escape strings * fix logger level * merge main * update README.md * clean for merge * fix link * fixed-extract-msg (#55) * fixed extract-msg revert * sorry for inconvenience i'm remote * added runtime :stable badge * Fix gui (#58) * update requirements.txt * remove state_of_the_union.txt update gitignore add multithreading * add poetry config * update streamlit version * update Dockerfile * update Dockerfile * fix Dockerfile * update Dockerfile * update README.md * update README.md * update convert.py & pyproject.toml * add tokenizer model * update README & lint * add pre-commit * run pre-commit * fix README.md * fix (?) convert.py * fix (?) convert.py * fix package versions * fix ingest.py * rollback README.md * fix Dockerfile and README.md for streamlit * cleaner document handling in ingest.py remove CI * add support for ptt, docx * add sample documents clean gitignore bump package version * load env variables in centralized file load more variables from env lint * check for empty query * print embedding progress allow reusing collection add env USE_MLOCK * fix model_stop * several minor improvements to startLLM.py fix empty MODEL_STOP add CHAIN_TYPE in env * pre-commit formatting * Add support for HuggingFace embeddings * - add custom prompt templates tailored for vic7b-5, and better than the default ones - update vic7b to 5.1 - add instructions for GPU support * update example.env * update prompts * fix typo * fix typo * update example.env * re-add strip * Add N_GPU_LAYERS to .env fix GPU instructions * move to package * refactor ingest.py as class, add saving on the fly * refactor ingest.py as class, add saving on the fly * refactor ingest.py as class, add saving on the fly * add script to ask questions to libgen * format prompts and text in startLLM.py * add formatting to ask_libgen.py * add formatting to ingest.py * handle errors * escape strings * fix logger level * merge main * update README.md * clean for merge * fix link * Refactor the GUI * add pypandoc_binary dep (#60) * update readme * fix pypandoc * revert README.md * automatically download models from HF (#61) * automatically download models from HF split load_env into load_env+utils * add parameter for number of docs to get * update env * fix formatting * fix default download * put downloaded model in local * fix downloading from datasets * fix symlinks * remove q4 model from readme, remove obsolete file * fixed link to samples * Revert "automatically download models from HF (#61)" (#64) This reverts commit ccdf849. * Fixed HF download + gui stable (#65) * Update load_env.py * Create utils.py * Update gui.py * Update startLLM.py * Update ask_libgen.py * Update ingest.py * Delete meta.json * Update README.md * broken link * Fix gui+dataset (#67) * fix gui import * fix dataset import * missed updating example.env * Better chain (#70) * put n_gpu_layers in args thanks to new langchain version * basic chain + fix get_num_tokens on llama + fix text formatting in HTML error * add chain to startLLM + add MODEL_MAX_TOKENS parameter * fix formatting issue in print_HTML * tweak prompt * add betterrefine * small fix * fix default chain * fix embedding with llama + model download path (#72) * fix embedding with llama * fix downloaded model path * Add issue templates (#73) * add bug template from langchain- wip * update bug-report.yml * add config.yml * update bug-report.yml * add feature-request.yml * add documentation.yml other.yml * update bug-report.yml (#78) * add update note * Better docker file (#80) * update Dockerfile * python >=3.10 * update lockfile * update Dockerfile update project version * update Dockerfile * fix gui in docker * fix for main * move files around (#81) * fix empty doc (#82) * Parallelize ingestion (#85) * fix empty doc * ingest multithreading * better multithreading * change default val * change default val * add .doc and .ppt support (#86) * add .doc and .ppt support * update lock * Update README.md * fix mp warning (#88) * added escape characters * Fix html formatting (#90) * fix formatting * fix formatting * Update README.md * Fix docker (#92) * Better docker image, update readme instructions * improved docker images * Update README.md --------- Co-authored-by: alxspiker <alxtheprogrammer@gmail.com> Co-authored-by: su77ungr <69374354+su77ungr@users.noreply.github.com> Co-authored-by: mlynar-czyk <64000170+mlynar-czyk@users.noreply.github.com> Co-authored-by: Patrick Gebert <patrick.gebert@web.de> Co-authored-by: PATRICK GEBERT <patrick.gebert@etecture.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Major:
TEXT_EMBEDDINGS_MODEL_TYPEto .envvic7b-5.1, and also run better than default prompts imo, and have better stop wordsCHAIN_TYPEto .envMODEL_STOPaccordinglyN_GPU_LAYERSin envMinor: