Merge branch 'su77ungr:main' into main

su77ungr · May 17, 2023 · e72bcd5 · e72bcd5
2 parents 9f0f8ae + 5ab6f03
commit e72bcd5
Show file tree

Hide file tree

Showing 24 changed files with 2,030 additions and 533 deletions.
diff --git a/.github/ISSUE_TEMPLATE/bug-report.yml b/.github/ISSUE_TEMPLATE/bug-report.yml
@@ -0,0 +1,111 @@
+name: "\U0001F41B Bug Report"
+description: Submit a bug report to help us improve CASALIOY
+labels: ["02 Bug Report"]
+body:
+  - type: markdown
+    attributes:
+      value: >
+        Thank you for taking the time to file a bug report. Before creating a new
+        issue, please make sure to take a few moments to check the issue tracker
+        for existing issues about the bug.
+
+  - type: textarea
+    id: env
+    attributes:
+      label: .env
+      description: Please share your exact .env file. *format it with ``` as in the example below.*
+      placeholder: |
+        ```
+        # Generic
+        MODEL_N_CTX=1024
+        TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2
+        TEXT_EMBEDDINGS_MODEL_TYPE=HF  # LlamaCpp or HF
+        USE_MLOCK=true
+
+        # Ingestion
+        PERSIST_DIRECTORY=db
+        DOCUMENTS_DIRECTORY=source_documents
+        INGEST_CHUNK_SIZE=500
+        INGEST_CHUNK_OVERLAP=50
+
+        # Generation
+        MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp
+        MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin
+        MODEL_TEMP=0.8
+        MODEL_STOP=[STOP]
+        CHAIN_TYPE=stuff
+        N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db
+        N_FORWARD_DOCUMENTS=6 # How many documents to forward to the LLM, chosen among those retrieved
+        N_GPU_LAYERS=4
+        ```
+
+    validations:
+      required: true
+
+  - type: input
+    id: system-info-python
+    attributes:
+      label: Python version
+      placeholder: python 3.11.3
+    validations:
+      required: true
+  - type: input
+    id: system-info-system
+    attributes:
+      label: System
+      placeholder: Ubuntu-22.04
+    validations:
+      required: true
+  - type: input
+    id: system-info-casalioy
+    attributes:
+      label: CASALIOY version
+      placeholder: A release number (ex. `0.0.8`) or a commit id (ex `13cce0e`)
+    validations:
+      required: true
+
+  - type: checkboxes
+    id: information-scripts-examples
+    attributes:
+      label: Information
+      description: "The problem arises when using:"
+      options:
+        - label: "The official example scripts"
+        - label: "My own modified scripts"
+
+  - type: checkboxes
+    id: related-components
+    attributes:
+      label: Related Components
+      description: "Select the components related to the issue (if applicable):"
+      options:
+        - label: "Document ingestion"
+        - label: "GUI"
+        - label: "Prompt answering"
+
+  - type: textarea
+    id: reproduction
+    validations:
+      required: true
+    attributes:
+      label: Reproduction
+      description: |
+        Please provide a [code sample](https://stackoverflow.com/help/minimal-reproducible-example) that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
+        If you have code snippets, error messages, stack traces please provide them here as well.
+        Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
+        Avoid screenshots when possible, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
+
+      placeholder: |
+        Steps to reproduce the behavior:
+
+          1.
+          2.
+          3.
+
+  - type: textarea
+    id: expected-behavior
+    validations:
+      required: true
+    attributes:
+      label: Expected behavior
+      description: "A clear and concise description of what you would expect to happen."
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1,2 @@
+blank_issues_enabled: true
+version: 2.1
diff --git a/.github/ISSUE_TEMPLATE/documentation.yml b/.github/ISSUE_TEMPLATE/documentation.yml
@@ -0,0 +1,19 @@
+name: Documentation
+description: Report an issue related to the LangChain documentation.
+title: "DOC: <Please write a comprehensive title after the 'DOC: ' prefix>"
+labels: [03 - Documentation]
+
+body:
+- type: textarea
+  attributes:
+    label: "Issue with current documentation:"
+    description: >
+      Please make sure to leave a reference to the document/code you're
+      referring to.
+
+- type: textarea
+  attributes:
+    label: "Idea or request for content:"
+    description: >
+      Please describe as clearly as possible what topics you think are missing
+      from the current documentation.
diff --git a/.github/ISSUE_TEMPLATE/feature-request.yml b/.github/ISSUE_TEMPLATE/feature-request.yml
@@ -0,0 +1,30 @@
+name: "\U0001F680 Feature request"
+description: Submit a proposal/request for a new CASALIOY feature
+labels: ["02 Feature Request"]
+body:
+  - type: textarea
+    id: feature-request
+    validations:
+      required: true
+    attributes:
+      label: Feature request
+      description: |
+        A clear and concise description of the feature proposal. Please provide links to any relevant GitHub repos, papers, or other resources if relevant.
+
+  - type: textarea
+    id: motivation
+    validations:
+      required: true
+    attributes:
+      label: Motivation
+      description: |
+        Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too.
+
+  - type: textarea
+    id: contribution
+    validations:
+      required: true
+    attributes:
+      label: Your contribution
+      description: |
+        Is there any way that you could help, e.g. by submitting a PR?
diff --git a/.github/ISSUE_TEMPLATE/other.yml b/.github/ISSUE_TEMPLATE/other.yml
@@ -0,0 +1,18 @@
+name: Other Issue
+description: Raise an issue that wouldn't be covered by the other templates.
+title: "Issue: <Please write a comprehensive title after the 'Issue: ' prefix>"
+labels: [04 - Other]
+
+body:
+  - type: textarea
+    attributes:
+      label: "Issue you'd like to raise."
+      description: >
+        Please describe the issue you'd like to raise as clearly as possible.
+        Make sure to include any relevant links or references.
+
+  - type: textarea
+    attributes:
+      label: "Suggestion:"
+      description: >
+        Please outline a suggestion to improve the issue here.
diff --git a/Dockerfile b/Dockerfile
@@ -7,6 +7,8 @@ WORKDIR CASALIOY
 RUN pip3 install poetry
 RUN python3 -m poetry config virtualenvs.create false
 RUN python3 -m poetry install
-RUN python3 -m pip install --force streamlit  # Temp fix, see pyproject.toml
+RUN python3 -m pip install --force streamlit sentence_transformers # Temp fix, see pyproject.toml
+RUN python3 -m pip uninstall -y llama-cpp-python
+RUN CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 python3 -m pip install llama-cpp-python  # GPU support
 RUN pre-commit install
 COPY example.env .env
diff --git a/README.md b/README.md
@@ -1,7 +1,9 @@
 <!--suppress HtmlDeprecatedAttribute -->
 <div align="center">
 
-> **NOTICE** NOW WITH
+<a href="https://www.buymeacoffee.com/cassowary" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-white.png" alt="Buy Me A Coffee" height="30" width="140"></a>
+
+**NOW** WITH
 <a href="#chat-inside-gui-new-feature"><img src="https://img.shields.io/badge/GUI-blue.svg" alt="Roadmap 2023">
 <br>
 <p align="center">
@@ -15,8 +17,8 @@
 <img height="300" src="https://github.com/su77ungr/GEEB-GPT/assets/69374354/2e59734c-0de7-4057-be7a-14729e1d5acd" alt="Qdrant"><br>
 
 <a href="https://github.com/su77ungr/CASALIOY/issues/8"><img src="https://img.shields.io/badge/Feature-Requests-bc1439.svg" alt="Roadmap 2023"> [![Docker Pulls](https://badgen.net/docker/pulls/su77ungr/casalioy?icon=docker&label=pulls)](https://hub.docker.com/r/su77ungr/casalioy/)</a>
+![example workflow](https://github.com/su77ungr/CASALIOY/actions/workflows/docker-image.yml/badge.svg)
 
- <a href="https://www.buymeacoffee.com/cassowary" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="30" width="140"></a>
 <br><br>
 </p>
 The fastest toolkit for air-gapped LLMs
@@ -38,17 +40,11 @@ docker pull su77ungr/casalioy:stable
 ```bash
 docker run -it su77ungr/casalioy:stable /bin/bash
 ```
+
 for older docker without GUI use `casalioy:latest` might deprecate soon
 
 > Fetch the default models
 
-```
-cd models
-wget https://huggingface.co/Pi3141/alpaca-native-7B-ggml/resolve/397e872bf4c83f4c642317a5bf65ce84a105786e/ggml-model-q4_0.bin &&
-wget https://huggingface.co/datasets/dnato/ggjt-v1-vic7b-uncensored-q4_0.bin/resolve/main/ggjt-v1-vic7b-uncensored-q4_0.bin
-cd ../
-```
-
 > All set! Proceed with ingesting your [dataset](#ingesting-your-own-dataset)
 
 ### Build it from source
@@ -59,24 +55,26 @@ cd ../
 python -m pip install poetry
 python -m poetry config virtualenvs.in-project true
 python -m poetry install
-python -m pip install --force streamlit  # Temporary bandaid fix, waiting for streamlit >=1.23
 . .venv/bin/activate
+python -m pip install --force streamlit sentence_transformers  # Temporary bandaid fix, waiting for streamlit >=1.23
 pre-commit install
 ```
 
-> Download the 2 models and place them in a folder called `./models`:
+If you want GPU support for llama-ccp:
 
-- LLM: default
-  is [ggjt-v1-vic7b-uncensored-q4_0](https://huggingface.co/datasets/dnato/ggjt-v1-vic7b-uncensored-q4_0.bin/resolve/main/ggjt-v1-vic7b-uncensored-q4_0.bin)
-- Embedding: default
-  to [ggml-model-q4_0](https://huggingface.co/Pi3141/alpaca-native-7B-ggml/resolve/397e872bf4c83f4c642317a5bf65ce84a105786e/ggml-model-q4_0.bin).
+```shell
+pip uninstall -y llama-cpp-python
+CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install --force llama-cpp-python
+```
 
 > > Edit the example.env to fit your models and rename it to .env
 
 ```env
 # Generic
 MODEL_N_CTX=1024
-LLAMA_EMBEDDINGS_MODEL=models/ggml-model-q4_0.bin
+TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2
+TEXT_EMBEDDINGS_MODEL_TYPE=HF  # LlamaCpp or HF
+USE_MLOCK=true
 
 # Ingestion
 PERSIST_DIRECTORY=db
@@ -86,42 +84,53 @@ INGEST_CHUNK_OVERLAP=50
 
 # Generation
 MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp
-MODEL_PATH=models/ggjt-v1-vic7b-uncensored-q4_0.bin
+MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin
 MODEL_TEMP=0.8
-MODEL_STOP=###,\n
+MODEL_STOP=[STOP]
+CHAIN_TYPE=stuff
+N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db
+N_FORWARD_DOCUMENTS=6 # How many documents to forward to the LLM, chosen among those retrieved
 ```
 
 This should look like this
 
 ```
 └── repo
       ├── startLLM.py
-      ├── ingest.py
+      ├── casalioy
+      │   └── ingest.py, load_env.py, startLLM.py, gui.py, ...
       ├── source_documents
       │   └── sample.csv
-      │   └── shor.pdfstate_of_the_union.txt
-      │   └── state_of_the_union.txt
+      │   └── ...
       ├── models
-      │   ├── ggjt-v1-vic7b-uncensored-q4_0.bin
-      │   └── ggml-model-q4_0.bin
+      │   ├── ggml-vic7b-q5_1.bin
+      │   └── ...
       └── .env, convert.py, Dockerfile
 ```
 
+      
+> 👇 Update your installation!
+      
+      
+      git pull && poetry install
+      
+
+      
 ## Ingesting your own dataset
 
-To automatically ingest different data types (.txt, .pdf, .csv, .epub)
+To automatically ingest different data types (.txt, .pdf, .csv, .epub, .html, .docx, .pptx, .eml, .msg)
 
-> This repo includes dummy [files](https://github.com/imartinez/privateGPT/blob/main/source_documents/)
+> This repo includes dummy [files](https://github.com/su77ungr/CASALIOY/tree/main/source_documents)
 > inside `source_documents` to run tests with.
 
 ```shell
-python ingest.py # optional <path_to_your_data_directory>
+python casalioy/ingest.py # optional <path_to_your_data_directory>
 ```
 
 Optional: use `y` flag to purge existing vectorstore and initialize fresh instance
 
 ```shell
-python ingest.py # optional <path_to_your_data_directory> y
+python casalioy/ingest.py # optional <path_to_your_data_directory> y
 ```
 
 This spins up a local qdrant namespace inside the `db` folder containing the local vectorstore. Will take time,
@@ -134,7 +143,7 @@ database. To remove dataset simply remove `db` folder.
 In order to ask a question, run a command like:
 
 ```shell
-python startLLM.py
+python casalioy/startLLM.py
 ```
 
 And wait for the script to require your input.
@@ -157,7 +166,7 @@ Type `exit` to finish the script.
 Introduced by [@alxspiker](https://github.com/alxspiker) -> see [#21](https://github.com/su77ungr/CASALIOY/pull/21)
 
 ```shell
-streamlit run .\gui.py
+streamlit run casalioy/gui.py
 ```
 
 # LLM options
@@ -166,7 +175,6 @@ streamlit run .\gui.py
 
 | Model                                                                                                                                            | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Avg. |
 |:-------------------------------------------------------------------------------------------------------------------------------------------------|:-----:|:----:|:---------:|:----------:|:-----:|:-----:|:----:|:----:|
-| [ggml-vic-7b-uncensored](https://huggingface.co/datasets/dnato/ggjt-v1-vic7b-uncensored-q4_0.bin/resolve/main/ggjt-v1-vic7b-uncensored-q4_0.bin) | 73.4  | 74.8 |   63.4    |    64.7    | 54.9  | 36.0  | 40.2 | 58.2 |
 | [GPT4All-13b-snoozy q5](https://huggingface.co/TheBloke/GPT4All-13B-snoozy-GGML/blob/main/GPT4All-13B-snoozy.ggml.q5_1.bin)                      | 83.3  | 79.2 |   75.0    |    71.3    | 60.9  | 44.2  | 43.4 | 65.3 |
 
 ### models inside of the GPT-J ecosphere
@@ -209,6 +217,7 @@ leaving your environment, and with reasonable performance.
 
 <br><br>
 
+
 # Disclaimer
 
 The contents of this repository are provided "as is" and without warranties of any kind, whether express or implied. We