Skip to content

Chat with your documents offline using AI + MongoDB Atlas Vector Search

License

Notifications You must be signed in to change notification settings

ranfysvalle02/chatdocs-mdb

 
 

Repository files navigation

Chat with your documents offline using AI. No data leaves your system. Internet connection is only required to install the tool and download the AI models. It is based on PrivateGPT and has more features.

Demo

Features

Show supported document types
Extension Format
.csv CSV
.docx, .doc Word Document
.enex EverNote
.eml Email
.epub EPub
.html HTML
.md Markdown
.msg Outlook Message
.odt Open Document Text
.pdf Portable Document Format (PDF)
.pptx, .ppt PowerPoint Document
.txt Text file (UTF-8)

Installation

Install the tool using:

pip uninstall chatdocs  
pip install ./

Download the AI models using:

chatdocs download

Now it can be run offline without internet connection.

Usage

Add a directory containing documents to chat with using:

chatdocs add /path/to/documents

The processed documents will be stored in db directory by default.

Chat with your documents using:

chatdocs ui

Open http://localhost:5000 in your browser to access the web UI.

It also has a nice command-line interface:

chatdocs chat
Show preview

Demo

Configuration

All the configuration options can be changed using the chatdocs.yml config file. Create a chatdocs.yml file in some directory and run all commands from that directory. For reference, see the default chatdocs.yml file.

You don't have to copy the entire file, just add the config options you want to change as it will be merged with the default config. For example, see tests/fixtures/chatdocs.yml which changes only some of the config options.

Embeddings

To change the embeddings model, add and change the following in your chatdocs.yml:

embeddings:
  model: hkunlp/instructor-large

Note: When you change the embeddings model, delete the db directory and add documents again.

C Transformers

To change the C Transformers model, add and change the following in your chatdocs.yml:

ctransformers:
  model: TheBloke/Wizard-Vicuna-7B-Uncensored-GGML
  model_type: llama

Note: When you add a new model for the first time, run chatdocs download to download the model before using it.

You can also use an existing local model file:

ctransformers:
  model: /path/to/ggml-model.bin
  model_type: llama

🤗 Transformers

To use 🤗 Transformers, add the following to your chatdocs.yml:

llm: huggingface

To change the 🤗 Transformers model, add and change the following in your chatdocs.yml:

huggingface:
  model: TheBloke/Wizard-Vicuna-7B-Uncensored-HF

Note: When you add a new model for the first time, run chatdocs download to download the model before using it.

GPU

Embeddings

To enable GPU (CUDA) support for the embeddings model, add the following to your chatdocs.yml:

embeddings:
  model_kwargs:
    device: cuda

You may have to reinstall PyTorch with CUDA enabled by following the instructions here.

C Transformers

Note: Currently only LLaMA GGML models have GPU support.

To enable GPU (CUDA) support for the C Transformers model, add the following to your chatdocs.yml:

ctransformers:
  config:
    gpu_layers: 50

You should also reinstall the ctransformers package with CUDA enabled:

pip uninstall ctransformers --yes
CT_CUBLAS=1 pip install ctransformers --no-binary ctransformers
Show commands for Windows

On Windows PowerShell run:

$env:CT_CUBLAS=1
pip uninstall ctransformers --yes
pip install ctransformers --no-binary ctransformers

On Windows Command Prompt run:

set CT_CUBLAS=1
pip uninstall ctransformers --yes
pip install ctransformers --no-binary ctransformers

🤗 Transformers

To enable GPU (CUDA) support for the 🤗 Transformers model, add the following to your chatdocs.yml:

huggingface:
  device: 0

You may have to reinstall PyTorch with CUDA enabled by following the instructions here.

License

MIT

About

Chat with your documents offline using AI + MongoDB Atlas Vector Search

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 77.8%
  • HTML 20.8%
  • Shell 1.4%