RAG basically unusable

[SD-fan]

I'm following this project since 0.3.x and love the approach very much. Unfortunately it's just a good idea but especially the RAG implementation is sadly far away from being usable. Here is why:

1. Install docu is far behind and it costs hours to circumvent all obstacles introduced with refactoring in 0.4.x. I'm happy to share details if anyone is interested.

2. Unless one speaks Chinese contributing bugs makes no sense as >70% of the issues are written in Chinese - as well as parts of the inline comments and even the "English" UI. 😕

3. Popular document formats like Excel are unavailable despite the documentation. CSV only works with some work around (on Windows front-end). Adding a website is not supported by crawling but just single URLs can be added one by one.

4. Changing the RAG tokenizer (for non Chinese) is not documented at all. By browsing the code I found adding the model to models_config.py finally helps.

5. Sophisticated and fast exllama-2 is not supported but just slow llama.cpp for local inference.

6. Adding lots of documents for RAG is a big pain as the chunking strategy needs to be selected for each document separately. Unfortunately re-syncing an existing document uses "Automatic" chunking instead of the originally chosen strategy - an annoying bug. Worse: sophisticated semantic chunking using SpaCy is only available "hard coded" if the UI is set to Chinese - with a "Chinese only" model of course.

7. Worst: The database constantly crashes and in turn DB-GPT terminates without any error when adding a batch of documents at once. After a restart the DB does no longer extract/match any documents. Probably the DB is corrupted but you don't get any feedback/error. You need to start all over with a new knowledge space - super frustrating!

All in all this results in a big waste of time not even showing a basic RAG PoC. The project in it's current state (0.4.6) is just far behind from being useful in practice - despite the bold claim "RAG is currently the most practically implemented" in the github intro. Very disappointing ☹️

Sorry, I would have loved to support this project but as non Chinese I feel bumped out purposely. Issues and requests in English are unfortunately ignored by the maintainers. 😥

I'm trying to find my luck with llamaindex now as they rather seem to be very supportive and there is a huge English speaking community in place.

Nevertheless, good luck and all the very best to China and your project. The idea is still great!

[csunny]

Thank you very much for your question. It does point out many problems that exist in the English context. However, we are also working hard to natively support multiple languages, especially English.

In addition, in the early versions, we did iterate very quickly, guiding the recently completed version v0.5.0. Version v0.5.0 is a compatible version that we maintain for a long time, and will address the shortcomings you mentioned.

Thank you again for your attention and questions.

Regarding questions about RAG, please @Aries-ckt for detailed answers and optimization.

[Aries-ckt]

Firstly, we would like to express our sincere gratitude for your continued interest and support for our project since its early versions. Your detailed feedback is incredibly valuable to us, and we genuinely appreciate the time you've taken to share your experiences.

We acknowledge that the RAG needs improvement in English, and we are actively working to update it to ensure a smoother process for everyone. Additionally, we understand the importance of language inclusivity, and we truly apologize for the difficulties you have encountered with the language barrier in our community. We are committed to addressing this issue and making our project more accessible to our international users.

Regarding the first point you mentioned. now db-gpt is installed by db-gpt modules, in our tutorial document will introduced in detail.

Regarding the second point you mentioned. our community will primarily use English for issues going forward, and we aim to respond to issues within one to two days.

Regarding the third point you mentioned. we will support excel and crawl website soon.

Regarding the fourth point you mentioned. we support bge-base-en(English) and bge-base-zh(Chinese) models, and we will illustrate on tutorial in detail.

Regarding the fifth point you mentioned. you can pull a feature in issues and we will integrated in the future.

Regarding the sixth point you mentioned. Currently, each document has its own chunk strategy, and you can opt for automatic sharding, which will automatically select the most suitable chunk strategy based on the attributes of your document. Of course, you can also customize this for individual documents. Regarding your mention of the lack of support for spaCy, we plan to add spaCy and other NLP-related sharding strategies to our chunk options in the future.

Regarding the seventh point you mentioned. It's possible you've encountered a performance issue. To address this, we need to consider your specific environment and the state at which the system was running. Please submit an issue so we can investigate the exact cause.

Lastly, we truly appreciate your feedback and suggestions. We will take each issue seriously and continuously refine our product. Our product will get better, and we are soon welcoming the release of version v0.5.0. We hope you will join us to witness this milestone. Thank you once again.