Skip to content

Commit

Permalink
Merge pull request #341 from ymcui/langchain_demo
Browse files Browse the repository at this point in the history 
update readme (add information of web-demo and LangChain-demo)
  • Loading branch information
@ymcui
ymcui committed May 16, 2023
2 parents a78ec2d + b041ff4 commit ecae290
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 14 deletions.
4 changes: 3 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,11 @@ Facebook官方发布的[LLaMA模型禁止商用](https://github.com/facebookrese
| text-generation-webui | 不适合chat模式 | 使用`--cpu`可在无显卡形式下运行，若生成内容不满意，建议修改prompt |
| LlamaChat | 加载模型时选择"LLaMA" | 加载模型时选择"Alpaca" |
| [HF推理代码](./scripts/inference_hf.py) | 无需添加额外启动参数 | 启动时添加参数 `--with_prompt` |
| [web-demo代码](./scripts/gradio_demo.py) | 不适用 | 直接提供Alpaca模型位置即可；支持多轮对话 |
| [LangChain示例](./scripts/langchain_demo) | 不适用 | 直接提供Alpaca模型位置即可 |
| 已知问题 | 如果不控制终止，则会一直写下去，直到达到输出长度上限。<sup>[2]</sup> | 目前版本模型生成的文本长度相对短一些，比较惜字如金。可在指令中要求详细回答。<sup>[2]</sup> |

*[1] llama.cpp/LlamaChat/[HF推理代码](./scripts/inference_hf.py)等已内嵌，无需手动添加模板。*<br/>
*[1] llama.cpp/LlamaChat/[HF推理代码](./scripts/inference_hf.py)/[web-demo代码](./scripts/gradio_demo.py)/[LangChain示例](./scripts/langchain_demo)等已内嵌，无需手动添加模板。*<br/>
*[2] 如果出现了模型回答质量特别低、胡言乱语、不理解问题等情况，请检查是否针对场景使用了正确的模型和正确的启动参数。*


Expand Down
28 changes: 15 additions & 13 deletions README_EN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,21 @@ The following table provides a basic comparison of the Chinese LLaMA and Alpaca

💡 **Plus versions** are trained on more data, which is highly recommended for use.

| Comparison Item | Chinese LLaMA | Chinese Alpaca |
| ---------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| Training Method | Traditional CLM (trained on general corpus) | Instruction Fine-tuning (trained on instruction data) |
| Input Template | Not required | Must meet template requirements<sup>[1]</sup> |
| Suitable Scenarios ✔️ | Text continuation: Given a context, let the model continue writing | 1. Instruction understanding (Q&A, writing, advice, etc.)<br/>2. Multi-turn context understanding (chat, etc.) |
| Unsuitable Scenarios ❌ | Instruction understanding, multi-turn chat, etc. | Unrestricted free text generation |
| llama.cpp | Use `-p` parameter to specify context | Use `-ins` parameter to enable instruction understanding + chat mode |
| text-generation-webui | Not suitable for chat mode | Use `--cpu` to run without a GPU; if not satisfied with generated content, consider modifying prompt |
| LlamaChat | Choose "LLaMA" when loading the model | Choose "Alpaca" when loading the model |
| inference_hf.py | No additional startup parameters required | Add `--with_prompt` parameter when launching |
| Known Issues | If not controlled for termination, it will continue writing until reaching the output length limit.<sup>[2]</sup> | Current version of the model generates relatively shorter texts, being more concise.<sup>[2]</sup> |

*[1] Templates are built-in for (llama.cpp/LlamaChat/[inference_hf.py](./scripts/inference_hf.py).*
| Comparison Item | Chinese LLaMA | Chinese Alpaca |
| -------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| Training Method | Traditional CLM (trained on general corpus) | Instruction Fine-tuning (trained on instruction data) |
| Input Template | Not required | Must meet template requirements<sup>[1]</sup> |
| Suitable Scenarios ✔️ | Text continuation: Given a context, let the model continue writing | 1. Instruction understanding (Q&A, writing, advice, etc.)<br/>2. Multi-turn context understanding (chat, etc.) |
| Unsuitable Scenarios ❌ | Instruction understanding, multi-turn chat, etc. | Unrestricted free text generation |
| llama.cpp | Use `-p` parameter to specify context | Use `-ins` parameter to enable instruction understanding + chat mode |
| text-generation-webui | Not suitable for chat mode | Use `--cpu` to run without a GPU; if not satisfied with generated content, consider modifying prompt |
| LlamaChat | Choose "LLaMA" when loading the model | Choose "Alpaca" when loading the model |
| [inference_hf.py](./scripts/inference_hf.py) | No additional startup parameters required | Add `--with_prompt` parameter when launching |
| [web-demo](./scripts/gradio_demo.py) | Not applicable | Simply provide the Alpaca model location; support multi-turn conversations |
| [LangChain-demo](./scripts/langchain_demo) | Not applicable | Simply provide the Alpaca model location |
| Known Issues | If not controlled for termination, it will continue writing until reaching the output length limit.<sup>[2]</sup> | Current version of the model generates relatively shorter texts, being more concise.<sup>[2]</sup> |

*[1] Templates are built-in for (llama.cpp/LlamaChat/[inference_hf.py](./scripts/inference_hf.py)/[web-demo](./scripts/gradio_demo.py)/[LangChain-demo](./scripts/langchain_demo).*

*[2] If you encounter issues such as low-quality model responses, nonsensical answers, or failure to understand questions, please check whether you are using the correct model and startup parameters for the scenario.*

Expand Down

0 comments on commit ecae290

Please sign in to comment.