[NeuralChat] Support language detection & translation for RAG chat

[letonghan]

Type of Change

feature
API parameters change for /v1/askdoc/chat.
API added: /v1/askdoc/translate

Description

Support language detection & translation for RAG chat.
Add new translate API for Askdoc.
JIRA ticket: https://jira.devtools.intel.com/browse/NLPTOOLKIU-1285

Expected Behavior & Potential Risk

API /v1/askdoc/chat will be able to detect and translate language automatically.

How has this PR been tested?

Local tested on SPR.
Integrate tested with frontend.

Dependency Change?

None.

[github-actions]

⚡ Required checks status: All passing 🟢

Groups summary

🟢 Format Scan Tests workflow

Check ID	Status	Error details
format-scan (pylint)	success		✅
format-scan (bandit)	success		✅
format-scan (cloc)	success		✅
format-scan (cpplint)	success		✅

These checks are required after the changes to intel_extension_for_transformers/neural_chat/server/restful/retrieval_api.py, intel_extension_for_transformers/neural_chat/tests/ci/server/test_askdoc_server.py.

🟢 NeuralChat Unit Test

Check ID	Status	Error details
neuralchat-unit-test-baseline	success		✅
neuralchat-unit-test-PR-test	success		✅
Generate-NeuralChat-Report	success		✅

These checks are required after the changes to .github/workflows/unit-test-neuralchat.yml, intel_extension_for_transformers/neural_chat/server/restful/retrieval_api.py, intel_extension_for_transformers/neural_chat/tests/ci/server/test_askdoc_server.py.

🟢 Chat Bot Test workflow

Check ID	Status	Error details
call-inference-llama-2-7b-chat-hf / inference test	success		✅
call-inference-mpt-7b-chat / inference test	success		✅

These checks are required after the changes to intel_extension_for_transformers/neural_chat/server/restful/retrieval_api.py.

---

Thank you for your contribution! 💜

Note
This comment is automatically generated and will be updates every 180 seconds within the next 6 hours. If you have any other questions, contact VincyZhang or XuehaoSun for help.