Supporting different languages

[nponeccop]

Duplicates

• I have searched the existing issues

Summary 💡

A few persons asked for a guide to porting ChatGPT to speak and think in Chinese, but I guess it applies to other languages as well. I personally can supervise Spanish and Ukrainian.

Note that there is a different task of changing the UI to be in other languages. This task is only about the replacement of core prompts.

What we need is fully engineered prompts in different languages, and the ability to select between them.

So, the goal of this issue is to prepare a guide and a code framework for such porting.

Examples 🌈

You can end your goals with "speak in Spanish", and I guess there were some language flags as well

Motivation 🔦

ChatGPT speaks many natural and artificial languages, and it speaks popular languages speaks pretty well. So getting Chinese, Spanish, Hindi, and Arabic support is essential in order for the project to get more testers and contributors.

Or any other relatively popular languages as I guess "what do you mean by Chinese" is a valid question.

[RiteshK-611]

I would like to help.
I can help adding Hindi support.

[nponeccop]

I guess you can get us the prompt, and then we integrate it. But for now we need to extract the language-specific parts and move into configs

[Jun10ng]

#1629

hi. @nponeccop Here are my thinks about i18n.
We can add search in multi-language first.

#1879
here are another PR about i18n, it support ai logging in multi language.

[abel9851]

could i help?
i can help korean support.

[frenchdevartist]

Hi,
I can help for the french version.
It would be great to have a guide of the application architecture.
I think there are two main topics :

• internationalize the tool --> not very relevant
• internationalize searches and outputs of the application --> very relevant

[kaiboesefeldt]

Hi there!

I would be more than happy to help you with the German language and development tasks. As I noticed that this issue has been labeled as a "good first issue," I would love to dedicate a few hours of my free time each week to working on it.

I understand that this issue also requires some discussion. Here are my thoughts on the requirements:

1. The language of the prompts and outputs should be customizable.
2. Multiple languages should be supported.
3. For each supported language, there should be a file in YAML format that contains prompt snippets and templates in the respective language.
4. These files should be stored in a designated folder.
5. The actual language used will be defined in the .env file, with English as the default.

Regarding the technical implementation, it appears that the text parts of the prompts are spread across the codebase. Therefore, it would make sense to identify them and provide them in a central location as a first step. Then, the prompt parts would be transferred to a YAML file with the language abbreviation in its name (e.g., prompt-de-de.yaml). Finally, the prompt file that corresponds to the configuration in the .env file will be read.

What do you think about my suggestions?

As for the next steps, I'm not quite sure how I can contact the active contributors to this project, aside from submitting issues and pull requests. Would it be best for me to post a PR for an implementation? Please let me know.

Thank you!

[PalAditya]

Hi there!

I would be more than happy to help you with the German language and development tasks. As I noticed that this issue has been labeled as a "good first issue," I would love to dedicate a few hours of my free time each week to working on it.

I understand that this issue also requires some discussion. Here are my thoughts on the requirements:

1. The language of the prompts and outputs should be customizable.
2. Multiple languages should be supported.
3. For each supported language, there should be a file in YAML format that contains prompt snippets and templates in the respective language.
4. These files should be stored in a designated folder.
5. The actual language used will be defined in the .env file, with English as the default.

Regarding the technical implementation, it appears that the text parts of the prompts are spread across the codebase. Therefore, it would make sense to identify them and provide them in a central location as a first step. Then, the prompt parts would be transferred to a YAML file with the language abbreviation in its name (e.g., prompt-de-de.yaml). Finally, the prompt file that corresponds to the configuration in the .env file will be read.

What do you think about my suggestions?

As for the next steps, I'm not quite sure how I can contact the active contributors to this project, aside from submitting issues and pull requests. Would it be best for me to post a PR for an implementation? Please let me know.

Thank you!

+1. Would like to help with the same, and if just language support is needed (customizing prompts), I can help add Bengali support 😄

[muhamrif]

I would like to help adding Urdu Language support on it, if approved. Thankyou!

[derekamsterdam]

Hi, I can help for the french version. It would be great to have a guide of the application architecture. I think there are two main topics :

• internationalize the tool --> not very relevant
• internationalize searches and outputs of the application --> very relevant

Hi there, I will glady help out for the french part of it also

[arthuRHD]

Hi, I'm available as a French reviewer if needed.

[minfenglu]

Hi, I would like to help adding Chinese support.

[luneice]

中文的支持可以参考如下：
找到 autogpt/llm/llm_utils.py 并加入以下代码。

    ...
    language: Message = {
        "role": "system",
        "content": "Your answer must use Chinese.",
    }

    messages.append(language)
    
    for plugin in cfg.plugins:
    ....

[VectorZhao]

中文的支持可以参考如下： 找到 autogpt/llm/llm_utils.py 并加入以下代码。

    ...
    language: Message = {
        "role": "system",
        "content": "Your answer must use Chinese.",
    }

    messages.append(language)
    
    for plugin in cfg.plugins:
    ....

nice，但是不够优雅。可以考虑加入chinese plugin支持。

[kaiboesefeldt]

Hello there! I hope you're having a great day so far. I just wanted to let you know that I've been working on this project and I'm excited to share my progress with you. You can find my latest updates in my cloned repository at https://github.com/kaiboesefeldt/Auto-GPT/tree/1563-supporting-different-languages.

I'm facing some challenges with the prompt fragments being spread over multiple parts of the code, as well as issues with plugins having to handle i18n themselves. Additionally, I'm wondering if translating the entire prompt is really necessary or if the approach suggested by @luneice (simply instructing the AI to respond in a specific language) would be simpler and just as effective.

If you have any suggestions or feedback, I would be grateful to hear it. Thank you for your time and support!

[kaiboesefeldt]

One key file is prompts_en.yaml, which contains the english version of all the prompt templates and snippets I collected so far. The concept is to have such a file for every supported language. So supporting another language would mean to copy prompts_en.yaml to e.g. prompts_zh.yaml and translate all prompt snippets and templates.

Please notice that this is still work in progress, but I nevertheless would appreciate some early feedback.

[pablorq]

I guess that could be more simple to dynamically select the used language from the terminal itself.

Maybe a "Language" configuration can be added to define that on every task, like:

Name:
Role:
Language: es
Goal:

Leaving "Language" empty, it is set to default (en).

[Boostrix]

Please see the comments regarding a feature to support dedicate "prompt profiles" so that people can experiment with different prompt profiles without stepping on anyone's toes: #1874 (comment)

[anonhostpi]

also see #471

[sameehj]

Arabic support is needed. I'm available as a reviewer for Arabic if needed.

[Boostrix]

• Prompt Profiles #3954

[yavuzibr]

I can help with Turkish if it needs.

[sudip550]

I need in italian, can anyone help??

[sudip550]

I would like to help. I can help adding Hindi support.

Please help me with Italian

[johnfelipe]

help me with spanish

[kaiboesefeldt]

I think I will stop working on this issue as my approach seems to be too complicated, because:

• Every other developer would have to remember not to simply add new prompt parts, but always also provide them through the central mechanism
• Adding a new prompt part would mean also to add new translations for that part in all supported languages
• Handling of modules is not well covered.

[ilantnt]

I can try help you with covering the missing parts + adding translation in Hebrew language.
Do you need any help @kaiboesefeldt ?

[github-actions]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions]

This issue was closed automatically because it has been stale for 10 days with no activity.