GPT-4 as a translation provider

[itbeard]

Hello folks,
My name is Lex, and I'm here with a proposal 😊

The Problem

Currently, Mastodon supports several translation services: DeepL and LibreTranslate. Unfortunately, both of these services do not support many of the languages that we use in the Fediverse. For example, as a Belarusian native, we have a Belarusian language server for 2,200+ users, but we cannot use translation because our language is not supported by either DeepL or LibreTranslate. It is really hard to push these services to add a new language (believe me, I've tried making requests several times, but still no result). Therefore, without translation, we have fewer opportunities to interact with the entire Fediverse.

Possible Solution

Over the last half a year, I believe many of you have started using ChatGPT for various reasons, one of them being translation. GPT-4 has a pretty good understanding of a lot of languages as input and can translate them quite accurately to our native language. Yes, it's not 100% accurate, but there are several benefits:

• It works with a ton of languages
• It can automatically detect the language used in a posts

So, I want to suggest adding two new configuration variables, something like OPENAI_API_KEY and OPENAI_TRANSLATE_INITIAL_PROMPT. Based on these variables, we could create a module for translating Mastodon posts. The OPENAI_TRANSLATE_INITIAL_PROMPT variable would include the very first prompt (or first part of a prompt) for GPT-4 and would contain something like "Act as a professional translator. Detect the language of the following text and translate it to {user_selected_lang}. Respond only with the translated text".

What do you think? Is this a useful enough feature to implement?
P.S. I'm not proficient in Ruby (I'm primarily a C# .NET developer), but I can help with the coding anyway.

[docvet]

Very necessary

[AGulev]

It sounds like a really useful feature.
Of course, I hope that DeepL will add more languages, but who knows when it happens.

[hleb-kastseika]

I agree, it can be quite useful for a lot of less common languages

[zubialevich]

It seems like not ideal, but good enough solution for this problem that can increase language support for everyone!

[MKras]

IMHO it worth to be implemented!

[Plinsboorg]

+1vote for this feature.

[ch3p3]

Good idea!

[fvadzim]

necessary for my switching to mastodon

[itbeard]

@Gargron @ClearlyClaire @grishka guys, is it possible to implement this from your point of view?

[grishka]

My personal opinion is that it's possible in principle, but there are two issues with this proposition:

1. OpenAI API costs money. Sometimes lots of money. Non-English languages cost more because of a less-efficient tokenization. There's definitely abuse potential for people to rack up huge bills for server admins.
2. These kinds of ChatGPT-based single-purpose services are notoriously leaky. It's fairly easy to make it ignore the initial prompt and output something else entirely. I'm convinced that this is due to the very nature of the current generation of AI (the "program" and the "data" are inseparable) and can't really be fixed.

[varshneydevansh]

What if we use GPT3 for providing translation and gave a limited usage to per user?

[itbeard]

Hi and thanks for the reply!
It seems that point 1 only makes sense for huge and medium instances, for small ones it might be a really good balance between quality and money. Especially since the OpenAI API has customizable limits on the use of money.
Point 2 seems to be the biggest problem with using GPT systems here...

[varshneydevansh]

I am using both GPT Plus and Merlin(built upon OpenAI) and the point 2 is extremely relatable.

And it's always amazing to see that tools which are built-up on these base tools perform better than them(same case with DallE vs Midjourney) and Merlin don't hallucinate that much. Maybe we can also explore the possibility of working with OpenAI function

How about looking for possibility of opensource GPT models?

[GlebAlexandrov]

As a Belarusian I second this proposal (and any other that would enable translation from Belarusian language).

[varshneydevansh]

Even I use GPT based tools a lot to simplify and translate stuff over the internet. Therefore, it would be a highly beneficial addition to Mastodon.

[Bachstelze]

The decoder-only GPT from "openai" is behind a paywall and we don't really know which languages are supported to which extent. There are multilingual models on huggingface with Belarussian included, e.g:
https://huggingface.co/facebook/m2m100_1.2B
Such models can be deployed via inference API or locally with easy translate.