Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Translate resource request #803

Closed
warpok opened this issue Jul 16, 2022 · 30 comments
Closed

Google Translate resource request #803

warpok opened this issue Jul 16, 2022 · 30 comments

Comments

@warpok
Copy link

warpok commented Jul 16, 2022

It turns out that Google Translate does a very interesting job of translating the Greek NT into Indonesian. Our translators and people visiting our Guest site often are hampered because of not having an interlinear that uses Indonesian glosses.

Would you be willing to make a resource that would take the Byzantine Text verse and run it through Google Translate?

Screenshot_20220716_091559
Screenshot_20220716_091317

It seems like this might be a resource other translators would appreciate, if it could be set up with variables:
Greek Text: _________ (one of the resources already in Bibledit)
Target Language: _______ (Instead of Indonesian, some might choose Chinese or French.)

@teusbenschop
Copy link
Member

I think that if having glosses in their own language helps, and if google translate does a reliable job, then this would be a worthwhile effort to put in.

@warpok warpok changed the title resource request Google Translate resource request Jul 17, 2022
@warpok
Copy link
Author

warpok commented Jul 17, 2022

Yes! I'm certain it will help. The heading for this should say that it is from Google Translate, since people know that it often makes mistakes. They will be warned that the translation might not be perfect.

@teusbenschop teusbenschop self-assigned this Aug 4, 2022
@teusbenschop
Copy link
Member

It is already possible right now to translate a resource from one language to another language.

This can be done through creating a new USFM resource.

These are the steps to do it.

  1. Convert the desired original resource to a Bible. See for example here: http://bibledit.org:8090/bible/settings?bible=Sample. Click the option "Import text from a resource".
  2. Create a new Bible with the name of the target language, for example "Greek to Indonesian NT".
  3. Copy the USFM of each chapter of the original Bible into Google Translate.
  4. Paste the translated USFM from Google Translate into the sample "Greek to Indonesian NT" Bible.
  5. Do this for all the chapters required.
  6. Optionally correct and update the text as needed.
  7. Optionally convert this Bible to a Resource (but this step is not needed).

This way you may be able to display the "Greek to Indonesian NT" alongside the other resources.

This will assist the translators greatly.

teusbenschop added a commit that referenced this issue Aug 5, 2022
teusbenschop added a commit that referenced this issue Aug 7, 2022
teusbenschop added a commit that referenced this issue Aug 8, 2022
teusbenschop added a commit that referenced this issue Aug 14, 2022
teusbenschop added a commit that referenced this issue Aug 14, 2022
teusbenschop added a commit that referenced this issue Aug 14, 2022
teusbenschop added a commit that referenced this issue Aug 14, 2022
teusbenschop added a commit that referenced this issue Aug 15, 2022
teusbenschop added a commit that referenced this issue Aug 16, 2022
teusbenschop added a commit that referenced this issue Aug 17, 2022
If the google JSON key is not found, then enable a section with info about how to set it up. #803
teusbenschop added a commit that referenced this issue Aug 20, 2022
On a client it should not say that the admin should be asked to set up the translated resources. It should just list none, that's all what it should do.
#803
teusbenschop added a commit that referenced this issue Aug 21, 2022
teusbenschop added a commit that referenced this issue Aug 23, 2022
@teusbenschop
Copy link
Member

The newest Bibledit (Cloud) in the Ubuntu PPA now has this Google Translate access and provides the "Translated resource".

@warpok
Copy link
Author

warpok commented Sep 2, 2022

Teus, this doesn't work!
Screenshot_20220902_104208
The user cannot choose the language to translate to, or the one to translate from.

@teusbenschop
Copy link
Member

It is related to issue #824. If access to Google Translate has not yet been set up, or is not yet working, then there's no language options to choose from. That is what you are seeing.

Here are the steps to set up Google Translate:
#803 (comment)

The first step points to a page to set it up. What does that page say in your situation?

@warpok
Copy link
Author

warpok commented Sep 2, 2022

Wow, Teus,
Your instructions for doing the task of setting up the Google Translate service are truly daunting. Setting up a server for Bibledit-cloud is a piece of cake. What kind of super-humans are you expecting to be able to set that up?! (Clearly I fail.)

Although I thought I might have succeeded. The instructions are like a treasure hunt with clues.
googletranslate.txt

Go to to the server to get the instructions, go five steps to the north and go to this site and do this procedure, then go to another site and find a key.

  • There are two editions of Google Translate. You don't tell us whether to choose basic or advanced.
  • You told me to get a json key, but I don't know what I am looking for. What will that key look like? Is it a string of about 24 numbers?

Does this look right?
Screenshot_20220902_131318

@warpok
Copy link
Author

warpok commented Sep 2, 2022

Is this where I get the Json key? Is it the link on the right or the link on the left, or is it somewhere else?
Screenshot_20220902_135527

Should I enable this?
Screenshot_20220902_135817

The instructions for installing Google-cloud-c11 say that the default place to install is in the home subdirectory. But Bibledit-cloud can't find my installation. I think I should install it in the /var/bibledit folder instead. Do you agree?

Screenshot_20220902_140130

It seemed like I installed the Google-cloud-c11 properly.

Screenshot_20220902_140642

Actually, after that install didn't work, I tried to install in the /var/bibledit folder, but I never got that last screen. Probably due to not being able to finish the Shell Command Completion step.

@warpok
Copy link
Author

warpok commented Sep 2, 2022

The problem could be that I should use the Key menu choice found here:
image

@warpok
Copy link
Author

warpok commented Sep 3, 2022

I thought I already had a key, so I didn't add another one.

I have clicked the link, and there is no response. The Red line stays red.

Screenshot_20220902_190737

Do I need Shell Command Completion?
Screenshot_20220902_190436

So, bottom line:

  • I don't know where to find the json key. What I put in the googletranslate.json file is probably wrong.
  • I don't know what I did wrong so that gcloud GLI is not found.

@warpok
Copy link
Author

warpok commented Sep 3, 2022

Sorry for the tone of the first part of this post. If you were offended, please forgive. I hope you can read such things with the tone of a friendly jibe. If you can help me with this, I can try to write a step by step article, not for Map.Bloomfire, but something to post elsewhere. I did my best to install things properly, which I hope shows that I am greatly desiring the Google Translate feature to work for us.

One thing occurs to me: The automatic install will place things in the /var/bibledit folder. But you have said that Aranggi was more correct in placing the files in a Home/user folder. Now the gcloud CCI said that it would be best to install that in Home, but then it wasn't found by Bibledit. So, if occurs to me that it might be best for the sake of add-on programs working as expected with bibledit-cloud to make the automatic install go to a home/user area.

Actually, the place where I want the Google Translate feature to work properly is the Indonesian Guest instance which is still at Bibledit.org. If you feel you can install it there, please make copious screenshots that we can use for making a clear tutorial.

@teusbenschop
Copy link
Member

[...] Your instructions for doing the task of setting up the Google Translate service are truly daunting. Setting up a server for Bibledit-cloud is a piece of cake. What kind of super-humans are you expecting to be able to set that up?! (Clearly I fail.)

I thought that giving the steps while referring to a guide written by Google should not have been to difficult. The steps in the file "googletranslate.txt" all refer to the Guide written by Google. I though that a big company like Google would be able much better than myself to write a guide that is helpful.

  • There are two editions of Google Translate. You don't tell us whether to choose basic or advanced.

That is correct, the file "googletranslate.txt" does not tell that. But when following the Guide written by Google, it never asks for that question. However the answer to your question is that Bibledit uses the basic edition of Google Translate. And the Guide does that too, albeit implicitly, without mentioning it. Thinking about the Guide, it might have been better if Google had left that remark about basic or advanced out entirely. The remark could lead to confusion. Anyway, the answer is: Basic edition.

  • You told me to get a json key, but I don't know what I am looking for. What will that key look like? Is it a string of about 24 numbers?

Normally the JSON key is a file. The contents of that JSON key looks like this:

{
  "type": "service_account",
  "project_id": "yourproject",
  "private_key_id": "<a long string of characters and number>",
  "private_key": "-----BEGIN PRIVATE KEY-----<a very long string of characters and numbers$
  "client_email": "test@project.iam.gserviceaccount.com",
  "client_id": "<a long number>",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test%40project.iam.gserviceaccount.com"
}

Does this look right? Screenshot_20220902_131318

This is the screen shot for creating a project, it looks right, it's one of the steps in the Google Guide.

@teusbenschop
Copy link
Member

Is this where I get the Json key? Is it the link on the right or the link on the left, or is it somewhere else? Screenshot_20220902_135527

Yes, this is where you will be able to get the JSON key. The Google Guide tells you where to click exactly. The Guide says this:

  1. In the Google Cloud console, click the email address for the service account that you created.

Should I enable this? Screenshot_20220902_135817

Yes, you should enable this.
Actually the Google Guide didn't mention whether to enable this, and that is a shortcoming in that guide.

The instructions for installing Google-cloud-c11 say that the default place to install is in the home subdirectory. But Bibledit-cloud can't find my installation. I think I should install it in the /var/bibledit folder instead. Do you agree?

It's not c11 but cli`. The word cli``` is an abbreviation for "command line interface".

I didn't find the link to the place where the instructions tell that the default location to install is in the "home" subdirectory. Could you provide a link to those instructions?

It should install it in the default location where the gcloud-cli want to get installed, not specifically in /var/bibledit. The key for Bibledit Cloud to find the cloud-cli is to add the program to the PATH. It's one of the questions the installation process will ask you. And Google didn't say that in their guide.

Screenshot_20220902_140130

It seemed like I installed the Google-cloud-c11 properly.

Yes, it looks that the cloud-cli is now installed properly, looking at the screenshot.

Screenshot_20220902_140642

Actually, after that install didn't work, I tried to install in the /var/bibledit folder, but I never got that last screen. Probably due to not being able to finish the Shell Command Completion step.

There's no need to install it in /var/bibledit. Best is to let the installer decide its own location.

@teusbenschop
Copy link
Member

The problem could be that I should use the Key menu choice found here:
image

Yes, use that key menu choice.

The Google Guide says that too:

Create a service account key:

  1. In the Google Cloud console, click the email address for the service account that you created.
  2. Click Keys.
  3. Click Add key, and then click Create new key.
  4. Click Create. A JSON key file is downloaded to your computer.
  5. Click Close.

@teusbenschop
Copy link
Member

I thought I already had a key, so I didn't add another one.

Did you store the key in the config folder of Bibledit?

This is what the file googletranslate.txt says:

Store the key in the Bibledit "config" folder as file "googletranslate.json".

I have clicked the link, and there is no response. The Red line stays red.

Screenshot_20220902_190737

The red line is self-explanatory. It means that Bibledit Cloud could not find the Google JSON key in the correct location.

Do I need Shell Command Completion? Screenshot_20220902_190436

The answer to that prompt will be Yes.

@teusbenschop
Copy link
Member

Sorry for the tone of the first part of this post. If you were offended, please forgive.

No problem at all.

I hope you can read such things with the tone of a friendly jibe. If you can help me with this, I can try to write a step by step article, not for Map.Bloomfire, but something to post elsewhere. I did my best to install things properly, which I hope shows that I am greatly desiring the Google Translate feature to work for us.

Yes, that would be very helpful, if you were able to write an article with step by step information.

One thing occurs to me: The automatic install will place things in the /var/bibledit folder. But you have said that Aranggi was more correct in placing the files in a Home/user folder. Now the gcloud CCI said that it would be best to install that in Home, but then it wasn't found by Bibledit. So, if occurs to me that it might be best for the sake of add-on programs working as expected with bibledit-cloud to make the automatic install go to a home/user area.

There's two ways of doing things. If installing Bibledit Cloud from an Ubuntu package, then the most logical and official place is /var/bibledit.

What Aranggi did is different. He didn't install Bibledit Cloud from a package, but he installed it from source code. If doing it from source code then, yes, it's easier to do that in the home directory.

Actually, the place where I want the Google Translate feature to work properly is the Indonesian Guest instance which is still at Bibledit.org. If you feel you can install it there, please make copious screenshots that we can use for making a clear tutorial.

I can install it there but, but still, you will need to provide the JSON key, because that JSON key will be generated from your own google account, and will be charged to that account too. But once you have the JSON key, I can install it for you there.

teusbenschop added a commit that referenced this issue Sep 3, 2022
@warpok
Copy link
Author

warpok commented Sep 3, 2022

Screenshot_20220903_183313
Screenshot_20220903_183326
Screenshot_20220903_183249

@warpok
Copy link
Author

warpok commented Sep 3, 2022

OK, with your information, one clicks on the link in the email column.
Screenshot_20220903_183313

That opens this page, and one has to click the Keys menu item.
Screenshot_20220903_183326

Then on this page, you have to click to add a key. The key is generated automatically and it was named, in my case, river-sunlight+longnumber.json.
Screenshot_20220903_183249

This is then renamed as googletranslate.json and put in the bibledit-cloud/config folder.

Now, please tell me how to install the gcloud cli program so that it is on the path. What folder do I un-tar it and run it in? Or what command do I give to put something in the 'path' so that will be found by Bibledit-cloud?

@teusbenschop
Copy link
Member

The Google Guide gives all the steps to install cloud.
https://cloud.google.com/sdk/docs/install
This guide is general, it covers many Linux flavours.

In case you'd like to install cloud on Ubuntu, then the steps are much simpler.

Here are the steps to take on Ubuntu:

sudo apt-get install apt-transport-https ca-certificates gnupg

echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

sudo apt update

sudo apt install google-cloud-cli

This automatically puts cloud into the PATH.

Once this is done, then Bibledit will be able to find the gcloud and then can start using it.

You will now be able to define a so called "Translated resource". (And then the translators will have access to a vastly expanded pool of resources...:)

teusbenschop added a commit that referenced this issue Sep 4, 2022
@warpok
Copy link
Author

warpok commented Sep 4, 2022 via email

@teusbenschop
Copy link
Member

So happy it works well and is useful. Thanks for the feedback on this.

I'll send a WeTransfer with the content of the 8084 port and will take the port offline now.

@warpok
Copy link
Author

warpok commented Sep 5, 2022 via email

@teusbenschop
Copy link
Member

What is the current original resource in Hebrew? And the destination language is Indonesian? I could try to see how it behaves here.

@warpok
Copy link
Author

warpok commented Sep 6, 2022 via email

@teusbenschop
Copy link
Member

I first tried the Westminster Hebrew text (one of the first options listed in the resource list). It seemed to work when I did it, but then didn't work.

It looks like Google is not doing a great job when it translates Biblical Hebrew, yet takes it as modern Hebrew, and also the vowels in the Hebrew seem to cause difficulties to Google.

Then I tried the E-sword module entitled Masoretic text (or something like that). If you search for Masoretic in the resource list window, that is about the second one to be found.

It's working well over here, but the translation is not too reliable.

I wonder if the bug could be that there are TWO active Translated resources, and the working of one conflicts somehow with the other.

I've got four Translated resources defined now, and they all work independently here. Not sure what's going on in your case.

@warpok
Copy link
Author

warpok commented Sep 7, 2022 via email

@teusbenschop
Copy link
Member

Which four are working well for you?

It's not all four that are using Hebrew, but two of them do:

Translated translated [edit]
Translated test [edit]
Translated westminster [edit]
Translated sword masoretic [edit]

The one using the Westminster text is having vowels and fails to translate properly.
Westminster Hebrew
וְהָי֤וּ לִמְאֹורֹת֙ בִּרְקִ֣יעַ הַשָּׁמַ֔יִם לְהָאִ֖יר עַל־הָאָ֑רֶץ וַֽיְהִי־כֵֽן׃
Translated westminster
והי֤ו למאורות֙ בִּ֣יעַהָשִעִם לִ֖יר אל-ה֑֑רז וֽיְהי-օֽ׃

The second one using Hebrew is the Sword Masoretic text.
Hebrew Masoretic OT
והיו למאורת ברקיע השׁמים להאיר על־הארץ ו͏יהי־כן׃
Translated sword masoretic
And let there be a light in the firmament of heaven to give light upon the earth, and let it be so.

The main issue is that the translation from Hebrew to any language appears not to be reliable. The reason seems to be that Google Translate assumes modern Hebrew, and that the Masoretic texts are Biblical Hebrew, which at times does not get translated too well.

The second issue I can see with Google Translate is that when the Masoretic text has the dots for the vowels, then Google Translate fails to translate it.

What may help in your situation is to have a kind of, say, literal translation in, say, English, and then translate that to Indonesian. I guess that may be more reliable than going from Biblical Hebrew to Indonesian. However these are just my thought at the moment.

@warpok
Copy link
Author

warpok commented Oct 3, 2022

Agreed. Thanks.

@warpok
Copy link
Author

warpok commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants