Support for keywords in non-English

[Felienne]

In discussions with kids (even with Dutch kids at the high school level) kids tell us they would love to have Dutch keywords. It surprised me a lot because print and echo are even Dutch words. But yeah, research shows kids learn better in their own language so it is not that weird maybe!

And in non-Latin alphabets, it is even harder than in Dutch I imagine!

So all in all, this feature is truly on the top of our wishlist but it is going to impact all the things so we have to consider it carefully. Some preparation work already was done, by putting the keywords all in the level 1 grammar (#463).

What makes it even harder is that from a discussion I had at PILATE with Mark Guzdial, it became clear to me that having "Dutch Hedy" and "English Hedy" and" Spanish Hedy" is not even a great strategy because there are so many bilingual kids, and even monolingual kids might pick up English examples from the site or from each other, and then mix them with their own language. So we should probably allow mixing of languages also!

It is not clear to me whether allowing this language mixing from the start will make our lives easier, or that it is better to start with the "single language but translated" strategy.

Other fun things to consider are that simply translating the keywords is not even enough for some languages. For example, for truly idiomatic Dutch, we might want to have split infinitive keywords... like print hallo allemaal uit. Of maybe we would like to have if x 5 is instead of if x is 5 because that's how we say it in Dutch. Other languages (as I understand it for example Japanese and Arabic) don't really have an are verb, so something like name is hedy would not have an obvious translation. So in addition to translating keywords, we might also want to allow the overriding of grammar rules! But I guess if we want to have flexible grammar, and also also the "à volonté" mixing of languages, parsing would become nearly impossible. Yet it is fun and smart to consider these things before we start on the simpler versions.

In short, some things to consider when localizing a programming language (from easier to hard)

1. Allowing for the translation and then parsing of keywords like if and is in other languages
2. Having a feature to translate programs written in language 1 to language 2
3. Allowing one program to use keywords in different languages (f.e. mix als (nl) with repeat (en)
4. Allowing languages to override the grammatical structure of expressions

[nichtich]

Is the last level of Hedy going to be a fully valid subset of Python or will it also support non-English keywords such as used in Hebrew Python or any grammar transpiled into Python such as Enkelt? With full support of custom grammatical structures you could end up with a totally different syntax, maybe this would be too much. A list of examples would help. I found at random (two words) would be (zufällig one word) in German.

In any case I'd add another point:

5. Automatically translate between languages

There should be a method to translate a program written with keywords (and possibly grammar) in language A to the same program in another language. Strings and variables names might be kept untranlated or suggested translations could be shown based in NLP.

[Felienne]

Is the last level of Hedy going to be a fully valid subset of Python or will it also support non-English keywords such as used in Hebrew Python or any grammar transpiled into Python such as Enkelt? With full support of custom grammatical structures you could end up with a totally different syntax, maybe this would be too much. A list of examples would help. I found at random (two words) would be (zufällig one word) in German.

Great question! For now, our goal is to teach Python, so when the multilingual version is fully released, English langs will get an extra level explaining the English keywords.

However, there might be value in a Python version that supports different languages and allows mixing in En keywords! So depending on pick up that vision can change.

In any case I'd add another point:

5. Automatically translate between languages

There should be a method to translate a program written with keywords (and possibly grammar) in language A to the same program in another language. Strings and variables names might be kept untranslated or suggested translations could be shown based in NLP.

There already is code for this in the code base: https://github.com/Felienne/hedy/blob/255066ee8a99b8632fda6ed076646a82a4b6c806/hedy_translation.py and an issue open to add a switcher to the UI.

[das-g]

3. Allowing one program to use keywords in different languages (f.e. mix als (nl) with repeat (en)

As we'll probably want to support more and more natural languages (if there are people to contribute corresponding translations), this could problematic, as not all of them might be free of mutual overlaps for different semantics. (Overlaps for the same semantic, of course are unproblematic, as simply several languages would use the same keyword for the same purpose.)

A (made up) example to illustrate the problem:

Assume that in Foobarish, the most idiomatic way to express what echo does is "ripetu". So the keyword translation for echo shall be ripetu to accommodate Foobarian learners. repeat <n> times <action> will just be <n> foje <action>

Assume further that meanwhile, in Bazquxish, the most idiomatic way to express what repeat <n> times <action> does is ripetu <n> fojoj po <action> and echo shall be expressed as rediru for Bazquxian learners.

So we'd have

English	Foobar	Bazqux
echo	ripetu	rediru
repeat <n> times <action>	<n> foje <action>	ripetu <n> fojoj po <action>

Now this might not be too bad, as

• echo is only available in Level 1
• repeat isn't available until Level 7
• repeat commands consist of more than just that keyword, so that context could help distinguish between Foobarish's ripetu and Bazquxish's ripetu, just like context helps in Python distinguish between Python's various uses of the keyword in. (Though this might still be rather tricky to implement correctly.)

But it's not hard to imagine that it could also affect keywords that can occur in the same level and that cannot be distinguished by context.

Given that the number of keywords is quite limited even seen over all Hedy levels combined (currently there are just 28), such collisions might not be a very frequent thing. Still, as there are many "false friends" in many pairs of languages (e.g. "gift", which means "present" in English, but "poison" in German), some might come up even in that limited set of keywords, if keywords for more and more languages shall be offered.

[Felienne]

Yes this is a risk, and this is one of the reasons that for now, we only mix languages with English and not all with each other to minimize overlap. Once we get languages with overlap, we can discuss then how to go about it.

[das-g]

for now, we only mix languages with English and not all with each other to minimize overlap.

Oh, I wasn't aware of that. I thought the goal was to allow mixing of all supported languages from the beginning.

If only mixing of any one language with English is allowed (i.e., mixing of two languages in the same program, one of which must be English) then the potential for overlap is indeed massively smaller.

A not entirely fair but pragmatic approach would then be, that no language for which we add translated keywords may introduce keywords already used by English Hedy with different semantics than they're used in English Hedy.

If we ever want to allow mixing arbitrary supported languages in the same program, this approach can be extended, in that no newly added translated keywords may collide with keywords of already-supported languages, unless used for the same semantics.

Once we get languages with overlap, we can discuss then how to go about it.

That's right. Though, if we initially only support mixing individual languages with English (but might later want to support mixing arbitrary languages) we might not notice overlap between languages added during that first phase unless we explicitly check for that. This can have the consequence that, when we then later want to support arbitrary mixing, the keywords of some languages would then have to be changed.

So if mixing arbitrary languages is any option, it might be preferable to check newly added translated keywords not just for overlaps with English (which would happen implicitly by figuring out how to make mixing with English work correctly in the implementation) but also for overlaps with other already-supported languages, even if those overlaps wouldn't cause problems just then.