[RFC] YML translations ?

[jjanvier]

I'm just wondering why we use xliff translations ? This causes so much pain when we have to add or edit translations.

Even if xliff is a standard, what are its advantages ?

Could not we use yaml translations for Sylius/Sylius and export them to xliff for Sylius/Standard if needed ?

[pjedrzejewski]

Is your question related to Crowdin integration?

[stloyd]

I'm not sure YAML is able to handle pluralization correctly (didn't used it with the rules like this below):
{0} Nenhuma etapa|{1} 1 etapa|]1,Inf] %1% etapas

[jjanvier]

@pjedrzejewski not necessarily (I'll try to work on Crowdin this week end). But anytime I need to add translations, I cry. We should at least provide a how to...

@stloyd from one of my project :

portal:
    contact:
        message:
            count: "{0} Pas de contact|{1} contact.|]1,Inf] %nb% contacts"

[winzou]

If we want to use crowdin one day, we must check that it supports yaml (it supports xliff for sure).

[jjanvier]

@winzou I think Akeneo (which uses Crowdin too) has its translations in yaml. I'll check it

[jjanvier]

@winzou yes, yaml is supported :) http://crowdin.net/project/akeneo/fr

[jjanvier]

ping @Sylius
this could be done alongside #133

[pjedrzejewski]

OK, I've given it some thought during the weekend and... while I hate the fact that we will need to switch BACK to yaml. I'm ready to agree with this, I know that export format of the crowdin is quite different than Symfony format, even if it's xliff. Importing is easy, but the other way it was not so simple.

We would benefit from XLIFF if crowdin would support some extras, but there seems to be nothing in favor of having xliff...

Things to consider.

1. Akeneo does not use translation keys, but full phrases/sentences. @jjanvier Does Crowdin support generating translation keys from nested yaml? (so nodes foo->bar->x become foo.bar.x?)
2. We should reconsider a lot of keys, to avoid duplication.

[jjanvier]

@pjedrzejewski

Check the French translation for messages.yml of ShippingBundle : https://crowdin.net/translate/sylius/78/en-fr
Our file has keys like sylius.form.shipping_category.name. Crowdin displays this key with its English translation Name. But the real key for Crowdin is still sylius -> form -> shipping_category -> name.

[jjanvier]

It makes sense. It's easier to translate Name than sylius.form.shipping_category.name.

[jjanvier]

@pjedrzejewski can I convert xliff to yaml asap, and then upload English files to Crowdin ? I'd like to clean and reorganize translations keys in another PR if possible.

[pjedrzejewski]

But, what's the point of uploading them to crowdin before reworking them? (Can we tackle this one together? :D)

[jjanvier]

I'd like to reorganize and clean only English translations. Not all of them. We need first to "save" Sylius translations to Crowdin. Either now, as xliff. Or after having converted them to yaml.

I don't know if it's the best.. Actually I'm afraid we will lose a little bit of translations whatever workflow we choose...

[pjedrzejewski]

Yes and it will be a bit tedious to restore them, BUT the current keys are not very nice. Anyway, let's convert all translations to YAML. @Sylius if anyone has something against... please say now. :) IIRC @umpirsky converted them to XLIFF so let's wait for his input at least.

[umpirsky]

IIRC we decided to go with XLIFF because it is widely used for translation purposes and adopted by Symfony itself. Gettext is also widely used and this are two formats that are supported by majority of i18n tools.

I am perfectly fine with any format, but lets not change this every now and then since it is not a quick task :)

I would decide general translation handling process first, then decide about format:

• How do we extract translations from sources?
• Translation domains naming convention.
• Keys naming convention.
• ...
• Then pick format that fits best into translation handling process.

[winzou]

Just a thought: what is the point of using yaml format if we use crowdin for translations? Staying in xlf would allow us more thing as @umpirsky said.

[jjanvier]

@umpirsky @winzou To explain you why xliff annoys me, I'll try to explain you how I work when I want to implement a new feature on the WebBundle. If it's the wrong way, of I do something bad, please let me know :)

1. Write Behat scenarios
2. Code
3. Launch my scenarios. Oh it fails because sentences are not translated. Let's launch JMS translation:extract to get them. Oh it fails, I get [Doctrine\Common\Annotations\AnnotationException] [Semantical Error] The annotation "@Given" in method Behat\MinkExtension\Context \MinkContext::iAmOnHomepage() was never imported. Did you maybe forget to add a "use" statement for this annotation?
4. Let's do like in Allowing translations to be easily extracted #212, this hack that can not be merged
5. Ok. Extraction complete. translation:extract adds useless jms statement that should be cleaned like said here [InstallerBundle] Fixing translations extension to be consistent with other... #830 (comment). I need to revert manually a lot of things in a file of 2500 lines.
6. Oh, but the extraction also changed all trans-unit ids with no reason. Let's revert it manually... On 2500 lines. Ok, my translations are clean now.
7. Relaunch Behat scenarios. What ? Some translations are still missing ? Yeah, translation:extract extracts only from templates. Not from controller or classes... Let's add these translations temporarily to a template to get them extracted.
8. Ok... Now I have to do 5 and 6 again...
9. Ok commit !
10. New feature ? Ah ah... Let's go back to the beginning.

My initial question was : why do you use xliff ? Even if xliff is a standard, what are its advantages ?
No answer till now :)

I'm not saying xliff is bad. Not at all. It just make me loose a lot of time and I dont think it is adapted for our developments that are constantly growing at the moment. With yaml, I only have to open the file and edit it.

Plus, yaml keys are sorted hierarchically, which is more readable IMO. It will also us to reorganize all the keys much more easily.

Translation domains naming convention and keys naming convention are out of this discussion IMO. This does not depend of the format of our translations.

[umpirsky]

@jjanvier You will have all this problems with any format if you use jms translation bundle.

You can help yourself by making translations from classes and controllers be parsed by jms or extract them to separate translation domain (step 7).
It changes ids, so what? It is auto generated, who cares?

But I get your point, workflow is far from perfect. Lets work together on simplifying it by sharing knowledge.

[jjanvier]

@umpirsky with yaml, It will not use jms command anymore. I don't need it anymore. I'd be happy to find a better workflow :)

[umpirsky]

I used https://github.com/umpirsky/Twig-Gettext-Extractor on one project. Just saying :)

[stloyd]

While expose stuff is really hacky, buggy and whatever 💃 I have always loved one thing while getting through translations in xliff i.e. in Symfony, resname is independent of source...

And this leads to thing I hate in translations... keys 😠... srsly how you can think that for someone something like:

sylius.form.shipping_category.name

Is more redeable than:

Shipping category name

🙈 🙉 🙊

ps. I know I'm annoying & not always talking directly about topic =)))

[jjanvier]

So... what do we do ?

[pjedrzejewski]

@jjanvier Let's go with YAML!

[jjanvier]

@pjedrzejewski ok. But not until next Wenesday sorry

@umpirsky what did you use at the time to convert from yaml to xliff ?

[pjedrzejewski]

I fear that there is no easy way if we want to have nested keys... We could write our own converter which would process foo.bar.zip keys in xml to foo: bar: zip in yaml.

[umpirsky]

@jjanvier I used jms translation bundle iirc.

[pjedrzejewski]

My idea for the translation keys.

Legend...

%resource% - name of the resource (model). product, shipment, order, order_item and so on.
%property% - concrete property of the resource.
%constraint% - validation constraint name.

• Validators.

sylius.%resource%.%property%.%constraint%
sylius.product.name.not_blank
sylius.cart_item.quantity.min

For custom constraints, we take the name as well.

sylius.variant.sku.unique.

All validator keys should live in the domain validators.

• Menus.

All translations for the menus rendered though KNP menu. Every menu & position should have unique key (in order to allow easy customization), even if the menu position is similar/same.

We should remove the frontend & backend prefix, these should go directly into the part with the menu name. All translations go to the menu translation domain.

sylius.%menu_name%.%node%
sylius.backend_sidebar.create_product
sylius.frontend_sidebar.about
sylius.user_panel.orders
sylius.user_panel.address_book

Anyway, I'd like to have most of the menus managed through Symfony CMF, so editable in the backend. So the translations would be handled there as well.

• Forms.

I'm not sure here, I'd love to go with a separate translation domain forms. If you think it has some drawbacks, speak now, please. :)

Proposal a)

The %form_type% is the name of the form type without the sylius_ prefix, so for sylius_product it is just product.

sylius.%form_type%.%property%
sylius.product.name
sylius.cart_item.quantity

Proposal b)

Same as first idea, but without differentiation of the resource, so basically, sylius.shipping_category.name and sylius.tax_category.name would be reduced to just sylius.name in the domain forms.

It would make it a lot easier to add new forms and remove duplication with the translations, but it will make it harder for the developer to customize the concrete field label. (he could do that in the custom form type class quite easily though)

I like proposal a) for its flexibility, but proposal b) is much simpler. What do you think?

• "General" terms

Right now this part is the most ridicolous and has tons of duplication. I started this bad practice, so now I'd like to fix it. :)

Basically, every "general" term (everything except forms, menus, validators) will go to the messages domain and will use sylius. as prefix.

So, instead of sylius.product.id and sylius.shipment.id, we will have just sylius.id, same for other properties and so on. sylius.order.billing_address will just become sylius.billing_address and will be used in any place needed.

Common words, like sylius.delete, sylius.create, sylius.yes will also follow this convention.

With YAML as the format, we can make nested translations, so this would be quite clean in the files and we'll edit it through Crowdin anyway, which has excellent UI for reviewing and accepting translations.

YAML also allows to easily add new translations, with or without using the JMS translation command.

So, what do you think? If we agree on these (or modified according to your comments) conventions, I could turn this into an article for the Contributing Guide and I could start updating the templates & translations. After we're done with this "refactoring" of translations, we could finally go and use only Crowdin for translations!

Let me hear your opinions!

[umpirsky]

Sounds good. If you want to still extract translations with JMS, we should check how it works with separate domain for forms.

[pjedrzejewski]

@Sylius We need more opinions. :)

[arnolanglade]

@pjedrzejewski : A different opinion : #133 (comment) ;)

[umpirsky]

https://localise.biz/free/converter/yml

[jjanvier]

@pjedrzejewski really like your proposals except the one about the forms.
I'm also against proposal b. Even if we have a few duplicated translations by keeping the resource in the key, it allows developers to customize the forms as they want. Plus, don't forget that we can use bundles standalone. So we need to provide these translations inside the bundles.

To end with the forms, I prefer @Arn0d's way. sylius.%form_type%.%property%.label and sylius.%form_type%.%property%.options if needed. More flexible IMO. It allows to add other keys related to the property if needed.

Totally OK for using the domains validators, menus, forms and messages.

Also I really like @Arn0d's proposal about using special keys button, modal, tooltip.

[jjanvier]

@pjedrzejewski and before changing all the keys (btw, I really can help on this), I think we should :

1. convert from xliff to yaml (PR this we :))
2. upload all translations on Crowdin to have a "backup". And we'll surely not change all the keys.
3. change the keys
4. re-upload to Crowdin

[jjanvier]

@umpirsky thanks, but I didn't want to upload files one by one. So I built a converter.

[umpirsky]

Just saying https://t.co/zLYpmK2A3Z

[jjanvier]

@Sylius any opinions about translations keys ?

[umpirsky]

@jjanvier Can you please share converter you built?

/cc @kayue

[jjanvier]

@umpirsky @kayue it's here at the moment https://github.com/SyliusBot/Sylius/blob/translator-tools/src/Sylius/Bundle/CoreBundle/Command/Xliff2YamlCommand.php Not perfect, but it works. I'll try to put it somewhere better.

[umpirsky]

@jjanvier Thanks!

[jjanvier]

@pjedrzejewski you closed this one but we didn't finish to set the translations keys. But a different ticket could be a better place...