Offline Description Column in CSV

[safi-manar]

Hello,

One of the challenges we're facing is being able to export descriptions into CSV's and being able to import them back into Weblate.
I've seen the following relevant issues:

1. Please export comments to .po files #407
2. [XLIFF] Allow to add notes #772
3. Download translation & Weblate-specific data #1194

I understand the complications that would be involved in exporting and merging back in changes to comments.
An alternative solution that might remedy this complication might be to add a new field into the Weblate interface: "Description". The data here would persist in the Weblate database, but could also be exported into the CSV and imported back in to update the database.

This solution would allow developers and an internal product team to collaborate on a solid description-- such as, discuss via the comments of Weblate, and then agree on description field. Then, they could export the CSV, which would include the description column for translators to be able to view offline.

Our team has a working proof of concept of this idea. However, we are concerned about maintaining it internally across Weblate upgrades. We were wondering about the feasibility of including this as a feature in Weblate-- that could perhaps be enabled in the admin interface with an explanation of its function in CSV import/exports.

What do you think?

[nijel]

We already have comment field on every unit, so maybe it's what you're looking for? I'm not sure how it translates to CSV though, but it's certainly supported at least by Gettext.

[safi-manar]

I've seen the translator-comments and developer-comments in the CSV exports, something that translate-toolkit seems to control?
The CSV import was previously broken, so we were not able to test whether changing these columns reflected the changes in Weblate on a CSV import.

In any case, I did not see an option to be able to edit the Comment field in the actual Weblate interface. Was there a way to do this?

[nijel]

It's not possible to edit it Weblate but it will show it from supported formats (at least Gettext and XLIFF). For editing there has been #772

[safi-manar]

Hmm, sorry, I might not be understanding.
So, it seems to me like this description exchange that supports editing in the Weblate interface and CSV import/export is not currently supported, correct? If that's the case, what do you think about adding an optional description field?

[nijel]

It doesn't support editing, it can just be imported from some file formats (at least Gettext and XLIFF). But it can certainly be done working for other formats as well. The thing that for example in Gettext you're not supposed to edit that (at least the "developer comment" part).

[safi-manar]

In that case, if I download a CSV, make edits to the translator-comments, and then reupload it back, it will persist in the Weblate database?

[nijel]

I probably won't work on upload right now (though it might have worked in the past). Currently it's really parsed only from the storage.

[safi-manar]

I see. So, are you against the idea of adding the CSV Description column? Or alternatively, adding support in the Weblate UI to edit translator comments?

[nijel]

There is already support for translator_comments/developer_comments in the CSV file and I think there is no need to add another column for that. The thing you want perfectly fits into developer_comments.

Though the question is how to utilize that more as currently it's used only from upstream and there is no editing available.

[Dchau95]

What do you think about the idea of adding a Description field to the Unit model, and having that connect with developer_comments?

The reason I bring this up is because I was playing around with the project on a private repo, and added the above field and managed to get it working, along with a CharField in the UI aspect. In my private repo, I can write a description, export that to CSV and see that description under developer_comments. I also edited the developer_comments and imported the edited CSV back into Weblate and saw the changes in the Weblate UI.

[nijel]

I'm not really sure about the editing part - for example with Gettext the developer comments are overwritten on every translation update, so editing it there doesn't really make sense.

Also from Weblate perspective (if we're going to ignore the input from the file), it would probably make more sense to attach this to the source string rather than to the unit...

[Dchau95]

I see your point about attaching it to source string than to unit. I'm willing to edit my private repo to take into account of this to test it out.

How about changing it from developer_comments to translator_comments? Would that still have the issue of overwriting on every update? Or does it still not make sense about editing there?

[nijel]

What you are trying to achieve? The developer comment is per source string and is usually is used to give translators more insight what the string is about (example). The translator comment is usually used to explain why certain translation has been used and makes sense only for given translation.

In gettext, the developer comment is extracted from the source code (here for the above example) and it's not possible to define it elsewhere. The translator comments can be defined while translating and are kept in the po files over the msgmerge. I'm not sure how other translation formats deal with these.

[Dchau95]

The use case(s) I'm trying to achieve is:

• As a developer, I want to be able to write a description(developer_comment) and have it be persisted in Weblate.
• As a translator, I want to be able to download the CSV and see the description(developer_comment) that the developer wrote, as well as the other information(translation, etc)
• As a translator, I want to be able to upload the CSV, with changes in the description(developer_comment) and see it persisted in Weblate, as well as, the other changes.

The last two bullet points, I want to emphasize the CSV portion. From your first example, it seems there is support for the first bullet point, correct? Now correct me if I'm wrong but it seems that the last two bullet points are not supported as of now. So I would like to have them supported, I just want to know how to go about it, what the Weblate flow should look like, or any other information.

[nijel]

Yes the first part should work, maybe it needs tweaks for some file formats (it definitely works for Gettext). Download of that does work as well, but there seems to be a bug that it's stored as translator_comments instead of developer one.

Updating the developer_comments from upload is thing which I see problematic, as this certainly won't be preserved on Gettext files update, not sure about other formats.

For translators, it might be better to import translator comments and store these as comments in Weblate.

[Dchau95]

Okay, so the only thing left is solving the updating developer_comments from upload. Is there no way to programmatically tell Weblate to not override developer_comments on each update?

Also, do you mind explaining how to recreate this problem for Gettext? Is it just downloading Gettext, making changes, then uploading the changed file?

And do you mean for translator comments, this is the comments section on Weblate?

[nijel]

When using Gettext the developer comments are updated from the source code and any changes done in po file are discarded:

1. xgettext extracts them from source code to pot file
2. msgmerge updates po files based on content of pot file

So whenever the translatable strings will be updated, the changes to developer comments are lost...

Yes, I was thinking about comments section in Weblate.

[Dchau95]

Okay, thank you explaining that.

I realized I have another question for the first bullet point. I noticed in your developer comment example, there is no "Source string comment" field for the other strings. So how would a developer add a new developer comment from Weblate UI? Is it only through the source file?

[Dchau95]

I've inspected the code and it seems like "Source string comment" field is not editable at all. Is it possible to change Weblate to make this field editable and viewed even if there are no comments? I feel like there is a use case for being able to edit/create developer_comments from Weblate UI.

[nijel]

Some time later: The CSV export includes user comments, suggestions and additional context from Weblate, see #1194.

[nijel]

I feel like there is a use case for being able to edit/create developer_comments from Weblate UI.

There is now "Explanation" which can be set in Weblate UI since d1267e3 (it got renamed since then) and it is exported to CSV when downloaded. The field won't be parsed on upload as that seems too risky to mess up with the data when translating.