Add ability to determine ID of file uploaded via import API

[siepkes]

While the tolgee-cli tool is a great tool I couldn't really get it to work comfortably with a Java backend application (which is a Maven multi-module project).

I currently have a working prototype of an application (which I intend to open-source) which makes it easier to use Tolgee with Java Maven multi-module projects (we have a lot of those). It consists of a CLI application and I plan to later on create a Jenkins plugin based on the same framework. It should work for most backend-style projects (or web projects which are more "backend orientated" such as Vaadin, htmx, etc.).

Basically what it does is it allows you to put a simple (TOML based) configuration file in directories where messages files are (for example files like Messages_de_DE.properties). The application then finds all these configuration files, then the messages files and uploads these to the Tolgee importer. The neat thing about this is that one can use the Tolgee interface for conflict resolution and such. It also works the other way around by calling the export API and locally updating the message files.

However, one thing I'm bumping in to is I would like to set the language and namespace of the files uploaded to the importer. But in order to do so I need the importFileId when calling the appropriate API's. But after uploading a file by calling /v2/projects/import it does not return the importFileId of the file which was just uploaded. Instead it returns all the files which are currently waiting to be imported. And I don't really have an reliable way of determining which file was just uploaded.

Would it be possible to extend the response in such a way that it becomes clear which ID was added by uploading the file? For example to add the "Import file ID" (I think this is also sometimes called the languageId in the API) in the response of the "Add files" (/v2/projects/import) endpoint of the file which was just uploaded?

[JanCizmar]

Hey! We plan to add exactly this features to the Tolgee CLI: tolgee/tolgee-cli#74
While also adding single step import allowing you to provide the data you normally set in the UI in the request. #2142

Is the stuff covered in the CLI issue what you looking for?

[siepkes]

The single step import API seems interesting! Judging from what I can see so far it seems like a good API fit what I'm doing. Do you have a (very rough) idea when the API part will be available in Tolgee?

There also seems to be a bit of overlap with the plans you folks have for tolgee-cli. Seems like a good addition for tolgee-cli (I mean I wouldn't be implementing roughly the same thing if I didn't think it had merit 😉 ). Though what I'm doing might work still be beneficial along side tolgee-cli for doing a Jenkins pipeline plugin (since it's a pure Java based thing). I don't know if you folks had any plans for that?

This is the readme for the tool I was working on to give you some idea what my (rough) idea's are: https://gist.github.com/siepkes/8537a30703922838b656c306da758b86

[siepkes]

@JanCizmar FYI: I've opensourced our prototype tool here: https://github.com/serviceplanet/tolgee-toolbox and created a discussion here: #2206 .

[JanCizmar]

Hey! Thanks a lot for that! 🚀

[JanCizmar]

Have you actually resolved the issue in the original question? I guess I didn't answer your question.

[siepkes]

So for now what we do is we just generate a UUID and specify that as file name and look it up in the results. Long term I'm thinking about using the one-step importer.

Though there might also be some merit to keep the current implementation. It's kinda useful to upload to the importer and then being able to resolve any conflicts in the web UI. That way we don't have to duplicate any conflict resolution UX. So in that sense "knowing which file you just uploaded" in the results would still come in handy.

[JanCizmar]

Interesting. I thought there is usually no reason to resolve conflicts when you import data using the CLI, because you always have single source of truth. When pushing the strings from the codebase, it's usually the codebase for the base language and Tolgee for the other languages.

The issue for the multi-step import is that there is one import instance per user and project, which makes it not optimal for CLI use, since multiple runs can affect each other. This was a bad design from the beginning.

[siepkes]

Interesting. I thought there is usually no reason to resolve conflicts when you import data using the CLI, because you always have single source of truth. When pushing the strings from the codebase, it's usually the codebase for the base language and Tolgee for the other languages.

I hadn't really considered the actual use of conflict resolution. But when you put it like this it probably doesn't have that much use for tolgee-toolbox (I only recently thought it was kind of a missed oppertunity not to call it "toolgee-box" instead 😉 ).

Once the one-step import is landed in Tolgee I'll give refactoring to it a shot and see how that works. I think it should work pretty well.

Once that's done we plan on taking a look at a Jenkins pipeline plugin. It's pretty useful to have Jenkins push the translations to Tolgee for the CI build for us.

The issue for the multi-step import is that there is one import instance per user and project, which makes it not optimal for CLI use,

That's definitely true and something we also considered when creating the tool. We even briefly thought about expanding the import functionality in Tolgee by allowing multiple "import sets" to be created so to say. Since the single step importer also takes care nicely of this issue I'm looking forward to refactoring tolgee-toolbox with it.

[JanCizmar]

Hello! Thanks again for creating tolgee-toolbox. Since we are going to work on the CLI enhancements soon, I am closing this for now. Hopefully, the issue will be resolved with single-step import implementation, which is integral part of the Tolgee CLI 2.0.