-
Notifications
You must be signed in to change notification settings - Fork 17.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: cmd/go: add language localization support to go doc #57254
Comments
Here’s another scenario:There’ an existing go package I’d like to import to my program, and (unsurprisingly) the doc is written in English. One way is to create my own version of documentation separately, guiding people to use the package. But this kind of documentation lacks the interactive catalog in a pkgsite manner, as well as IDE support in the form of a hint bubble. To get these functionalities, one would need to edit the code comments in the local cached version of the package source, which sounds like a nasty thing to do. There’s golang-china/golangdoc which modified the x/tools/godoc package to provide a switch to the Chinese version of documentation. However, this only applies to the modified godoc http frontend interface, and the workflow of translating docs for this golang-china/golangdoc to use is poorly automated. As a result, the repo (as well as the repo containing translations) has not received a commit for 4 years. |
Why don’t I try to fix/improve golang-china/golangdoc metioned above?Well, key point is, I really hate the design of golangdoc, as it uses the modified version of package source code as alternative source of godoc. This is just slightly different from
only difference is making a copy prior to editing, and afterwards making the source feed of godoc switchable. While it may fit the need of hosting a mirror site of godoc.org with translations, the design is counterproductive, since it defines the translation workflow as modifying the original package codebase and committing the modified version to the “golangdoc” repo, not the respective repo of the original package. By doing this, golangdoc is not eliminating language barriers for the co-development of open-source projects, on the contrary it made copies of repos that aren’t properly linked to the original version, making the “Chinese (or whatever language) mirror” isolated from the mainstream. |
How would I expect it to be like for an ideal workflow of writing go doc with translations?
|
Dupe of #21666 |
It's not clear this is a dupe. #21666 refers to x/website documentation. This appears to be more fundamental - enable This would benefit from a concrete proposal addressing the various challenges. |
When I've worked with professional translators to localize software in the past, their typical workflow has required providing them the source text to translate in some specific existing file format like the Gettext However, that has tended to be UI text rather than documentation. I'm not sure if a translator who specializes in prose-style technical writing rather than UI text would have a similar requirement. Do you have experiences with that, @Sunshine40? I mention this only because it might help decide what is a good format to use either as the final storage format for translated text or at least for a separate translation-workflow helper program to be able to produce and consume. Separately: it sounds like the initial focus here would be on supporting localized docs in Is there a need (or even just a desire) for any other tooling to be able to consume localized docs? I assume that each additional thing which needs to be able to consume localized docs will add some more constraints on possible solutions, such as which component(s) would be responsible for finding, loading, and parsing separate localized docs files. |
@apparentlymart The answer to the last question is no, I don't have the exact experiences, but having worked on projects with i18n support does give a hint.
What we did was in an even more primitive way. The dev team defined where the translated text fields should locate, created spreadsheet files listing those text fields, and provided screenshots of the original UI alongside the sheets. The professional translators then populated the sheets, and the dev team manually inserted the translated text back to an equivalent of the Gettext .po file. Technically our dev team was doing more “baby-sitting” than in your description, but I don’t see a real difference in the principles of a reasonable workflow, namely:
For the 1st point, there might be an addition as we arranged the translations to different languages of a single UI in the same sheet, which proved helpful since the original text is in Chinese. Regarding the 2nd point, actually using screenshots gives more vivid images than simply grouping related messages together, only downside is that it hinders the automation of the workflow. Back To This ProposalAs for the difference in document/UI translation, I think the requirements would mostly be the same. The key point is that the translator might (in most cases) not be the original author, and he would need the above-mentioned information to do this job.
If I read it correctly, this is a matter about API and, well I mostly want localized docs to be consumed in a way compatible with how existing docs are consumed, with an optional language flag/parameter. In addition to the language option itself, other things that are important for the doc consumer API would be:
A Proposal for a Simpler ImplementationNow that I’ve come to think of it, the quickest way to get go doc localization working, in the form of a third-party plugin, might be focused in:
Such implementation would certainly have no effect on Nevertheless, it would be enough if you’re ok with viewing localized doc through a local instance of pkgsite, viewing/editing localized doc in code comments and get localized IDE hints, after a manual swap doc language command on the repository. The merit of this implementation is that it makes absolutely no change to
@mpx I've listed some of my thoughts above. Any advice or further question is appreciated. However I still hope language localization support would be integrated in PS:Ideally all doc localizations should be treated the same way, with no difference between the “original localization” and the “translated localizations” (one might set a preferred language option for reference though, and a localization file might specify another language as a default reference base / fallback option). After all, when it comes to a worldwide-contributed open-source project, it’s not reasonable to force a single language as the “official language” for documentation. Such idea is the initial inspiration of this proposal – it’s not about “asking for an option to make Chinese (or another language) the official language for the go doc of a package”. |
When talking about the necessity of doc translation, no one can stay unbiased(Warning: this comment is more or less complaining about the status quo, you can skip it if you're not interested in the non-tech details) For example, I’m a native Chinese speaker, maybe I’d hesitate between choosing Chinese / English as the doc language, but Japanese? I shouldn’t care about whether a package has Japanese documentation, as it isn’t MY preferred language, nor is it the default language (English) in exchanging ideas internationally. A Japanese developer would think the same way as I do, albeit changing “Chinese” for “Japanese”. Same goes for doc translation for existing packages. When googling documentation for go packages, I could find a portal hosting the Chinese translation for the Go StdLib documentation collection. I can get the image of “Chinese go developers matters” “packages ought to come with Chinese doc translations to be widely used by Chinese developers” from my point of view, but what about others? If someone published a package with Chinese doc only, then hardly anyone who doesn’t speak Chinese would bother to import that package. I, as a matter of fact, would never import a package with only Japanese doc published on Truth is, if you don’t provide English documentation, users of your package would be filtered by the doc language you choose, with rare exceptions. And the repository would get limited feedback/contribution as a result. So why are you so stubborn in writing docs in Chinese?There are situations where English as the only doc language option is simply not suitable. I’m current working on a strategy(card/board) game simulator, an implementation of which is the famous game, mahjong (in its Japanese variation). Given the fact that I have already been struggling through terminology translation catalogs, in order to keep the naming of Types/Methods and Variables from being flooded with pinyin/romaji, what would happen if I restrict my self to use English in the “official” documentation? Supposedly, the users of the package would firstly be Chinese speakers (my friends / mahjong enthusiasts in China who I can advertise my package to), they would prefer Chinese>English=Japanese as doc language. Secondly, mahjong pro players from Japan, who actively seek tools to do research on the game, might find the package (if I add a sentence to doc overview briefly introducing the package in both English and Japanese). They’d prefer Japanese>English=Chinese. Finally, there’re other mahjong enthusiasts who don’t speak Chinese or Japanese. Of course, Chinese/Japanese doc is rendered useless to them, but would an English documentation provided by me be adequate? Mahjong is a game originated from China, and got popular in Japan. There’re lots of people writing/translating books on this topic in both countries. In doing this, the terminologies in Chinese/Japanese got widely accepted. The English world, on the other hand, hasn’t made the terminologies standardized when it comes to mahjong. I often come across a concept which has 2 or 3 English translations, as well as people referring to it with romaji directly. So how am I supposed to know which expression is most widely accepted? So much for complaining, but the point is clear. Providing English doc instead a Chinese one, in this specific case, would make the most users who speak Chinese inconvenient/confused, bring myself more trouble disturbing code development to think of a proper expression to write the English doc, do no good to the potential Japanese users, and provide low quality translation to the English-speaking community. And mahjong is probably not a unique case facing this problem. This is where community contribution should kick-in IMO, with me providing Chinese doc as the first choice explaining everything I can explain, and English doc as an alternate option, waiting for contributor refinement. But in current go package publishing workflow, the package probably wouldn’t even reach the English-speaking community without providing the package doc in English via |
There's not an obvious technical answer to this. Supporting docs in multiple languages in a systematic way is significant complexity for really quite small benefit in general. In this specific case, where you expect to have many readers of different languages without a shared language, it seems fine to write both in the comment:
If that became widespread then it might make sense to do something more systematic, but it seems like if that was going to happen, it would have happened already. Also, if LLMs are good for anything at all, it should be language translation. I wonder whether it makes sense to have a third-party tool that is like 'go doc' but translates the output to a new language. |
This proposal has been added to the active column of the proposals project |
Based on the discussion above, this proposal seems like a likely decline. |
How is this a likely decline? There has been no discussion at all! I am disappointed by the attitude displayed here. In some countries, there are multiple official languages, sometimes even three or four and for some projects there needs to be written documentation in all of them, usually plus English as well. Just putting the documentation with all languages together makes it very hard to read. The simplest solution for this problem would be to support something like //go:lang:[language code] tags in go doc documents, and then filter and display documentation based on the user's language env LANG settings. An other simple solution would be to allow for every foo.go file a foo.[language code].godoc file with the doc translations and signature only functions. Please consider more seriously the language related problems many Go users are facing. |
This sounds simple but then we have to maintain all that infrastructure, think about which languages to translate our doc comments into, and so on, for the lifetime of the Go project. It adds significant complexity, and Go is maintained by a relatively small team. |
So the problem is a lack of resources if the Go team then? That's too bad but somewhat understandable. People in my or the OP's situation will have to deal with it ourselves then and make our own tools. |
No change in consensus, so declined. |
Think of this scenario:
I’m working on a go package, which is intended to be open-source. But the original dev team consists of native Chinese speakers only.
Meanwhile, I hope the package can be used by developers around the globe, and get feedbacks/contributions from them.
In what language should I write the comments (which would be auto-documented by the go doc cmd) then?
You see, the goal of development efficiency and documentation for potential users who speak different languages contradict each other, when it comes to deciding a single language to be used in go doc comments.
Using English in go doc comments from the beginning of a project is an option, chosen by a number of developers who are not native English speakers. But it would break the internal usage of comments, especially when members of the dev team might not be fluent in English.
“Restricting go doc comments to English text” as a coding style guide may lead to most developers leave the comment sections untouched, resulting in outdated or absent documentation.
The text was updated successfully, but these errors were encountered: