Native Multilingual support in Hugo. #1734
Conversation
|
|
| if p.lang == "" { | ||
| return outfile | ||
| } | ||
| return p.lang + "/" + outfile |
There was a problem hiding this comment.
I would use filepath.Join here -- to get the filepath separator correct
There was a problem hiding this comment.
filepath uses an OS dependent path.. is that what we want here ? thought it was destined to build URLs.. I'll have to look into more details how URLs are generated.
What do you think of the general idea though ?
There was a problem hiding this comment.
On line 940, we take a url-style path and convert it to an OS dependent path with filepath.FromSlash. filepath.FromSlash will convert posts/index.html to posts\index.html on Windows.
@bep is correct. In this context, you want to use filepath.Join.
There was a problem hiding this comment.
I like the general approach.
On Tue, Dec 29, 2015 at 11:11 PM Cameron Moore notifications@github.com
wrote:
In hugolib/page.go
#1734 (comment):@@ -937,5 +953,12 @@ func (p *Page) TargetPath() (outfile string) {
outfile = helpers.ReplaceExtension(p.Source.LogicalName(), p.Extension())
}
- return filepath.Join(strings.ToLower(helpers.MakePath(p.Source.Dir())), strings.TrimSpace(outfile))
- return p.addLangPrefix(filepath.Join(strings.ToLower(helpers.MakePath(p.Source.Dir())), strings.TrimSpace(outfile)))
+}
+func (p *Page) addLangPrefix(outfile string) string {
- if p.lang == "" {
return outfile- }
- return p.lang + "/" + outfile
On line 940, we take a url-style path and converts it to an OS dependent
path with filepath.FromSlash. filepath.FromSlash will convert
posts/index.html to posts\index.html on Windows.@bep https://github.com/bep is correct. In this context, you want to
use filepath.Join.—
Reply to this email directly or view it on GitHub
https://github.com/spf13/hugo/pull/1734/files#r48586001.
|
@spf13 I was also looking at https://godoc.org/github.com/nicksnyder/go-i18n/i18n ...wouldn't it be awesome if, in addition to good support for i18n in terms of content, we had great translation capabilities? We could load I can see quite a nice mapping from the examples there to something Hugo-ish: to something like Or I'd add a native What do you think ? What's your take on vendoring other libraries ? |
|
I like this idea a lot. I think it should be an independent commit/branch
|
|
Also see golang/go#12750 |
abourget
force-pushed
the
master
branch
3 times, most recently
from
ed91974
to
7beb617
Compare
|
Oh I thought appveyor was broken for other reasons, but Multilingual mode breaks under Windows.. will fix and push again. |
|
@spf13 do you agree with the general approach ? Think it can be merged eventually ? |
|
Totally. Well done so far. |
abourget
changed the title
|
ok, this should be ready. works on Windows / Linux.. anything missing that you see ? |
|
This is very VERY good. In reviewing I had a few ideas in ways we can further refine this so that it's a bit easier to use.
I would think perhaps the best accomplish to do this would be the following approach. Each piece of content has a base file. This is the one without a language extension. It would specify the metadata and content that each enabled language would use when rendering if the given metadata or content wasn't present for that specific language. For things like categories and taxonomy it would mean that you would only need to specify them in the base and apply the translation to the terms. This would keep the content nicely in sync. I believe that with translations that usually the case that a partially translated site is better than a partial site. By taking this approach any content that isn't yet translated will still be available, but in the base language. While in my personal sites I have little experience with this and in particular the subtleties of the actual translation, I was responsible for the documentation of both MongoDB and Docker and we dealt with this a lot there. This follows more closely the approach we took with those sites. Alternatively if you wish to keep the content independent you could just not provide the base and this approach would still work. Does this make sense? I'm happy to merge the existing code and discuss this further as we look to implement this further. |
|
Dear @abourget, Thank you so much, really, from the bottom of my heart, for your bravery and devotion in tackling multilingual support for Hugo. Indeed, after seeing the multilingual efforts at Drupal, especially how it has evolved in Drupal 7, and now in D8MI (Drupal 8 Multilingual Initiative), I came to the appreciation how immense and complicated this subject can be, and I had been shying away from it. Thank you @abourget for stepping forward and facing the challenge head on, and thank you for your thoughtfulness and extensive work that have gone into it thus far. There are several use cases that I am personally interested in about Hugo's multilingual initiative. For example, Hugo's multilingual support should:
But indeed, instead of speculating, I should start actually trying out your excellent PR and try building test websites with the above scenarios and see how it works. 😁 I will go experiment and report back. By the way, do you already have some example/test websites that can demonstrate the multilingual features that you have written thus far? :-) Many thanks again! Anthony |
|
For what it's worth, for us to do what I suggested above we need to first transition into a new content data model where a single file no longer is tied to representing a page. I'm currently working on the selective reading (only reading files that have changed) into Hugo, but I think I'm going to tackle that next and will do it with multilingual in mind. |
|
quick note @spf13 .. the "DefaultContentLang" does your 1., if |
|
Yeah a simple site is mine: https://blog.abourget.net Le jeu. 7 janv. 2016 13:50, Steve Francia notifications@github.com a
|
|
The core idea here is linking the translations using a base filename ( Regarding history and diffing, Git is the best we could bring to the table. Collaboration through GitHub and organizations there is really all you need, especially on large sites. Simple tooling could be built to diff the changes to the "base" language that occurred after the last translation occurred for each piece of content... so smart git-fu and we're done. Perhaps this could be added to hugo itself a bit later. All of that, thanks to filename-based linking. The next thing most important I think is what we offer to the template designer. Right now, a Now there are many ways to address i18n in a CMS. My references come from Typo3 couple of years ago, where they offered two superbly divergent i18n models:
In both cases, the pieces of content could be translated and linked together. This PR currently implements this behavior: the Now this isn't implemented here, but we could have a top-level switch for two different behaviors:
@anthonyfok regarding the @spf13 regarding translation of the taxonomies.. I feel this should be solved using my other PR, i18n, which adds a sort of native place for translations. On my site, right now, I'm using a hacky @anthonyfok regarding the
@anthonyfok also, regarding the zh.wikipedia.org .. this PR does not facilitate the generation of subdomains. For that I thought maybe in the future we could support a translatable @spf13 regarding base metadata, if we had feew.. does that make sense ? sorry for the wall of text! :) |
|
I have just taken the first step to test your PR: by trying to add multilingualization (m17n) to gohugo.io documentation. What I have done so far (very little):
Speak for myself as a newbie and a somewhat lazy user, some of expectation at this stage (after merely enabling multilingual mode) would be:
That said, I think what you already have is wonderful and is a great step forward, but there are a few details that we need to "get it as right as we can (@spf13)" first before presenting it for the first time to our end users, "especially the interface, that's the hardest thing to change for users (@spf13)". So, I think it is time that we start a new topic branch—let's call it |
Totally agree. Also with the other stuff you are saying, and you do a fantastic job with this thorough testing. But I don't think a topic branch is a way to go at this point. This is a complex problem to solve, but it should be possible to get this PR up to a working state before involving other people. |
|
Adressing a couple of things here @anthonyfok:
|
Sorry, I wasn't clear in my previous comment. What I was thinking, first and foremost, was actually how to get abourget#1 into the fold. With the newly created Also, I was also thinking, because it is such a complex problem to solve, we might need several PRs together to get all the features in, and working in the (Though I was also hoping I could contribute a PR or two on top of @abourget's existing PR to address the minor issues that I pointed out above. 😉) Nevertheless, whichever way we work doesn't really matter too much. Branches in Git are cheap, and we can create and delete them any time we wish. If you prefer working entirely within the |
|
I've got a problem when rendering menu's. If I enable "SectionPagesMenu" my sections are added to the menu but with an URL without the language prefix. For example the URL for the "post" section just becomes "/post/" instead of "/en/post/". The menu items defined in the front matter work correctly, i.e.. have the language prefix in the URL. Also, I would expect the "menu" tree to be defined in the Multilingual block so it is possible to create custom localized menu entries. Or am I missing something? One last item: There is a great example of using "apply" to generate tags and categories URL's. But because variables in a partial are limited to the scope you have supplied it is not possible to add the $.Site.CurrentLanguage variable to the URL you generate. I've now solved it with the "range" command but I really liked the simplicity of the "apply" example. Perhaps I'm missing something to get this working. |
|
@abourget what is the status of this PR? I can understand people having life outside of open source development, but this is dragging out. If we cannot close this PR in a week or two I suggest we just close it, so we don't getting people's hopes filled up for no reason. |
|
@jasongonzales23 I'm not sure I understand the |
|
Last time I checked, this PR was in good shape.. I just didn't have time to address the list of comments.. I think mostly they are about better documentation.. and I agree with that. I'd like those who test-drive the PR to add to the docs directly.. as it will come from experience.. I'm not 100% sure I'd cover all the things merely by reviewing or addressing a few comments. The other thing is about exported/unexported vars. I don't have strong opinions about exported/unexported Go functions.. as I don't personnally use The only thing that I'd want to get right are the variables exposed to template developers.. like I'd also need to rebase this PR.. Since 0.16 is released, can we merge it when I rebase it.. so everyone can start contributing to it ? in terms of docs, or tweaks.. I think we agree on the large chunks.. and having it in the master branch would allow everyone to start poking at it. |
|
Checking at the conflicts just now.. and merging it after this rebase would be great, so I don't have to deal with things like that down the road: - switch v.(type) {
+ switch val := v.(type) {
case bool:
++<<<<<<< 5388211c1158b81725af04152c6e73eddc1435a0
+ return v
+ case time.Time:
+ return v
++=======
+ return val |
|
Ok, just rebased. Not much was in conflict. Tests passed over here. We'll see how it goes. @bep can you list the blockers to a merge ? I'll make sure I address them, and then let's merge and have the community jump in to fine tune with the different use cases. I use this branch on 2 sites right now, and planning a third.. so I'll put my hands back into it soon. https://golangmontreal.org and https://blog.abourget.net/ you can check the source for both sites ( https://github.com/gomtl/golangmontreal.org and https://github.com/abourget/blog ) |
|
I've tweaked a few of my previous comments, perhaps you want to read them on Github instead of by e-mail :) |
|
@abourget, thanks so much. I am using this branch in prod, so would very much see it merged in soon! |
|
@abourget my main comment is still untouched: |
|
Ok @bep I pushed fixes and simplifications. |
|
|
||
| ``` | ||
| {{if .IsPage}} | ||
| {{ range $txLang := .Site.Languages }} |
There was a problem hiding this comment.
This and below isn't what I talked about in my previous commit, is it?
There was a problem hiding this comment.
yes.. that's the simplest form I found and use.. if you have something better to propose.. I'm all ears.
There was a problem hiding this comment.
I have proposed a simpler form above.
There was a problem hiding this comment.
I do not see how the form I documented can be simplified. Your propositiin doesn't take into account the differences between Pages and Nodes and the potential different decisions a user might take.
I see this as complete , unless we want to abstract those decisions into yet another concept different than Pages and Nodes, something that would unite them into a Translatable or something.
Implements: * support to render: * content/post/whatever.en.md to /en/2015/12/22/whatever/index.html * content/post/whatever.fr.md to /fr/2015/12/22/whatever/index.html * gets enabled when `Multilingual:` is specified in config. * support having language switchers in templates, that know where the translated page is (with .Page.Translations) (when you're on /en/about/, you can have a "Francais" link pointing to /fr/a-propos/) * all translations are in the `.Page.Translations` map, including the current one. * easily tweak themes to support Multilingual mode * renders in a single swift, no need for two config files. Adds a couple of variables useful for multilingual sites Adds documentation (content/multilingual.md) Added language prefixing for all URL generation/permalinking see in the code base. Implements i18n. Leverages the great github.com/nicksnyder/go-i18n lib.. thanks Nick. * Adds "i18n" and "T" template functions..
.Page.Translations now excludes the current page from in there..
Applied .AllPages throughout and fixed some tests, refactored a bit to avoid massive duplication. Got rid of unused `possibleTaxonomies`.
|
@jasongonzales23 the latest code should solve a bunch of your issues.. namely the LiveReloading not picking up translations (because of the way .Pages and .TranslatedPages worked).. Now .AllPages includes all translations and watch all those files for changes before rebuilding. Everything is rebased on master now. |
|
I really want this feature. When I didn't know about this issue. I've tried to make multilingual different way. The section is the language type what I want to publish. |
|
When merge? :) |
|
I really want this feature too. Is there an estimate when this will be merged? Will I be safe if I use @abourget his fork in production? I really need this for production sites or I am afraid I have to resort to something different |
|
@spil-rkoopmans It will be merged when it is solid enough to merge, estimate is impossible to make in an open source project like this with contributions from unknown entities. The API will change, I can say that, so any use of this PR will have to adapt. |
|
thank you @bep. I understand. I have decided to go with Hugo anyways using the method described in the v0.16 documentation. I really love this product and think you guys are doing a great job! We will see what happens in the future and adjust our project accordingly! |
|
Merged into https://github.com/spf13/hugo/tree/multilingual I will walk this one into master when I get some time to spare. Thanks for the great job. |
|
Continued here: #2303 |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
The goal is to:
where the translated page is (with .Page.Translations)
(when you're on /en/about/, there's a "Francais" link
that points to /fr/a-propos)
Missing:
{{ ( index
to do server-side translation if we want all the bells and whistles
of a translation engine.. Has tooling to build translation workflows
too.
something that writes the root "index.html" and redirect to something
that either considers the users' browser, or has a fixed language
redirection.
This is basic code to open up a discussion as to how best we could handle that.
I was thinking of exposing more
Translationsfunctionalities directly in the.Site, to crawl through one set of pages (instead of looping through all Pages, even though some are duplicated). Something likeBaseLanguagePages? and all thosePagewould have pointers to their translations.We could run the generation once for each language, and each time generating a
/fr, or/ensubtree.. consistent in itself.. and knowing about "the other side" of translations (through .Page.Translations). In that case, maybeSite.Pagescould have only those of the current translation,but
.Page.Translationswould know of the others.. that might simplify menu generation and all, as itwould never need to know about other translations, unless explicitly asked to.
So @spf13, what do you think of all that ?