Multilanguage: Implementing a 301 when Removing URL Language Code #5129

infograf768 · 2014-11-17T07:21:11Z

See wrong PR here (because the true is not placed where it should) and discussion: #5092

Test instructions, set a multilanguage site with en-GB as default site

Remove URL Language Code is set to YES in the language filter
Language to display will be the default site language. Note it.
Search Engine Friendly URLs is set to Yes in Global configuration

When hovering on the language switcher module to get to the default site language, copy the link
It will be of the type http://mysite.com/en/ (if en-GB is the default site language)

Open this online checker: http://redirectdetective.com/
Enter the link you just copied. See results.
=>303 redirect
Apply patch, and test again the same link in http://redirectdetective.com/
=> 301 redirect

Here is a live example of a site where the patch has been applied:
Go to:
http://info-graf.fr/multi/fr/blog-fran%C3%A7ais-%C3%A5-tester/2-article-fr-fr.html

Hover the English flag of the language switcher.
The link is http://info-graf.fr/multi/en/home-en-gb/8-category-en-gb/1-article-en-gb.html
Copy the link in http://redirectdetective.com/ and Trace url.
It will show that the redirect is a 301

fontanil · 2014-11-17T08:59:28Z

@test
Patch works well for me. Thanks!

lunalars · 2014-11-17T09:06:32Z

@test
Success if "Remove URL Language Code" is set to "YES" (with and without SEF enabled in global config)

infograf768 · 2014-11-17T09:08:57Z

Looks like you are confusing SEF and SEF rewriting. :)
I guess you meant SEF rewriting on or off. Right?

ManuxGR · 2014-11-17T09:12:57Z

I have already test this, it's working for both cases "remove language code from url" set to "yes" or set to "no"

lunalars · 2014-11-17T09:12:57Z

@infograf768
Oh, yes :-)
Just checked again, and test is still successful.

ManuxGR · 2014-11-17T09:58:41Z

I will post back as soon as i have the duplicated content fixed. Right now i have about 350 duplications.

301 is working properly since 2 days. Probably in about 2 weeks they are going to be fixed.

infograf768 · 2014-11-17T10:36:40Z

I have already test this, it's working for both cases "remove language code from url" set to "yes" or set to "no"

In the last case there is no redirect indeed

ManuxGR · 2014-11-17T11:07:39Z

You are right once again.
I forgot i had also add "true" to line 315 and 320.

Hackwar · 2014-11-17T13:18:06Z

why are you doing the whole $uri->base().$uri->toString(array()) stuff? $uri->toString() would be the same...

infograf768 · 2014-11-17T14:31:02Z

@Hackwar
This is not the issue here. Some code comes from 1.6. beta and I am sure it can be modernised/corrected.
Let's separate issues please.

smanzi · 2014-11-17T16:08:09Z

@infograf768 Tested:

Condition are:

browser en-US
default site language: it-IT
secondary site language: en-GB

1st interaction. Being already in the Italian site go to another page:

2nd interaction. Language Switcher to English:

3rd interaction. Language Switcher back to Italian:

4th interaction. Login with user associated to en-GB:

5th interaction. Language switcher to Italian:

infograf768 · 2014-11-17T16:12:19Z

Good find!
This means we also have to send a 301 when the user logs in and has the default site language as own language, right?

smanzi · 2014-11-17T16:18:47Z

Well... it is more the 5th that concerns me from an architectural point of view (303 for a user logging in and being redirected is totally OK).
4th and 5th are a non-issue SEO-wise (spiders do not log-in, hopfully...), but IMHO and really without the minimal intention to offend, everything here speak "bad implementation", and I'm not talking about this PR: something is wrong ab initio, I think. Again, do not take this personally this is not my intention.

infograf768 · 2014-11-17T16:27:56Z

I did not get the 5th one here. Just the 303 when user logs in. When switchinh back to the default language I have a 301

infograf768 · 2014-11-17T16:29:14Z

Concerning the implementation:
The functionnality was not designed to be used without language code at the origin.
But people kept asking for it and we did our best...

smanzi · 2014-11-17T16:31:22Z

OOPS... that's strange... How can this be?
If you want I can try to replicate in the on-line version of my test site and give you admin rights...

infograf768 · 2014-11-17T16:32:03Z

I am using Tamper Data addon for Firefox to trace this.

Hackwar · 2014-11-17T16:34:14Z

@smanzi fully agree with you here. Having 7 $app->redirect()s in the parse rule with mostly the same code looks pretty bad. Likewise with half a dozen checks if self::$sefs[$sef]. Simply said, this code is so complex, that I have huge difficulties reading/understanding it. Equally with imploding the array at one point and concatenating strings at the other point.

smanzi · 2014-11-17T16:34:26Z

My screenshots are from the native Firefox inspector, Network view: try with that...

smanzi · 2014-11-17T16:38:59Z

@Hackwar pfuiiii.... you make me feel a bit less dull: I broke my head trying to get a grasp of it!

infograf768 · 2014-11-17T17:07:34Z

@Hackwar
Apart your mistake above in the code suggestion, you should very well remember which were the conditions Christophe and I had to deal with to force this minimal implementation of multilinguism in Joomla 1.6 and upper.

Evidently, the base is flawed!
This series will NOT provide a Joomla multilingual for big sites. It is a well known fact.

We were told to wait for the UCM, etc. to get a real multilingual Joomla!. I.e. was to be for version 3....
You very well know where ucm is now... Yeah, deep in it.

I have fought since I co-founded Joomla! for languages and multilanguage.
And I was very much alone until Christophe came with the code abilities, as we had nobody else volunteering... including yourself.
Then we happily had bembelimen who extended the associations.

Apart from these 3, no one has worked on it deeply. No one. We have simply corrected and improved on the base we had to be B/C and I had to always beg to get testers.

This PR is a little improvement for those who care for SEO.

infograf768 · 2014-11-17T17:12:25Z

@smanzi
I confirm 5. when switching to the default language I have a 301 (Firefox Inspector)

smanzi · 2014-11-17T17:20:33Z

@infograf768 OK, I will re-test, try to replicate on-line and then get in touch with you...

Hackwar · 2014-11-17T18:53:12Z

JM, I'm not saying that your work is not appreciated or that what you are trying to achieve is wrong. Quite the contrary. But I'm saying that when you are going to invest work into this, then please make it right and rewrite this plugin. I'm more than happy to help you out with this, but to be able to help you, I need to be able to understand what it is that you are trying to achieve with that plugin. The way the code is written, that is not possible right now, since it simply is too complex.

As you rightly said, UCM is dead. But I don't see why we couldn't get a proper multilang implementation in 3.x. And you know very well that I have to beg for every test and change as well.

smanzi · 2014-11-17T19:38:03Z

@infograf768 I confirm I have both 303 and 301 also with my on-line test site.

You can check at:

http://test.smz.it
Username: test-en
Password: test

If you wish, just tell me and I'll send you credentials for admin/FTP/cPanel

smanzi · 2014-11-17T19:40:08Z

... there is also an Akeeba Backup .jpa file to download, if you prefer...

smanzi · 2014-11-18T02:32:39Z

@infograf768
Well, you know something? We are both right!
It seems to depend from the fact if you have already switched language in your state - either not-logged-in or logged-in.

If you want to consistently see my behavior, after (or even before!) point 5 click again on the flag of the same language you are already into: then you will start seeing 303 + 301

Something similar happens also on the home page even for not logged-in users:

Starting with a fresh (no cookie) browser:

go to the home page: GET
switch to other language: GET
back to default language: 301 + GET
again to other language: GET
again back to default language: 303 + GET

smanzi · 2014-11-18T02:46:54Z

Some more info. Even for not logged in users:

go to some (not home) page: GET
switch to other language: GET
back to default language: 301 + GET
again to other language: GET
again back to default language: 303 + 301 + GET

smanzi · 2014-11-18T05:02:26Z

May it be possible that everything derives from the need of having (explicitly for NON-SEF, implicitly for SEF) the lang=xx parameter on the URI, even for the default language?

If without SEF enabled I go to the home page I get this:

GET http://example.com/
303 -> http://example.com/index.php?lang=it

Now think of this: if we modify the rules of the game and we say: we levy the need of the lang parameter and if it is missing from the NON-SEF URL we just serve the default language?
At the same time we should also alter a little bit the code of the Language Module.

In Language Module:

add a "?switchlang=xx" parameter to every URI in the "flags" (both SEF or NOT SEF)

in languagefilter:

if (we have the switchlang param set)
{
    set the cookie for it
    remove it from the URI
}


if (the lang=xx parameter or /xx/ is present)
{
    honour it, serve the page and be happy
}
else // no lang
{
    if (this is the home page)
    {
        if (the cookie is set && cookie != default language)
        {
            redirect 303 to the correct language home page (with &lang=xx or /xx/, depending on SEF)
        }

        serve the default lang home page
    }
    else // no lang, not the home page
    {
        if (SEF && !remove_default_language_code)
        {
            T.B.D.: probably just a 404 or a 301 to the page with /xx/, maybe according to a new option
        }
        serve the default language page
    }
}

Everything seems to me to be B/C too...
Am I missing some point?

smanzi · 2014-11-18T05:12:54Z

the switchlang=xx parameter should also be added to the login logic...

infograf768 · 2014-11-18T07:16:43Z

@smanzi:
Ref: #5129 (comment)
I still do not get any 303, even on your site.

Please test my demo site above

and

If without SEF enabled I go to the home page I get this:
GET http://example.com/
303 -> http://example.com/index.php?lang=it

No, it is impossible and I do not confirm it at all. SEF off we always get a 200 directly.
(Except when it is a first visit and the param is to Browser settings, but this is to be expected)

infograf768 · 2014-11-18T08:19:55Z

@smanzi
my skype is infograf768

Let's share screen there and I will show you

stellainformatica · 2014-11-18T10:21:31Z

@test
The patch works as expected, with the patch applied I see now a 301 redirect

smanzi · 2014-11-18T13:06:27Z

@infograf768
Jean-Marie, I'm getting 303 + 301 on your site too (for English page)... then It must be something else: browser?

My site is stock 3.3.6 with the following exceptions:

Installed Akeeba Backup
Installed com_patchtester
Installed a content plugin of mine that does markdown parsing

I'm going to open Skype later (about 1 hour...) Don't you have TeamViewer for screen sharing? I normally use that... never tried with Skype...

smanzi · 2014-11-18T15:38:22Z

smanzi · 2014-11-18T15:43:48Z

On Jean-Marie site try switching between fr-FR and en-GB several times: at one point it start issuing 303 as well. Once started it does consistently. At least, this is what I'm seeing here: if nobody else has this problem it must be something local on my PC (Windows 8.1, Firefox 33.1)

smanzi · 2014-11-18T15:47:44Z

303 is to: http://info-graf.fr/multi/en/home-en-gb/8-category-en-gb/1-article-en-gb.html
301 is to: http://info-graf.fr/multi/home-en-gb/8-category-en-gb/1-article-en-gb.html

infograf768 · 2014-11-18T16:28:26Z

I think we are going to merge this as we have already many positive tests and you are the only one to get a 303 for current navigation.
Getting two 3xx redirects before the 200 is anyway weird. I have never seen that on the web,
(BTW, all tests should be done on staging, not on 3.3.6)

I am on skype right now if you want to see here what goes on

ManuxGR · 2014-11-18T17:02:14Z

Sorry but i can not follow sometimes, it gets complicated for me. But if you need any help testing or an opinion let me know.
Anyway, my skype is manos-sx feel free to add me and ask for help.

smanzi · 2014-11-18T17:20:56Z

@infograf768
Hi Jean-Marie: sorry but I've been out for a while...

I think we are going to merge this as we have already many positive tests and you are the only one to get a 303 for current navigation.

As you wish, but as I told you in the other PR I have other concerns about having many links generating 301 on a site and namely:

Is this going to have a SEO impact?
Is this going to create problems to sitemap generators?
Will this litter my Webmaster Tools?

Getting two 3xx redirects before the 200 is anyway weird. I have never seen that on the web,

I trust you, but as you've probably seen on my last screenshot I'm getting those also on your site...

I will probably be on-line later if... I don't have to rush to the hospital: kidney stones are haunting me since a couple of days 😧

Hackwar · 2014-11-18T18:35:13Z

Providing a bunch of improvements to the languagefilter plugin: #5135

lunalars · 2014-11-19T07:07:50Z

I think i found a way to reproduce the 303s ...
Tested on Ubuntu 14.04 with Firefox 33.0 (Firebug and Firefox Dev Tools) and Chromium 38 (Dev Tools).

First clear browser cache and then go to http://info-graf.fr/multi/

In Firefox switch to italian, french, italian, english (301), italian, french, italian, english -> 303 and you are still on italian.

In Chromium i had to go through this cycle one more time and on the last switch to english i get 301 + 303 and am still on italian.

Now deactivate caching in Firebug and Chromium Dev Tools (didn't find a way for Firefox Dev Tools) and try again: all should be fine (no more 303).

So maybe the question is: do crawlers use cache?

infograf768 · 2014-11-19T07:22:19Z

I guess that crawlers do not use your browser, therefore not the browser cache.

infograf768 · 2014-11-19T10:54:04Z

This patch is now included in Hannes PR #5135

infograf768 · 2014-11-20T07:57:17Z

#5135 includes this code. Closing.

Change language string names Add extra large class to field Don't check for empty new url if in advanced mode Remove not null constraint in db Remove not null constraint in db (2/2) Add extra check for a 3xx code Port to all databases Add missing update files

Multilanguage: Implementing a 301 when Removing URL Language Code

0f7d735

jissues-bot added the PR-staging label Nov 17, 2014

This was referenced Nov 17, 2014

[#33194] change language redirect with language code from 303 to 301 #5092

Closed

Language Switcher: Remove the language code for the default language #5109

Closed

should also work when sef_rewrite is Off

08351c4

Hackwar mentioned this pull request Nov 18, 2014

Initial codereview of languagefilter plugin #5135

Merged

infograf768 closed this Nov 20, 2014

Multilanguage: Implementing a 301 when Removing URL Language Code #5129

Multilanguage: Implementing a 301 when Removing URL Language Code #5129

Conversation

infograf768 commented Nov 17, 2014

fontanil commented Nov 17, 2014

lunalars commented Nov 17, 2014

infograf768 commented Nov 17, 2014

ManuxGR commented Nov 17, 2014

lunalars commented Nov 17, 2014

ManuxGR commented Nov 17, 2014

infograf768 commented Nov 17, 2014

ManuxGR commented Nov 17, 2014

Hackwar commented Nov 17, 2014

infograf768 commented Nov 17, 2014

smanzi commented Nov 17, 2014

infograf768 commented Nov 17, 2014

smanzi commented Nov 17, 2014

infograf768 commented Nov 17, 2014

infograf768 commented Nov 17, 2014

smanzi commented Nov 17, 2014

infograf768 commented Nov 17, 2014

Hackwar commented Nov 17, 2014

smanzi commented Nov 17, 2014

smanzi commented Nov 17, 2014

infograf768 commented Nov 17, 2014

infograf768 commented Nov 17, 2014

smanzi commented Nov 17, 2014

Hackwar commented Nov 17, 2014

smanzi commented Nov 17, 2014

smanzi commented Nov 17, 2014

smanzi commented Nov 18, 2014

smanzi commented Nov 18, 2014

smanzi commented Nov 18, 2014

smanzi commented Nov 18, 2014

infograf768 commented Nov 18, 2014

infograf768 commented Nov 18, 2014

stellainformatica commented Nov 18, 2014

smanzi commented Nov 18, 2014

smanzi commented Nov 18, 2014

smanzi commented Nov 18, 2014

smanzi commented Nov 18, 2014

infograf768 commented Nov 18, 2014

ManuxGR commented Nov 18, 2014

smanzi commented Nov 18, 2014

Hackwar commented Nov 18, 2014

lunalars commented Nov 19, 2014

infograf768 commented Nov 19, 2014

infograf768 commented Nov 19, 2014

infograf768 commented Nov 20, 2014