Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Routing: Implementing routing stages #5264

Closed
wants to merge 15 commits into from

Conversation

Hackwar
Copy link
Member

@Hackwar Hackwar commented Nov 30, 2014

The issue

The current system only allows to modify the URL before it is send to the component router. This means that for example the language filter plugin has to add the SEF prefix before the component router is even started and also has to delete the language value from the URL before that time. That means, that that data is not available for the component router. This is a more general problem that our routing system is very inflexible.

The proposal

This PR implements a 3-stage processing for the URLs: preprocessing, main stage and postprocessing. This allows the language filter plugin to add the SEF prefix in the preprocessing stage and to delete the language query parameter in the postprocessing stage, making that data available to the component router in between. At the same time, this introduces deprecation notes for all the methods that should be handled as rules in this 3-stage process.

Backwards Compatibility

This PR should be backwards compatible. There are 4 methods that have been changed, the methods to attach rules and the methods to process these rules. Since the new, second parameter is optional, existing code should not create any issues. Existing rules are still stored in their respective arrays and only the pre- and postprocessing steps add new keys to that array, which would otherwise be ignored. That said, no old software should be affected by these changes. The position of the main stage for parse and build, which represents the rules that we have so far, does not change either, which should prevent any changes in our intermediate results.

Personal thoughts

All the code that is in the methods around these process*Rules methods should go into a rule and be pushed into the specific stage. The problem is that that change WILL change the way our routers behave towards third party software. That is most likely a break in backwards compatibility, which would force us to postpone that step until Joomla 4.0, which frustrates me immensely...
Another thing that bugs me personally is, that we call the main stage '' in the method parameters, which looks bad. However that makes the nicest implementation from my perspective.

How to test

Testing here means that nothing changes between applying these changes. So please first check that your site works fine, then apply the change and see that it still works fine.

This PR needs a review by a maintainer.

This was made possible through the generous donation of the people mentioned in the following link via an Indiegogo campaign: http://joomlager.de/crowdfunding/5-contributors

@Bakual
Copy link
Contributor

Bakual commented Dec 1, 2014

14 errors in unit tests and some minor codestyle issues.

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

Sorry, the majority of those errors was a typo, a missing +. Now everything should be fine.

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

BTW: The PHP 5.6 tests throw a notice before the test is started. Something is wrong there... (Unrelated to this PR)

@smanzi
Copy link

smanzi commented Dec 1, 2014

Hello Hannes, thanks for delivering this!

I've yet to test it and even to look at the code, but just reading your description, and especially your frustration for having to drag behind the weight of the B/C anchor, made me think something:

Wouldn't it be possible to introduce in com_config "experimental" switches for testing new (possibly B/C breaking) upcoming technologies? Those switches should be accompanied by humongous warnings stating that they are meant only for testing purposes, that most 3rd party extensions will not work when they are turned on and that there is no firm commitment that the functionality will be implemented in upcoming official releases.

This will allow 3rd party extensions developer to prepare for "the future" and core developer to receive an early feedback...

Just my 2 (probably O.T.) cents...

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

Hi @smanzi, that would indeed be a possibility, but I would strongly advise against it. There are 4 reasons against this:

  1. Joomla already has too many options and actually needs to cut down on those. Users will be confused for sure with further options. They are already unable to cope with the FTP settings and we have way to many people that have that set, even though they don't need it and they don't have an idea why they did it.
  2. At the same time, it means that we have to maintain 2 separate code blocks and have to find ways to elegantly wove this into our system. Should the new system have the same class names as the old one? Which one gets put into the default position and which solution overrides the other system? You will have such discussions for each and every change, in addition to our current discussions.
  3. Third party developers will have a nightmare to support all this. Considering the routing, a third party developer might say that he wants to use the new routing system, since it is far less code to write and its easier to understand. That means a user that wants to use that extension, needs to enable the new routing system. Now if for some reason, the old and new system are incompatible and the user wants to use an extension that only supports the old routing, he is screwed. That means that every third party developer has to cater for both situations, old and new, actually increasing the work instead of reducing it.
  4. Our project is notoriously incapable of supporting 2 branches of a code at the same time. A good example is/was JRegistry and JParameter. JParameter was one of the corner stones of Joomla 1.5 and was made obsolete in 1.6 by JForm and using JRegistry. Only after 1.6 was released did someone notice that JParameter has not been touched for 2 years and did not work anymore.

Based on these points, I would not introduce such options as a general measure. Especially the first point is important to me. Compare Joomla and how much options Joomla provides simply when writing an article compared to Wordpress. Yes, the two systems are very different, but having several magnitudes more options than a system that aims at the same market, does not throw the best light on J.

@smanzi
Copy link

smanzi commented Dec 1, 2014

@Hackwar To be honest I agree only with your points 2 (strongly agree) and 4 (possibly agree).

Solutions for the other points:

  • the switches could be totally hidden (no UI for setting them, just tinker with configuration.php or a separate "experimental.php" config file)
  • In no way 3rd party developers should deliver officially supported (by them, I mean) code that assumes any of such experimental switches is turned on.

On the other hand I concur that smoothly integrating two "lines of code execution" in the core can be quite a mess and will most likely imply altering the "current line of code".

I see this as something that should be reserved for exceptional cases, like probably a "new router" is, not something to be used for trivial alternatives.

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

The router is by 80% aimed at third party developers, since it provides them a way to write a router in 5 lines of code and they don't get a serious headache just because of that aspect of their component. There would be little gained if third party devs did not adopt this code as quickly as possible.

@smanzi
Copy link

smanzi commented Dec 1, 2014

So... how do you see the future? An "early" 4.0 (will allow breaking B/C) or the frustration of having at hand fantastic code ... that can't be used?

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

First of all, there are ways to get what we want right now without breaking B/C. And then I don't know what you mean with an "early" 4.0. Nothing is promised for 4.0. If the PLT decides to start work on 4.0 after 3.4 is released, so be it. 4.0 could mean that we have this router and some other minor details and that is it. Or it means something completely different.

@smanzi
Copy link

smanzi commented Dec 1, 2014

by "early 4.0" I was meaning exactly what you said: "this router and some other minor details". Well, maybe not just minor ones: I don't have examples at hand right now, but I would say everything that we are eager to do but can't because of the B/C commitment

@smanzi
Copy link

smanzi commented Dec 1, 2014

but... can you elaborate on the concept of "there are ways to get what we want right now without breaking B/C"?

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

These stages things for example are a way to rewrite the current router to a rule based concept which is backwards compatible in a lot of ways. There are also questions how far our backwards compatibility goes. If we for example said that non-SEF URLs are not covered by that backwards compatibility claim, then we could do different things while still be part of 3.x.

@smanzi
Copy link

smanzi commented Dec 1, 2014

@Hackwar I want to test this: should I apply #5105 too?

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

You can test this without #5105. 5105 however needs testing, too. 😉

@smanzi
Copy link

smanzi commented Dec 1, 2014

got it! 😜

@smanzi
Copy link

smanzi commented Dec 1, 2014

@test success

  • Applying this patch doesn't seems to have any negative impact with the tested component (com_content, several views)
  • SEF URLs exactly as before (which is a mixed blessing, IMHO....)
  • NON SEF URLs same as before

Question: should this be tested with SEF engines (like e.g. sh404) for compatibility or is this guaranteed to be B/C in this context?

Personal note: ehmm... are you aware many of those who crowdfunded your initiative are expecting you to get rid of the ArticleID from SEF URLs, aren't you? I Understand this is not acceptable in the core because of B/C, but do you plan to release a system plugin (or anything else) to achieve that?

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

Regarding sh404sef: Old code will be backwards compatible, but if you are using code written for whatever Joomla version this one will be released in, you might get conflicts. If sh404sef does not use the core router and instead deploys its own stuff, they would be disabling these stages effectively.

Regarding your personal note: This is a place where I plan to add an option to disable or enable certain features of the routing system, which would include the IDs. So it will be part of Joomla and it will be in in a backwards compatible way.

@smanzi
Copy link

smanzi commented Dec 1, 2014

Perfect! 😄

@mbabker
Copy link
Contributor

mbabker commented Dec 1, 2014

I found an older version of sh404SEF in my filesystem and quickly glanced through its code to find its router integration. From what I could tell, they're extending JRouterSite but none of the methods or vars changed in this URL are in their extended class, so this shouldn't break compatibility with their code from that aspect. @shumisha if you'd like to test this and confirm that things will continue to work well, it would be greatly appreciated.

In general with regards to backward compatibility, we can't control what happens if an extension is extending a library class and we change the internals of a method to extend what it is able to do. If the base API continues to behave the same pre- and post- change, then to me it is a B/C change. Unfortunately, once code is rewritten to make use of those newer features, that would cause an arguable B/C break if the extended class isn't updated as well. It's a risk we take any time we change the internals of a class or method to add new features.

@weeblr-dev
Copy link

Hi all,
Thanks Michael for the head up. I don't have time right now to get to the details of this, hopefully tomorrow. I can make the following very quick comments:

1 - strictly about sh404SEF, I do extend JRouterSite, but to be honest that's not really needed and I can probably put proxies in place if some methods I need/use were changed, which I doubt. sh404SEf is really API-compliant in that matter and only uses the additional rules it attaches to the site router.

2 - Foreseeable issues may more likely be if the routing system changes its behavior. LanguageFilter plugin is a good example. If Hannes suggestion to change it's behavior to NOT set/unset the language code were applied, that is broken between a pre-processing and a post-processing stage, then the data received by whatever routing rules are added will be different from what it is today. LanguageFilter routing has been causing much issues, and all SEF extensions (except sh404SEF) simply disable it for multilingual sites (and replicate its features elsewhere).
I have found some logic to keep it enabled and still be able to properly parse urls, but again I would expect that to break if the data provided to me is changed.

As Michael outlined, I think a B/C break would happen if changes were made to the language filter plugin routing rules to take advantage of this. Otherwise, we should be good and I'm sure I can accommodate that PR, maybe without having to do anything actually. If the LF plugin is modified in such a way however, then I think that should not happen before 4.0

I'll get back to this post in the coming days with some test results.

@Hackwar I think something that could be quite useful would be to keep and make accessible by all parsing or building rules the original input data (ie the original $uri) before it's been modified by any of the attached rules or the component itself. One reason for the difficulties with the language filter plugin is what you outlined, ie the language information is lost on the other parser/builder after the LF plugin has handled it.
So instead/on top relying on the LF to only remove the query string (when building) at the postpreocessing stage, it'd be very nice to simply store the original data in a place where any parsing rule can access it.

@Hackwar
Copy link
Member Author

Hackwar commented Dec 1, 2014

Hi @weeblr-dev, I've been working on the language filter plugin in PR #5140. Please look there, too.

Regarding storing the original data somewhere: I'm a bit hesitant regarding this. I personally decided against it, since I think it is not only a clean way to clean up after you, but also a way to override something. It allows me to prevent the default code from working on this stuff by removing it from the query. But if there is a majority to implement something like this, I would not fight for dear life. 😉

@smanzi
Copy link

smanzi commented Dec 3, 2014

Hannes, I've been testing with your PRs #5140, #5264 and #5276 in a multilingual environment and I have some results I would like to share with you hoping that you find them useful.
I'm posting the results here as this seems to be "the mother of all PRs": sorry if I'm wrong.

The test have been performed using:

2 languages:

  • English en-GB en (default)
  • Italiano it-IT it

3 menu structures

  • Main Menu 1 item: home (Featured Articles, default home for all)
  • Italiano 1 item: pagina-principale (Featured Articles, home for it)
  • English 1 item: home-page (Featured Articles, home for en)

3 categories

  • uncategorized (all) ID=2
  • test (en) ID=8
  • prova (it) ID=9

2 articles (both flagged as featured)

  • foobar (en, in test, ID=1)
  • casino (it, in prova, ID=2)

menu items, categories, and articles have been associated between themselves

"Remove URL Language Code" (for default language) is set to: No

Multilingual Status is:

multilingualstatus

From the above structure I had expected the following URLs to be valid and generated:

/en/
/en/8-test/1-fubar
/it/
/it/9-prova/2-casino

What I get is instead:

/en/
/en/home-page/8-test/1-fubar
/it/
/it/pagina-principale/9-prova/2-casino

i.e.: the aliases of the home pages are not stripped away

Language switcher does his duty, no problem.

The "expected URLs" do works if entered manually, but the bad news are that also "chimeric" URLs do work, like e.g.:

/it/home-page/8-test/2-casino
/en/pagina-principale/9-prova/1-fubar
/it/pagina-principale/8-test/2-casino
etc...

While others do give a 404, like:

/it/home-page/8-test/1-fubar
/en/pagina-principale/9-prova/2-casino

The discriminating factor seems to be that the language code must match the language of the item requested (disregarding categories and menu items in between)

P.S: Of course the above has been performed with the latest 'staging' at the time of this writing (head at b858931), integrated with the 3 above PRs.
I've been unable to integrate #5285 and did not tried integrating #5294 & #5295

@smanzi
Copy link

smanzi commented Dec 3, 2014

I have the awkward and embarrassing feeling I was badly wrong about the URLs I was expecting... 😳

The 'chimeric' URLs issue remains, as before (without this PRs), so I would say... 'business as usual', with 100% B/C

Sorry! Sergio

@Hackwar
Copy link
Member Author

Hackwar commented Dec 8, 2014

fixed

@Hackwar
Copy link
Member Author

Hackwar commented Dec 12, 2014

Ok, I implemented the changes that Johan proposed. Could we then get a review by a maintainer now and merge this stuff? 😉

@johanjanssens
Copy link
Contributor

@Hackwar Thanks looking good.

One final remark. I'm not sure about the additional class properties that have been added without the underscore. I understand that in a future version the properties should not be underscored but adding them without using them makes little sense ?

Proposal

What about using the correct properties and removing the underscored ones and then dynamically assigning them again by reference when the object is instantiated ?

Example : $this->_vars =& $this->vars;

This should work, I think.

The benefit of this is that you don't need to add unused properties to the class definition. Keeping things nice and tidy. Removing the legacy support also becomes easier later.

@Hackwar
Copy link
Member Author

Hackwar commented Dec 17, 2014

Hi @johanjanssens, those properties are not added by me. That is stuff from 1.6 times, I think. In any case, the by-reference-trick will only work as long as no one assigns a new array to that property. I agree that we should get rid of that old crap, but I fear that it wont be possible before 4.0. Feel free to duke this out with the PLT. 😉

@johanjanssens
Copy link
Contributor

@Hackwar You are right sorry, forgot to check the blame. This really smells bad. But if it was there, let's leave it there.

@Hackwar
Copy link
Member Author

Hackwar commented Dec 29, 2014

I updated the unittests to test properly for the new routing stages.

@Hackwar
Copy link
Member Author

Hackwar commented Jan 5, 2015

I had to change the order in which the methods are called in the router to solve a logical error that I made. So far the order was:

  • Preprocess build rules
  • Process build rules
  • Call preprocess() of the component router
  • Call build() of the component router
  • Postprocess build rules

This would not allow the language system and the Itemid-lookup to work correctly and since the Itemid lookup should be pulled into the router, all plugins depending on the Itemid to be present in the buildRules step would fail subsequently, so I had to reorder this to the following:

  • Preprocess build rules
  • Call preprocess() of the component router
  • Process build rules
  • Call build() of the component router
  • Postprocess build rules

It should be noted that the first order (preprocess() after buildRules) has been shipped with 3.3.6. So this would technically be a break in backwards compatibility if we change the order now. However, this requires that code in the preprocess() component router step has to rely on data from the buildRules step to even create an issue. This again means, that someone had to implement this so far undocumented and in the core unused feature. I would take the chance here and change this, especially since otherwise none of my routing changes can be merged before Joomla 4.0.

@Bakual
Copy link
Contributor

Bakual commented Jan 5, 2015

Imho we can take that B/C risk since I doubt anyone already used that new method. It would mean that they built an extension with a minimum requirement of J3.3.6 but they didn't really gain anything out of it.
Also there is a really slim chance if someone is using it, that the order actually matters.

@Hackwar
Copy link
Member Author

Hackwar commented Jan 5, 2015

Could we then merge this stuff?

@infograf768
Copy link
Member

@test
I did not get any issue on my multilingual test site.

@mbabker mbabker closed this Jan 9, 2015
mbabker pushed a commit that referenced this pull request Jan 9, 2015
@mbabker mbabker added this to the Joomla! 3.4.0 milestone Jan 9, 2015
// Process the parsed variables based on custom defined rules
$vars = $this->_processParseRules($uri);
// This is the main parse stage
$vars += $this->_processParseRules($uri);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hackwar For code readability sake it would change this to $vars += $this->_processParseRules($uri, self::PROCESS_DURING);

@johanjanssens
Copy link
Contributor

@Hackwar The changes order for the build rules makes total sense and it more logical. Should the same change be made for the parse rules too to keep the implementation consistent ?

@Hackwar
Copy link
Member Author

Hackwar commented Jan 12, 2015

There is no preprocess step for the parsing for component routers, so there is nothing that we could change there. 😉

@johanjanssens
Copy link
Contributor

True. All good then.

@Hackwar Hackwar deleted the router_stages branch January 6, 2016 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants