Case insensitive redirects #9755

jessehouwing · 2018-07-26T20:33:27Z

While migrating from Blogger to Ghost I ran into the issue that Blogger is case-insensitive in a couple of places:

Tag urls
Post urls

I've integrated Google Search Console with my current blog and I'm seeing all kinds of people hitting 404's due to them using urls in all-lower-case.

I've updated my Blogger2Ghost.NET application to generate redirects like these

  {
    "from": "^\\/search\\/label\\/[Hh][Dd][Mm][Ii]$",
    "to": "/tag/windows-devices",
    "permanent": false
  }

But that's not really nice. This Pull request adds the ability to add case-insensitivity to the redirects.json format:

    { 
        "from": "^\\/case-insensitive",
        "caseInsensitive": true,
        "to": "/redirected-insensitive"
    },
    {
        "from": "^\\/Case-Sensitive",
        "caseInsensitive": false,
        "to": "/redirected-sensitive"
    },
    {
        "from": "^\\/Default-Sensitive",
        "to": "/redirected-default"
    }

The caseInsensitive is a boolean option, it's optional and the default is to keep the current behavior. By adding caseInsensitive=true the redirect will be initialized with the "i" RegExp option.

I've considered checking the start of the regex for (?i:) as well and settign the flag, which is the way for most regex flavors to set case insensitivity inline. But since that's not part of the JavaScript regexp standard, that may be confusing.

This change is

jessehouwing · 2018-07-26T20:36:01Z

Currently the redirect API doesn't return a 302, which is what I expect, but it does return 301 to the passed in URL (preserving case). I'm not sure why, this seems the standard behavior.

kirrg001 · 2018-07-26T20:40:16Z

Thanks will review asap 👍

jessehouwing · 2018-07-26T20:55:09Z

On windows there are some issues running tests and the eslint configuration.

There are more _spec files than the windows commandline will grok, causing a "command to long" error half-way running the tests. No clue how to easily fix this.
The repo doesn't enforce checkout as-is commit-as-is. So on Windows it defaults to windows line ends on a default git install. This causes so many lint errors it becomes impossible to use. Setting the repository default will help people on Windows. See: https://help.github.com/articles/dealing-with-line-endings/

kirrg001 · 2018-07-30T10:58:14Z

Hey!

I am not sure about adding a new option 🤔

If we would allow passing the options in the from field, we could parse it out and forward the regex options to RegExp?

e.g. "from": "^\/EXAMPLE/i",

jessehouwing · 2018-07-30T11:27:50Z

Yeah that was the alternative thought I had. Either use PCRE option inlining `from: "(?:i)^\\/ACTION\\/$"` Or the JS synrax: `from: "/^\\/ACTION\\/$/i"` But since there was already quite a bit of code manipulating the regex (trailing slash, ^, $ etc), it was a more invasive change. Plus with all the escaping it just looks really bad. Another issue I had with it was whether you'd want other modifiers to work as well. The inline syntax would suggest that could work. Separate property would not. Happy to take it in that direction though.

jessehouwing · 2018-07-30T12:53:43Z

The latest version of my Blogger Importer now generates case insensitive links and can also generate lower-case links if needed.
jessehouwing/Blogger2Ghost@26fa340

kirrg001 · 2018-07-30T15:20:35Z

Or the JS synrax: from: "/^\\/ACTION\\/$/i"

👍

jessehouwing · 2018-07-30T15:26:11Z

I'll have a look if I can smurf this in that direction.

…

On Mon, 30 Jul 2018, 17:21 Katharina Irrgang, ***@***.***> wrote: Or the JS synrax: from: "/^\\/ACTION\\/$/i" 👍 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#9755 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AD-uS0L1nqC69UTEE99U4550_4jFr6BTks5uLyRkgaJpZM4VigMa> .

jessehouwing · 2018-08-06T16:43:00Z

Pushed the requested changes. It now uses the requested format. Dropped the parameter and updated the tests to reflect.

kirrg001 · 2018-08-06T18:56:08Z

Thanks @jessehouwing - will review asap.

.gitattributes

@@ -0,0 +1,2 @@
+# enforce unix style line endings


core/server/web/middleware/custom-redirects.js

@@ -27,6 +27,13 @@ _private.registerRoutes = function registerRoutes() {
             *   - you define /my-blog-post-1/ as from property
             *   - /my-blog-post-1 or /my-blog-post-1/ should work
             */
+
+            var options = '';


jessehouwing · 2018-09-13T18:49:39Z

Rebased against master and incorporated comments.

ErisDS

LGTM 👍

core/server/web/middleware/custom-redirects.js

@@ -27,6 +27,13 @@ _private.registerRoutes = function registerRoutes() {
             *   - you define /my-blog-post-1/ as from property
             *   - /my-blog-post-1 or /my-blog-post-1/ should work
             */
+
+            let options = '';
+            if (redirect.from.match(/\/.*\/i/)) {


jessehouwing · 2018-09-18T05:33:45Z

Not sure what you mean with the last comment... Are you suggesting to drop the first forward slash?

I didn't do that cause one may have a pattern that ends with /i and there would be no way to distinguish. That's why I did only the variant where both leading and trailing slash are present eg: /..../i

kirrg001 · 2018-09-24T11:10:17Z

Not sure what you mean with the last comment... Are you suggesting to drop the first forward slash?

No. I only was suggesting to match the url with "ends with" /i.

This PR needs a rebase again 😕

jessehouwing · 2018-09-24T12:08:41Z

Right, just scanning for /i won't be enough, if we're following the /... /i format in javascript. The leading slash is part of the expression syntax. I could split up the check in a Startswith('/') and EndsWith('/i'), but regex seems the to be tool of trade in this class :D.

I did find a bug in the regex I used (it didn't use ^...$ to restrict this regex to the whole input.

I can rewrite it as: from.match(/^\//) && from.match(/\/i$/) if that's preferred.

jessehouwing · 2018-09-24T12:08:48Z

Rebased.

kirrg001

LGTM - thanks so much :)

* master: (25 commits) Version bump to 2.1.4 Updated Ghost-Admin to 2.1.4 Version bump to 2.1.4-beta.1 Updated Ghost-Admin to 2.1.4-beta.1 Updated private-sites to not redirect to full urls Bumped ghost-ignition to version 2.9.6 Upgrading Casper to 2.6.3 Refactored url utility to generate multiple API version URLs (#9897) Added new endpoint to upload square profile images with dimension validation (#9862) Removed change frequency and priority fields from sitemap generator (#9771) 🎨Added case insensitive support for redirects (#9755) Refactored direct usages of api controllers Normalised how we require models Refactored how we require shared middlewares from web/ (#9893) Removed duplicate 'id' for User permittedOptions Updated permittedOptions to correctly call super Updated base model to remove extraAllowedProperties Extended uncapitalise unit tests (#9891) Removed `res.isAdmin` from admin express app (#9889) Refactored `web/middleware` and `web/utils` to `web/shared` (#9892) ...

kirrg001 self-assigned this Jul 26, 2018

kirrg001 changed the title ~~Case insensitive redirects~~ [WIP] Case insensitive redirects Aug 3, 2018

kirrg001 reviewed Sep 10, 2018

View reviewed changes

.gitattributes Outdated

@@ -0,0 +1,2 @@

# enforce unix style line endings

This comment was marked as abuse.

Sign in to view

kirrg001 reviewed Sep 10, 2018

View reviewed changes

kirrg001 changed the title ~~[WIP] Case insensitive redirects~~ Case insensitive redirects Sep 17, 2018

ErisDS approved these changes Sep 17, 2018

View reviewed changes

kirrg001 reviewed Sep 17, 2018

View reviewed changes

jessehouwing added 6 commits September 24, 2018 13:51

Adds tests for case sensitive redirects

175168a

Fixing lint issues

f8d4a62

Correct the tests

b4d87d0

Switching to /expression/i

ad253e4

Move comment to correct if block

7a94c63

Only detect case sensitive flag when fully enclosed

8e1db43

kirrg001 approved these changes Sep 24, 2018

View reviewed changes

kirrg001 merged commit e3234bc into TryGhost:master Sep 24, 2018

woolfyx mentioned this pull request Oct 3, 2018

Upgrade to v2.2.0 woolfyx/azure-ghost#12

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Case insensitive redirects #9755

Case insensitive redirects #9755

jessehouwing commented Jul 26, 2018 •

edited

jessehouwing commented Jul 26, 2018

kirrg001 commented Jul 26, 2018

jessehouwing commented Jul 26, 2018

kirrg001 commented Jul 30, 2018

jessehouwing commented Jul 30, 2018 via email •

edited

jessehouwing commented Jul 30, 2018 •

edited

kirrg001 commented Jul 30, 2018

jessehouwing commented Jul 30, 2018 via email

jessehouwing commented Aug 6, 2018

kirrg001 commented Aug 6, 2018

This comment was marked as abuse.

This comment was marked as abuse.

jessehouwing commented Sep 13, 2018

ErisDS left a comment

This comment was marked as abuse.

jessehouwing commented Sep 18, 2018

kirrg001 commented Sep 24, 2018

jessehouwing commented Sep 24, 2018

jessehouwing commented Sep 24, 2018

kirrg001 left a comment

Case insensitive redirects #9755

Case insensitive redirects #9755

Conversation

jessehouwing commented Jul 26, 2018 • edited

jessehouwing commented Jul 26, 2018

kirrg001 commented Jul 26, 2018

jessehouwing commented Jul 26, 2018

kirrg001 commented Jul 30, 2018

jessehouwing commented Jul 30, 2018 via email • edited

jessehouwing commented Jul 30, 2018 • edited

kirrg001 commented Jul 30, 2018

jessehouwing commented Jul 30, 2018 via email

jessehouwing commented Aug 6, 2018

kirrg001 commented Aug 6, 2018

This comment was marked as abuse.

This comment was marked as abuse.

jessehouwing commented Sep 13, 2018

ErisDS left a comment

Choose a reason for hiding this comment

This comment was marked as abuse.

jessehouwing commented Sep 18, 2018

kirrg001 commented Sep 24, 2018

jessehouwing commented Sep 24, 2018

jessehouwing commented Sep 24, 2018

kirrg001 left a comment

Choose a reason for hiding this comment

jessehouwing commented Jul 26, 2018 •

edited

jessehouwing commented Jul 30, 2018 via email •

edited

jessehouwing commented Jul 30, 2018 •

edited