Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preventing 301 redirects on URLs with no trailing slashes (Netlify) #9207

Open
lloydh opened this issue Oct 18, 2018 · 27 comments

Comments

Projects
None yet
@lloydh
Copy link
Contributor

commented Oct 18, 2018

Summary

URLs with no trailing slash on sites hosted by Netlify lead to an immediate 301 redirect to the page with a trailing slash.

foo.com/bar --> foo.com/bar/

This has a performance cost and implications for SEO.

Is there a Netlify configuration that resolves these URLs without redirecting?

Relevant information

While this question is specific to Netlify, I did a quick review of other Gatsby sites featured in the Showcase and saw the same behaviour in many, but not all cases, for example:

Hopper /company - 301 redirect (Netlify)
Impossible Foods /mission - 301 redirect (unknown)
Cajun Bow Fishing /bows - 301 redirect (Netllify)
Braun /shavers-for-men - 200 no redirect (unknown)

Environment (if relevant)

Same behaviour in Gatsby v1 and v2.
I'm using gatsby-plugin-remove-trailing-slashes and gatsby-plugin-netlify.
Within the project all Links point to the non-trailing slash version.

@Yurickh

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2018

I'm not really familiar on how Netlify runs the static site, but I know that this is the default behaviour for express when serving static folders.
I'm not sure which are the performance costs you're implying here, can you link me to a reference? Would love to read more about that.

@lloydh

This comment has been minimized.

Copy link
Contributor Author

commented Oct 18, 2018

The performance cost is the synchronous delay for the first byte of useful data caused by the redirect.

Using the Hopper example above, visiting https://www.hopper.com/company takes 150-300ms for the redirect, before any page data is received. On cellular connections with high latency it can add up to 1s.

@Yurickh

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2018

I see.

From my experience with static sites, we always redirect to the /-ending url whenever it represents a folder (with an implicit index.html), not a file (like shavers-for-men.html). If you don't, the static resolution will simply fail.

This is also what the "pretty url" option on netlify does.

I guess the best approach here is to use the /-ending url as the canonical one, unless you have really solid reasons to do otherwise.

EDIT: Note that my position here is nowhere near an official position from the gatsby team. This is solely my personal opinion on the subject.

@lloydh

This comment has been minimized.

Copy link
Contributor Author

commented Oct 20, 2018

@Yurickh I agree specifying the /-ending urls as canonical is a pragmatic option but it does seem like advocating a schizophrenic url scheme for something that was easily solved with nginx / apache but not today's popular static site hosts.

If this really is the best approach then perhaps gatsby-plugin-remove-trailing-slashes should have this as an option (at least mentioned in the docs). Alternatively gatsby-plugin-canonical-urls could have an option for trailing slashes. I haven't found any other plugins that can set canonical meta tags.

I did come across #9025 but to be honest I'm surprised this hasn't been a bigger issue for a lot of folks given the popularity of /-less Gatsby sites.

@kakadiadarpan

This comment has been minimized.

Copy link
Contributor

commented Oct 23, 2018

@lloydh did you had a chance to look at this documentation of Redirects|Netlify?

@luukdv

This comment has been minimized.

Copy link

commented Oct 26, 2018

Same issue here. Turned on 'Pretty URLs' in Netlify, but when I visit a page and remove the trailing slash in the address bar afterwards, I land on the non-trailing variant.

Maybe there should be an option (or by default?) in gatsby-plugin-canonical-urls to enforce a trailing slash. Right now it's not really a canonical, since the current pathname is used (https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-plugin-canonical-urls/src/gatsby-browser.js#L9). @kakadiadarpan what do you think?

@Yurickh

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2018

Just to clarify, the 'Pretty URLs' option in netlify WILL redirect you to the trailing slash variant:

In addition to forwarding paths like /about to /about/ (a common practice in static sites and single page apps), it will also rewrite paths like /about.html to /about/.

@dimoFeeld

This comment has been minimized.

Copy link

commented Nov 2, 2018

We are haveing exactly the same problem. And we are now getting penelties from google. Is there a solution?

@lloydh

This comment has been minimized.

Copy link
Contributor Author

commented Nov 2, 2018

Just to clarify, the 'Pretty URLs' option in netlify WILL redirect you to the trailing slash variant

This is true but I haven't noticed any difference in behaviour with or without "Pretty URLs"; /foo redirects to /foo/ in either case. The desired behaviour is to rewrite slashless urls to resolve without redirecting.

Making /foo/ canonical is a workaround for the SEO penalties but AFAIK a full solution would only be possible in Netlify's routing layer… or by abandoning Netlify in favour of custom url rewriting …or by adopting slash/ urls. It's an unfortunate situation.

@gatsbot

This comment has been minimized.

Copy link

commented Jan 27, 2019

Old issues will be closed after 30 days of inactivity. This issue has been quiet for 20 days and is being marked as stale. Reply here or add the label "not stale" to keep this issue open!

@gatsbot gatsbot bot added the stale? label Jan 27, 2019

@abohannon

This comment has been minimized.

Copy link

commented Jan 29, 2019

I'm not having this issue with Netlify specifically, but my UTM query params are being deleted for this same reason. /mypage?utm_source=google becomes /mypage/ and this is causing tracking issues.

@gatsbot

This comment has been minimized.

Copy link

commented Feb 9, 2019

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

@gatsbot gatsbot bot closed this Feb 9, 2019

@dja

This comment has been minimized.

Copy link

commented Mar 4, 2019

This is definitely still an issue, and we’ve experienced SEO penalties as well. Additionally, turning off Netlify’s pretty URLs feature seems to result in errors stating “Missing resources for /“ or “Missing resources for /slash/“. We’ve tried solutions recommended here: #11524 but haven’t had any luck.

@0505gonzalez

This comment has been minimized.

Copy link

commented Mar 6, 2019

Experiencing this issue as well.

@0505gonzalez

This comment has been minimized.

Copy link

commented Mar 6, 2019

@dja Are you also on Netlify or are you using Github pages?

@dja

This comment has been minimized.

Copy link

commented Mar 6, 2019

@0505gonzalez

This comment has been minimized.

Copy link

commented Mar 6, 2019

@dja I'm on github pages, but the issue you're facing might be the same as mine. I've opened a new ticket and plan to create a PR shortly: #12364

@0505gonzalez

This comment has been minimized.

Copy link

commented Mar 6, 2019

TLDR: Github pages (and probably Netlify) add trailing forward slashes to folders. If you have something like /public/somepage/index.html and you visit https://yourpage.com/somepage, Github (and probably Netlify) will add a trailing slash because somepage is a directory.

@dja

This comment has been minimized.

Copy link

commented Mar 6, 2019

@0505gonzalez

This comment has been minimized.

Copy link

commented Mar 7, 2019

@dja That's what I'm currently trying to figure out in the other github issue I opened

@himynameistimli

This comment has been minimized.

Copy link
Contributor

commented Apr 3, 2019

@0505gonzalez were you able to figure out a solution for this issue?

Just landed here as we're having the same problems with the url parameters getting lost in the redirect from the version without the trailing slash to the version with the trailing slash.

Only difference is that we're on S3 + Cloudfront.

We might look into using Lambda@Edge to handle the redirect unless we can figure out a way to get it to work with gatsby.

Update:

For our case, we implemented the fix from https://www.ximedes.com/2018-04-23/deploying-gatsby-on-s3-and-cloudfront/ with the following lambda js function:

const querystring = require('querystring');
exports.handler = (event, context, callback) => {
    const request = event.Records[0].cf.request;

    /* Parse request query string to get javascript object */
    const params = querystring.parse(request.querystring.toLowerCase());
    const sortedParams = {};
    const uri = request.uri;

    /* Sort param keys */
    Object.keys(params).sort().forEach(key => {
        sortedParams[key] = params[key];
    });

	/* Simple way return the index.html */
    if (uri.endsWith('/')) {
        request.uri += 'index.html';
    } else if (!uri.includes('.')) {
        request.uri += '/index.html';
    }

    /* Update request querystring with normalized  */
    request.querystring = querystring.stringify(sortedParams);

    callback(null, request);
};

I'm still trying to figure out what's the best thing to do here, because I think as-is, there's a negative impact to our SEO just because now we're delivering the same page for the trailing and non-trailing slash version. Likely I will add a permanent redirect for the trailing slash version here too.

If you're not using AWS/Cloudfront, I think you'd be able to accomplish this with Cloudflare Workers.

@0505gonzalez

This comment has been minimized.

Copy link

commented Apr 9, 2019

@himynameistimli I did find a solution, proposed a code change in another thread. But seems like it might not be accepted so have not created a PR.

The gist of it:

  • Hosting on github pages. Github follows the directory structure when serving files. E.g. if you hit /somepage, it will redirect to /somepage/ because it's a directory (actual file is /somepage/index.html.
  • My proposed solution was that gatsby generate /somepage.html instead of /somepage/index.html
@garethgd

This comment has been minimized.

Copy link

commented Apr 15, 2019

Still experiencing this trailing slash redirect with or without Netlify's pretty URL option enabled.

@decimoseptimo

This comment has been minimized.

Copy link

commented May 1, 2019

@KyleAMathews

@himynameistimli I did find a solution, proposed a code change in another thread. But seems like it might not be accepted so have not created a PR.

The gist of it:

  • Hosting on github pages. Github follows the directory structure when serving files. E.g. if you hit /somepage, it will redirect to /somepage/ because it's a directory (actual file is /somepage/index.html.
  • My proposed solution was that gatsby generate /somepage.html instead of /somepage/index.html

@KyleAMathews this is seo danger
This subject isn't explained at all in https://www.gatsbyjs.org/docs/gatsby-link/
If someones decides to use no-trailing-slash-urls, a couple of days later the google serp becomes full of 301 redirections for your site.

@gatsbot gatsbot bot added the stale? label May 22, 2019

@gatsbot

This comment has been minimized.

Copy link

commented May 22, 2019

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

@anhnk

This comment has been minimized.

Copy link

commented May 23, 2019

We are having the same problems as described in this PR. I'm just wondering if it is being actively worked on at the moment.

@wardpeet

This comment has been minimized.

Copy link
Member

commented May 28, 2019

Not at this moment but we're open for Pull requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.