New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can no longer use colons in permalinks #5284

Closed
peterjc opened this Issue Aug 24, 2016 · 18 comments

Comments

Projects
None yet
8 participants
@peterjc

peterjc commented Aug 24, 2016

  • I believe this to be a bug, not a question about using Jekyll.
  • I updated to the latest Jekyll (or) if on GitHub Pages to the latest github-pages
  • I read the CONTRIBUTION file at https://jekyllrb.com/docs/contributing/
  • This is a feature request.

  • I am on (or have tested on) _macOS_ 10+
  • I am on (or have tested on) _Debian/Ubuntu_ GNU/Linux (via GitHub Pages & TravisCI)
  • I am on (or have tested on) _Fedora_ GNU/Linux (tested on CentOS 6)
  • I am on (or have tested on) _Arch_ GNU/Linux
  • I am on (or have tested on) _Other_ GNU/Linux
  • I am on (or have tested on) _Windows_ 10+

  • I was trying to install.
  • There is a broken Plugin API.
  • I had an error on GitHub Pages, and I have not tested locally.
  • I had an error on GitHub Pages, and GitHub Support said it was a Jekyll Bug.
  • I had an error on GitHub Pages and TESTED it locally.
  • I was trying to build.
  • It was another bug.

My Reproduction Steps

TravisCI log https://travis-ci.org/biopython/biopython.github.io/builds/154676985 from same commit biopython/biopython.github.io@665c0c9 where GitHub Pages broke (unrelated edit in this commit, change is the dependencies):

$ bundle exec jekyll build
Configuration file: /home/travis/build/biopython/biopython.github.io/_config.yml
            Source: /home/travis/build/biopython/biopython.github.io
       Destination: /home/travis/build/biopython/biopython.github.io/_site
 Incremental build: disabled. Enable with --incremental
      Generating... 
  Liquid Exception: The URL /wiki/Category:Cookbook is invalid because it contains a colon. in /_layouts/default.html
             ERROR: YOUR SITE COULD NOT BE BUILT:
                    ------------------------------------
                    The URL /wiki/Category:Cookbook is invalid because it contains a colon.
The command "bundle exec jekyll build" exited with 1.

We use colons in permalinks to preserve legacy MediaWiki URLs, e.g.:

$ head -n 8 wiki/Category_Cookbook.md 

---
title: Cookbook Entries
permalink: wiki/Category:Cookbook
layout: tagpage
tag: Cookbook

---

The entries contained in the Cookbook category are designed to

We link to them using e.g. from wiki/Documentation.md using:

-   [Wiki documentation](Category%3AWiki_Documentation "wikilink")

Testing locally, I have simplified _layout/default.html to remove the header and sidebar,

<!DOCTYPE html>
<html lang="en-us">

  <body>

    <div class="content container">
      <hr class="head"/>
      {{ content }}
    </div>

  </body>
</html>

Now:

$ jekyll build
WARN: Unresolved specs during Gem::Specification.reset:
      rouge (~> 1.7)
      jekyll-watch (~> 1.1)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Configuration file: /Volumes/Hitachi-3TB-mirrored/repositories/biopython.github.io/_config.yml
            Source: /Volumes/Hitachi-3TB-mirrored/repositories/biopython.github.io
       Destination: /Volumes/Hitachi-3TB-mirrored/repositories/biopython.github.io/_site
 Incremental build: disabled. Enable with --incremental
      Generating... 
             ERROR: YOUR SITE COULD NOT BE BUILT:
                    ------------------------------------
                    The URL /wiki/Category:Cookbook is invalid because it contains a colon.

The Output I Wanted

Want to continue to use colons in permalinks, and be able to link to those pages.

It appears check if relative URL contains a colon (#5069) breaks our use case.

peterjc added a commit to biopython/biopython.github.io that referenced this issue Aug 24, 2016

Percent encode colon in redirect_from
Suggested in jekyll/jekyll-redirect-from#119
as a workaround from the regression in Jekyll 3.2, see also:
jekyll/jekyll#5284

This alone does not fix jekyll build in local testing.

peterjc added a commit to biopython/biopython.github.io that referenced this issue Aug 24, 2016

Percent encode colon in permalink
Workaround for regression in Jekyll 3.2 (GitHub issue #96),
see also: jekyll/jekyll#5284
@peterjc

This comment has been minimized.

Show comment
Hide comment
@peterjc

peterjc Aug 25, 2016

It appears we can work-around this by percent encoding any colons (: as %3A) in our redirect_from and permalink header settings, and internal URLs.

peterjc commented Aug 25, 2016

It appears we can work-around this by percent encoding any colons (: as %3A) in our redirect_from and permalink header settings, and internal URLs.

@Karlheinzniebuhr

This comment has been minimized.

Show comment
Hide comment
@Karlheinzniebuhr

Karlheinzniebuhr Aug 29, 2016

+1 here. I'm getting a build error on github pages

Karlheinzniebuhr commented Aug 29, 2016

+1 here. I'm getting a build error on github pages

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Sep 28, 2016

Member

Our concern is writing files to the filesystem with colons. Is there a way to encode them for the FS that's safer? Or am I crazy?

Member

parkr commented Sep 28, 2016

Our concern is writing files to the filesystem with colons. Is there a way to encode them for the FS that's safer? Or am I crazy?

@peterjc

This comment has been minimized.

Show comment
Hide comment
@peterjc

peterjc Sep 28, 2016

In order that our website git repository could be edited on Windows, we also wanted to avoid colons in the filenames. Therefore we had e.g. wiki/Category_Wiki_Documentation.md (using an underscore) with a permalink (using a colon) in the header, originally:

permalink: wiki/Category:Cookbook

That was working perfectly until #5069 and Jekyll 3.2.0, but we were eventually able to sidestep this biopython/biopython.github.io@3af6e87 using:

permalink: wiki/Category%3ACookbook

and likewise in redirect_from biopython/biopython.github.io@d86f08e

peterjc commented Sep 28, 2016

In order that our website git repository could be edited on Windows, we also wanted to avoid colons in the filenames. Therefore we had e.g. wiki/Category_Wiki_Documentation.md (using an underscore) with a permalink (using a colon) in the header, originally:

permalink: wiki/Category:Cookbook

That was working perfectly until #5069 and Jekyll 3.2.0, but we were eventually able to sidestep this biopython/biopython.github.io@3af6e87 using:

permalink: wiki/Category%3ACookbook

and likewise in redirect_from biopython/biopython.github.io@d86f08e

@dret

This comment has been minimized.

Show comment
Hide comment
@dret

dret Oct 10, 2016

i am having the same problem (but not even in permalinks, simply in title/description fields):

---
layout:      page
title:       "OAuth URI: urn:ietf:params:oauth:client-assertion-type:saml2-bearer"
description: "To use a SAML Bearer Assertion for client authentication, the client uses the following parameter values and encodings. The value of the \"client_assertion_type\" parameter is \"urn:ietf:params:oauth:client-assertion-type:saml2-bearer\"."
---

triggers the error message mentioned above. it seems strange to disallow characters that are perfectly legal. is this going to be fixed, or is it better to escape : as %3A? if escaping is the advised method, are there any side-effects to it? thanks!

dret commented Oct 10, 2016

i am having the same problem (but not even in permalinks, simply in title/description fields):

---
layout:      page
title:       "OAuth URI: urn:ietf:params:oauth:client-assertion-type:saml2-bearer"
description: "To use a SAML Bearer Assertion for client authentication, the client uses the following parameter values and encodings. The value of the \"client_assertion_type\" parameter is \"urn:ietf:params:oauth:client-assertion-type:saml2-bearer\"."
---

triggers the error message mentioned above. it seems strange to disallow characters that are perfectly legal. is this going to be fixed, or is it better to escape : as %3A? if escaping is the advised method, are there any side-effects to it? thanks!

@peterjc

This comment has been minimized.

Show comment
Hide comment
@peterjc

peterjc Oct 10, 2016

One unwelcome but relatively minor side effect is the URL bar of the web browser will show the colons as %3A which is ugly, likewise if copy/pasting the URL into other documentation the ugly percent escape persists.

peterjc commented Oct 10, 2016

One unwelcome but relatively minor side effect is the URL bar of the web browser will show the colons as %3A which is ugly, likewise if copy/pasting the URL into other documentation the ugly percent escape persists.

@pathawks pathawks referenced this issue Oct 12, 2016

Closed

Jekyll breaks when URIs contain a colon #5480

5 of 17 tasks complete
@sbliven

This comment has been minimized.

Show comment
Hide comment
@sbliven

sbliven Nov 20, 2016

So this is on github too now? I am getting this problem locally but our current github page (not edited in several months) is still allowing colons. It's frustrating to have to hold off making changes until either this issue gets fixed or I implement the %3A workaround.

sbliven commented Nov 20, 2016

So this is on github too now? I am getting this problem locally but our current github page (not edited in several months) is still allowing colons. It's frustrating to have to hold off making changes until either this issue gets fixed or I implement the %3A workaround.

@peterjc

This comment has been minimized.

Show comment
Hide comment
@peterjc

peterjc Nov 21, 2016

@sbliven Yes - as soon as GitHub updated their copy of Jekyll used in GitHub Pages, our builds broke. As noted above we have worked around this by using lots of %3A escaping, but this was an unwelcome hassle and imposes a visible change to the website end user.

peterjc commented Nov 21, 2016

@sbliven Yes - as soon as GitHub updated their copy of Jekyll used in GitHub Pages, our builds broke. As noted above we have worked around this by using lots of %3A escaping, but this was an unwelcome hassle and imposes a visible change to the website end user.

sbliven added a commit to biojava/biojava.github.io that referenced this issue Nov 21, 2016

Removing illegal characters from wiki page titles
The most frequent change was replacing ':' with '_'. A permlink was
added to these pages to prevent the URL from changes, as suggested in
jekyll/jekyll#5284

@DirtyF DirtyF added the bug label Nov 21, 2016

@jekyllbot

This comment has been minimized.

Show comment
Hide comment
@jekyllbot

jekyllbot Jan 21, 2017

Contributor

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the 3.3-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

Contributor

jekyllbot commented Jan 21, 2017

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the 3.3-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

@jekyllbot jekyllbot added the stale label Jan 21, 2017

@sbliven

This comment has been minimized.

Show comment
Hide comment
@sbliven

sbliven Jan 31, 2017

I confirm that including a colon in a permalink still causes a build failure in jekyll 3.4.0.

sbliven commented Jan 31, 2017

I confirm that including a colon in a permalink still causes a build failure in jekyll 3.4.0.

@jekyllbot jekyllbot removed the stale label Jan 31, 2017

@philnash

This comment has been minimized.

Show comment
Hide comment
@philnash

philnash Feb 16, 2017

Contributor

To keep this from going stale, I just upgraded from Jekyll 3.2.1 to 3.4.0 and one of my posts with a colon in the frontmatter title broke the build.

Replacing the colon with &#58; has fixed it for now.

Contributor

philnash commented Feb 16, 2017

To keep this from going stale, I just upgraded from Jekyll 3.2.1 to 3.4.0 and one of my posts with a colon in the frontmatter title broke the build.

Replacing the colon with &#58; has fixed it for now.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Feb 16, 2017

Member

@philnash Happy to see this check be removed altogether. If you re-run jekyll with --trace, you'll see the stacktrace and you can just remove the method and the method call (and any associated tests). Should be an easy contribution.

Member

parkr commented Feb 16, 2017

@philnash Happy to see this check be removed altogether. If you re-run jekyll with --trace, you'll see the stacktrace and you can just remove the method and the method call (and any associated tests). Should be an easy contribution.

@philnash

This comment has been minimized.

Show comment
Hide comment
@philnash

philnash Feb 17, 2017

Contributor

Oops, my mistake. I don't believe this is a Jekyll issue (not with titles anyway, I'll leave this as an issue for permalinks). Turns out I'm manually passing titles into the uri_escape filter and that is causing my error. Looks like a regression in addressable.

Edit: Or indeed as I'm not trying to escape a URI I should have been using cgi_escape. Well, I've learned something this evening. I'll be on my way now.

Contributor

philnash commented Feb 17, 2017

Oops, my mistake. I don't believe this is a Jekyll issue (not with titles anyway, I'll leave this as an issue for permalinks). Turns out I'm manually passing titles into the uri_escape filter and that is causing my error. Looks like a regression in addressable.

Edit: Or indeed as I'm not trying to escape a URI I should have been using cgi_escape. Well, I've learned something this evening. I'll be on my way now.

@sbliven

This comment has been minimized.

Show comment
Hide comment
@sbliven

sbliven Feb 17, 2017

I'm pretty sure this is a Jekyll issue, since a permalink with a colon breaks the build.

sbliven commented Feb 17, 2017

I'm pretty sure this is a Jekyll issue, since a permalink with a colon breaks the build.

@sbliven

This comment has been minimized.

Show comment
Hide comment
@sbliven

sbliven Feb 17, 2017

Building with --trace isn't giving me any new information:

$ jekyll --version
jekyll 3.4.0
$ head -4 _wiki/BioJava_BioJavaInside.md
---
title: BioJava:BioJavaInside
permalink: wiki/BioJava:BioJavaInside
---
$ jekyll build --trace
Configuration file: /Users/blivens/dev/biojava.github.io/_config.yml
            Source: /Users/blivens/dev/biojava.github.io
       Destination: /Users/blivens/dev/biojava.github.io/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
  Liquid Exception: The URL /wiki/BioJava:BioJavaInside is invalid because it contains a colon. in sitemap.xml
             ERROR: YOUR SITE COULD NOT BE BUILT:
                    ------------------------------------
                    The URL /wiki/BioJava:BioJavaInside is invalid because it contains a colon.

Changing the permalink colon to %3A causes the build to pass.

sbliven commented Feb 17, 2017

Building with --trace isn't giving me any new information:

$ jekyll --version
jekyll 3.4.0
$ head -4 _wiki/BioJava_BioJavaInside.md
---
title: BioJava:BioJavaInside
permalink: wiki/BioJava:BioJavaInside
---
$ jekyll build --trace
Configuration file: /Users/blivens/dev/biojava.github.io/_config.yml
            Source: /Users/blivens/dev/biojava.github.io
       Destination: /Users/blivens/dev/biojava.github.io/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
  Liquid Exception: The URL /wiki/BioJava:BioJavaInside is invalid because it contains a colon. in sitemap.xml
             ERROR: YOUR SITE COULD NOT BE BUILT:
                    ------------------------------------
                    The URL /wiki/BioJava:BioJavaInside is invalid because it contains a colon.

Changing the permalink colon to %3A causes the build to pass.

@philnash

This comment has been minimized.

Show comment
Hide comment
@philnash

philnash Feb 17, 2017

Contributor

Yeah, sorry, I was jumping on this because it was breaking for me with titles, but that turned out to be my fault. I believe that this is still an issue for permalinks.

Contributor

philnash commented Feb 17, 2017

Yeah, sorry, I was jumping on this because it was breaking for me with titles, but that turned out to be my fault. I believe that this is still an issue for permalinks.

pkra added a commit to pkra/peterkrautzberger.org that referenced this issue Mar 17, 2017

@sbliven

This comment has been minimized.

Show comment
Hide comment
@sbliven

sbliven Apr 5, 2017

Just to recap, colons will continue to need to be escaped. This is because colons are used for permalink keys (e.g. :num for pagination).

sbliven commented Apr 5, 2017

Just to recap, colons will continue to need to be escaped. This is because colons are used for permalink keys (e.g. :num for pagination).

@peterjc

This comment has been minimized.

Show comment
Hide comment
@peterjc

peterjc Apr 5, 2017

@sbliven Are you sure? I've not tested it yet, but the discussion on #5889 sounds like it would allow a plain colon in a URL once again - and that's certainly the impression from the entry in History.markdown which reads:

Minor Enhancements

  • ...
  • Don't raise an error if URL contains a colon (#5889)

peterjc commented Apr 5, 2017

@sbliven Are you sure? I've not tested it yet, but the discussion on #5889 sounds like it would allow a plain colon in a URL once again - and that's certainly the impression from the entry in History.markdown which reads:

Minor Enhancements

  • ...
  • Don't raise an error if URL contains a colon (#5889)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment