New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jekyll fails to detect front matter on files with windows line endings #3304

Closed
benbalter opened this Issue Jan 13, 2015 · 6 comments

Comments

Projects
None yet
4 participants
@benbalter
Contributor

benbalter commented Jan 13, 2015

Steps to reproduce (hint: line-ending-converter for Atom helps):

  1. Create a new HTML file, adding YAML front matter
  2. Save the file with all windows line endings

Expected:

Front matter is detected, file is read in

Actual:

File is treated as static HTML, front matter is converted raw.

Originally, I suspected this was my fault, introduced in #2211, but it appears it's \r safe. Perhaps it needs (\r|\n|\r\n)?

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jan 13, 2015

Member

Do you have a test file? We'd need to know what the status of the special characters is.

Member

parkr commented Jan 13, 2015

Do you have a test file? We'd need to know what the status of the special characters is.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jan 13, 2015

Member

For reference, here's the presently-used Utils.has_yaml_header? function:

    def has_yaml_header?(file)
      !!(File.open(file, 'rb') { |f| f.read(5) } =~ /\A---\r?\n/)
    end

It's \r-safe. It should always have \n. LF = \n, CR = \r\n IIRC.

Member

parkr commented Jan 13, 2015

For reference, here's the presently-used Utils.has_yaml_header? function:

    def has_yaml_header?(file)
      !!(File.open(file, 'rb') { |f| f.read(5) } =~ /\A---\r?\n/)
    end

It's \r-safe. It should always have \n. LF = \n, CR = \r\n IIRC.

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jan 14, 2015

Contributor

CR = \r and CR+LF (Windows) is \r\n but that said, messengers use CR (Pidgin, Empathy, Adium - which all use libpurple) and unless your text editor adjusts the line endings it will remain CR, and for a text editor to adjust that without user intervention or user explicit automatic intervention is bad IMO but also this is a sign of a much bigger problem that shouldn't need exist.

Jekyll should normalize in and customize out:

str1 = "hello\r"
str2 = "hello\r\n"
str1.encode(str1.encoding, :universal_newline => true) # => "hello\n"
str2.encode(str2.encoding, :universal_newline => true) # => "hello\n"

Then on the way out do the same thing from an opt that Jekyll accepts, or if you detect Windows then default to :crlf_newline => true the craziness of should not need to be hardcoded into the logic 😉 because it should be normalized and assumed internally then re-normalized and customized on the way out.

Contributor

envygeeks commented Jan 14, 2015

CR = \r and CR+LF (Windows) is \r\n but that said, messengers use CR (Pidgin, Empathy, Adium - which all use libpurple) and unless your text editor adjusts the line endings it will remain CR, and for a text editor to adjust that without user intervention or user explicit automatic intervention is bad IMO but also this is a sign of a much bigger problem that shouldn't need exist.

Jekyll should normalize in and customize out:

str1 = "hello\r"
str2 = "hello\r\n"
str1.encode(str1.encoding, :universal_newline => true) # => "hello\n"
str2.encode(str2.encoding, :universal_newline => true) # => "hello\n"

Then on the way out do the same thing from an opt that Jekyll accepts, or if you detect Windows then default to :crlf_newline => true the craziness of should not need to be hardcoded into the logic 😉 because it should be normalized and assumed internally then re-normalized and customized on the way out.

@benbalter

This comment has been minimized.

Show comment
Hide comment
@benbalter

benbalter Jan 14, 2015

Contributor

I'm with @envygeeks that we should normalize, if possible (for our own sanity, and for our users' sanity).

https://github.com/dweinberger/newclues/blob/master/newclues.html is an example file (with \rs), if you add front matter. FWIW, GitHub doesn't properly detect the line breaks for numbering purposes.

Contributor

benbalter commented Jan 14, 2015

I'm with @envygeeks that we should normalize, if possible (for our own sanity, and for our users' sanity).

https://github.com/dweinberger/newclues/blob/master/newclues.html is an example file (with \rs), if you add front matter. FWIW, GitHub doesn't properly detect the line breaks for numbering purposes.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jan 14, 2015

Member

Jekyll should normalize in and customize out

I'm good with this. I'll work on this next.

Member

parkr commented Jan 14, 2015

Jekyll should normalize in and customize out

I'm good with this. I'll work on this next.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Dec 13, 2015

Member

I believe this has been fixed in Jekyll 3. THanks!

Member

parkr commented Dec 13, 2015

I believe this has been fixed in Jekyll 3. THanks!

@parkr parkr closed this Dec 13, 2015

@jekyll jekyll locked and limited conversation to collaborators Feb 27, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.