Gitinfo fails on unicode filename #3071

Closed
rac2030 opened this Issue Feb 21, 2017 · 4 comments

Comments

Projects
None yet
2 participants
@rac2030

rac2030 commented Feb 21, 2017

Just for fun I added a movie reference, while it works for hugo server run, it fails when I build the output be it locally or on wercker.
I have a File named π.md under content (see http://rac.su and look closer at the footer bar on the right)

Adding it to git works but I noticed git status did list it as "content/\317\200.md"
Hugo fails when building it at
ERROR: 2017/02/21 04:38:02 gitinfo.go:63: Failed to find GitInfo for "content/π.md"

Another example of weird file names is 💩.md which git translates to "content/\360\237\222\251.md" which you can find at http://rac.su/💩 if EnableGitInfo is set to false.

While my case is more fun than anything else, this may also have an effect to other unicode characters like chinese filenames, cyrillic filenames, ... that use unicode.

Searching for unicode in filenames issues, seems like many projects fail on that but there should be some way to enhance hugolib/gitinfo.go to actually handle them correctly.

@bep bep added the Bug label Feb 21, 2017

@bep

This comment has been minimized.

Show comment
Hide comment
@bep

bep Feb 21, 2017

Member

Are you running macOS?

Member

bep commented Feb 21, 2017

Are you running macOS?

@rac2030

This comment has been minimized.

Show comment
Hide comment
@rac2030

rac2030 Feb 21, 2017

Yes I'm on OSX 10.10.5 and on wercker it runs on a Debian box.
Hugo itself can handle those files and render them to HTML pages just fine.
I think it's just the way how gitinfo.go normalizes filenames and hands it over to git binary

// Git normalizes file paths on this form:
filename := path.Join(filepath.ToSlash(contentRoot), contentDir, filepath.ToSlash(p.Path()))

rac2030 commented Feb 21, 2017

Yes I'm on OSX 10.10.5 and on wercker it runs on a Debian box.
Hugo itself can handle those files and render them to HTML pages just fine.
I think it's just the way how gitinfo.go normalizes filenames and hands it over to git binary

// Git normalizes file paths on this form:
filename := path.Join(filepath.ToSlash(contentRoot), contentDir, filepath.ToSlash(p.Path()))

@bep bep removed the Bug label Feb 21, 2017

@bep

This comment has been minimized.

Show comment
Hide comment
@bep

bep Feb 21, 2017

Member

The reason I ask about the macOS is the NFD way they store filenames.

So, given a Git repo:

▶ echo "test" >> π.txt                                                                       ◒
▶ git  add -A && git commit
[master (root-commit) a5dd87f] Test
 1 file changed, 1 insertion(+)
 create mode 100644 "\317\200.txt"

So, this is an issue that lives outside of Hugo.

Try:

git config --global core.quotePath false

Alternatively:

git config --global core.precomposeunicode true
Member

bep commented Feb 21, 2017

The reason I ask about the macOS is the NFD way they store filenames.

So, given a Git repo:

▶ echo "test" >> π.txt                                                                       ◒
▶ git  add -A && git commit
[master (root-commit) a5dd87f] Test
 1 file changed, 1 insertion(+)
 create mode 100644 "\317\200.txt"

So, this is an issue that lives outside of Hugo.

Try:

git config --global core.quotePath false

Alternatively:

git config --global core.precomposeunicode true

@bep bep closed this Feb 21, 2017

@rac2030

This comment has been minimized.

Show comment
Hide comment
@rac2030

rac2030 Feb 21, 2017

Thanks @bep that was the issue, I searched for a solution on the wrong side.
Setting core.quotePath to false resolves the issue on OSX and on debian as well.
So just for those reading this issue text later, here is my wercker.yml that allows it to build:

box: debian
build:
  steps:
    - install-packages:
        packages: git
    - script:
        name: Set git config core.quotePath to false for emoji filenames
        code: git config --global core.quotePath false
# Build the HUGO page in public folder
    - arjen/hugo-build@1.14.1:
        version: "0.18.1"
        theme: mainroad
        flags: --buildDrafts=false

rac2030 commented Feb 21, 2017

Thanks @bep that was the issue, I searched for a solution on the wrong side.
Setting core.quotePath to false resolves the issue on OSX and on debian as well.
So just for those reading this issue text later, here is my wercker.yml that allows it to build:

box: debian
build:
  steps:
    - install-packages:
        packages: git
    - script:
        name: Set git config core.quotePath to false for emoji filenames
        code: git config --global core.quotePath false
# Build the HUGO page in public folder
    - arjen/hugo-build@1.14.1:
        version: "0.18.1"
        theme: mainroad
        flags: --buildDrafts=false
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment