Accents on path will crash jekyll serve

[oremrodrigo]

• I believe this to be a bug, not a question about using Jekyll.
• I updated to the latest Jekyll (or) if on GitHub Pages to the latest github-pages
• I ran jekyll doctor to check my configuration
• I read the CONTRIBUTION file at https://jekyllrb.com/docs/contributing/
• This is a feature request.

---

• I am on (or have tested on) macOS 10+
• I am on (or have tested on) Debian/Ubuntu GNU/Linux
• I am on (or have tested on) Fedora GNU/Linux
• I am on (or have tested on) Arch GNU/Linux
• I am on (or have tested on) Other GNU/Linux
• I am on (or have tested on) Windows 10+

---

• I was trying to install.
• There is a broken Plugin API.
• I had an error on GitHub Pages, and I have reproduced it locally.
• I had an error on GitHub Pages, and GitHub Support said it was a Jekyll Bug.
• I had an error on GitHub Pages and I did not test it locally.
• I was trying to build.
• It was another bug.

My Reproduction Steps

If the path where jekyll project is located contains accented characters, jekyll serve will crash after any modification on the directory. Building the project works fine, but it will crash when the watcher detects a change on the directory.

On version 3.6.2 it works correctly, so I believe this is a regression.

Let's use the latest jekyll release (v3.8.3):

1. Create a directory called test-olé, cd into it.
2. bundle init, bundle add jekyll, bundle install
3. touch index.html
4. Run bundle exec jekyll serve, it will build and start the server correctly.
5. On another terminal, run touch hi on the same directory.
6. Jekyll will crash:

➜  test-olé $ jekyll serve
Configuration file: none
            Source: /Users/rodrigoorem/Documents/test-olé
       Destination: /Users/rodrigoorem/Documents/test-olé/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
                    done in 0.01 seconds.
 Auto-regeneration: enabled for '/Users/rodrigoorem/Documents/test-olé'
    Server address: http://127.0.0.1:4000
  Server running... press ctrl-c to stop.
      Regenerating: 1 file(s) changed at 2018-06-18 23:21:46
E, [2018-06-18T23:21:46.231114 #1578] ERROR -- : exception while processing events: incompatible character encodings: ASCII-8BIT and UTF-8 Backtrace:
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `sub'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `block (2 levels) in listen_handler'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `map'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `block in listen_handler'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/config.rb:23:in `call'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:115:in `_process_changes'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:19:in `block in loop_for'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:15:in `loop'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:15:in `loop_for'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/loop.rb:84:in `_wait_for_changes'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/loop.rb:42:in `block in setup'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/internals/thread_pool.rb:6:in `block in add'

Now let's use an older jekyll version.

1. Set jekyll version to 3.6.2 on Gemfile and run bundle update.
2. Run again bundle exec jekyll serve.
3. On another terminal, run touch hey.
4. Jekyll will not crash.

I also tried the master branch and I got the same error of the latest version.

[chrisfinazzo]

Can you escape the accented characters in the pathname? Seems like something else is awry here with how this is handled 🤔.

[oremrodrigo]

Sorry, how would I escape the accented characters? File names are handled by Jekyll.

[chrisfinazzo]

My bad, it wasn't clear to me initially what you meant. I know there are some Liquid functions that deal with escape characters and encodings in body text, but I have not tried this with directory structures and honestly don't know what the limitations are.

(Can you run this again with --trace?)

Seems like something that @jekyll/build should look into.

[oremrodrigo]

Running with --trace:

➜  test-olé $ bundle exec jekyll serve --trace
Configuration file: none
            Source: /Users/rodrigoorem/Documents/test-olé
       Destination: /Users/rodrigoorem/Documents/test-olé/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
                    done in 0.013 seconds.
 Auto-regeneration: enabled for '/Users/rodrigoorem/Documents/test-olé'
    Server address: http://127.0.0.1:4000
  Server running... press ctrl-c to stop.
      Regenerating: 2 file(s) changed at 2018-06-21 21:05:04
E, [2018-06-21T21:05:04.062601 #26024] ERROR -- : exception while processing events: incompatible character encodings: ASCII-8BIT and UTF-8 Backtrace:
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `sub'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `block (2 levels) in listen_handler'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `map'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.0.0/lib/jekyll/watcher.rb:62:in `block in listen_handler'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/config.rb:23:in `call'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:115:in `_process_changes'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:19:in `block in loop_for'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:15:in `loop'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/processor.rb:15:in `loop_for'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/loop.rb:84:in `_wait_for_changes'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/event/loop.rb:42:in `block in setup'
 -- /usr/local/lib/ruby/gems/2.5.0/gems/listen-3.1.5/lib/listen/internals/thread_pool.rb:6:in `block in add'

[oremrodrigo]

Maybe this issue should be on jekyll/jekyll-watch?

I'm not familiar with ruby but I tried to change this file https://github.com/jekyll/jekyll-watch/blob/master/lib/jekyll/watcher.rb#L62 :

-        c.map { |path| path.sub("#{site.source}/", "") }.each do |file|
+        c.map { |path| path.force_encoding('UTF-8').sub("#{site.source}/", "") }.each do |file|

With this modification, I don't get the error and jekyll serve seems to work correctly.

[ashmaroli]

@oremrodrigo Thank you for reporting this issue. I went ahead and opened a PR at jekyll/jekyll-watch. I opted to not go with force_encoding as that did not seem to resolve the issue on my Windows system.