New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invalid byte sequence in US-ASCII #960

Closed
emzett opened this Issue Apr 13, 2013 · 23 comments

Comments

Projects
None yet
10 participants
@emzett
Copy link

emzett commented Apr 13, 2013

When I have a generated link that contains a german 'Umlaut' (e.g. ä,ü,), WEBrick will throw an error message:

Internal Server Error
invalid byte sequence in US-ASCII
WEBrick/1.3.1 (Ruby/1.9.3/2012-04-20) at 0.0.0.0:4000

from the log:

ttp://0.0.0.0:4000/Kategorie/logging-apps-fur-android.html -> /bootstrap/js/bootstrap.min.js
[2013-04-13 18:37:58] ERROR ArgumentError: invalid byte sequence in US-ASCII
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:384:in `fnmatch'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:384:in `block in nondisclosure_name?'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:383:in `each'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:383:in `nondisclosure_name?'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:324:in `check_filename'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:336:in `shift_path_info'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:299:in `set_filename'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:267:in `exec_handler'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:202:in `do_GET'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/abstract.rb:106:in `service'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/filehandler.rb:198:in `service'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpserver.rb:138:in `service'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpserver.rb:94:in `run'
    /home/user/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'
localhost.localdomain - - [13/Apr/2013:18:37:58 CEST] "GET /tag/Eichh%C3%B6rnchen HTTP/1.1" 500 311
http://0.0.0.0:4000/Kategorie/logging-apps-fur-android.html -> /tag/Eichh%C3%B6rnchen
@parkr

This comment has been minimized.

Copy link
Member

parkr commented Apr 13, 2013

Upgrade to 1.0.0.beta4 with gem install jekyll --pre! It supports utf 8 :)

@parkr parkr closed this Apr 13, 2013

@rkh

This comment has been minimized.

Copy link

rkh commented Jul 8, 2013

Still seeing this (or something similar) on 1.0.3. With the travis-ci website on Ruby 2.0.

@parkr

This comment has been minimized.

Copy link
Member

parkr commented Jul 8, 2013

@rkh What is your output?

@rkh

This comment has been minimized.

Copy link

rkh commented Jul 8, 2013

Configuration file: /Users/konstantin/Workspace/travis-ci.github.com/_config.yml
            Source: /Users/konstantin/Workspace/travis-ci.github.com
       Destination: /Users/konstantin/Workspace/travis-ci.github.com/_site
      Generating... Error reading file /Users/konstantin/Workspace/travis-ci.github.com/_layouts/en.html: invalid byte sequence in US-ASCII
Error reading file /Users/konstantin/Workspace/travis-ci.github.com/_layouts/es.html: invalid byte sequence in US-ASCII
Error reading file /Users/konstantin/Workspace/travis-ci.github.com/_layouts/foundation-en.html: invalid byte sequence in US-ASCII
Error reading file /Users/konstantin/Workspace/travis-ci.github.com/_layouts/fr.html: invalid byte sequence in US-ASCII
Error reading file /Users/konstantin/Workspace/travis-ci.github.com/_layouts/ja.html: invalid byte sequence in US-ASCII
Error reading file /Users/konstantin/Workspace/travis-ci.github.com/_layouts/pt-BR.html: invalid byte sequence in US-ASCII
Error reading file /Users/konstantin/Workspace/travis-ci.github.com/blog/_posts/2011-11-13-first_class_php_support_on_travis_ci.md: invalid byte sequence in US-ASCII
/Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/post.rb:352:in `scan': invalid byte sequence in US-ASCII (ArgumentError)
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/post.rb:352:in `extract_excerpt'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/post.rb:101:in `read_yaml'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/post.rb:57:in `initialize'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:185:in `new'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:185:in `block in read_posts'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:183:in `each'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:183:in `read_posts'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:140:in `read_directories'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:159:in `block in read_directories'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:154:in `each'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:154:in `read_directories'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:110:in `read'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/site.rb:42:in `process'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/command.rb:18:in `process_site'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/commands/build.rb:23:in `build'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/lib/jekyll/commands/build.rb:7:in `process'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/jekyll-1.0.3/bin/jekyll:85:in `block (2 levels) in <top (required)>'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/command.rb:155:in `run'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/runner.rb:402:in `run_active_command'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/runner.rb:78:in `run!'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/delegates.rb:11:in `run!'
    from /Users/konstantin/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/commander-4.1.3/lib/commander/import.rb:10:in `block in <top (required)>'

This is on 1.9.3 now.

@parkr

This comment has been minimized.

Copy link
Member

parkr commented Jul 8, 2013

What's the first paragraph of your post about first_class_php_support_on_travis_ci? Does it contain any non-US-ASCII chars?

@parkr

This comment has been minimized.

Copy link
Member

parkr commented Jul 8, 2013

Hm, it all works for me on a fresh clone of your repo, master branch. Same version of Jekyll.

~/code/travis-ci.github.com$ be jekyll build
Configuration file: /Users/parker/code/travis-ci.github.com/_config.yml
            Source: /Users/parker/code/travis-ci.github.com
       Destination: /Users/parker/code/travis-ci.github.com/_site
      Generating...
       Deprecation: Starting in 1.1, permalinks for pages in subfolders must be relative to the site source directory, not the parent directory. Check http://jekyllrb.com/docs/upgrading/ for more info.
                    ...done.
@rkh

This comment has been minimized.

Copy link

rkh commented Jul 8, 2013

I'm just using 1.8.7 for now.

@parkr

This comment has been minimized.

Copy link
Member

parkr commented Jul 8, 2013

@rkh Hm! Works for me with 1.8.7-p371 just fine. Not sure what's going on. Is GH:Pages building it properly?

@rkh

This comment has been minimized.

Copy link

rkh commented Jul 8, 2013

Yes, on 1.8.7 it works fine as Ruby 1.8 does not have encoding support.

@benbalter

This comment has been minimized.

Copy link
Contributor

benbalter commented Jul 21, 2013

I can recreate this issue after upgrading from Ruby 1.8.7 to 1.9.3. Happens consistently with multiple sites.

See bundler/bundler#1570 for background, which doesn't seem to resolve the issue in Jekyll's case, at least not for me.

Edit: LANG="en_US.UTF-8" and LC_ALL="en_US.UTF-8" prior to jekyll build resolved the problem.

/cc #836, #188, #1205

@slopjong

This comment has been minimized.

Copy link

slopjong commented Dec 30, 2013

LC_ALL seems to be sufficient.

This worked for me:

LC_ALL="en_US.UTF-8" jekyll serve
@omgmog

This comment has been minimized.

Copy link

omgmog commented Feb 11, 2014

Hate to be another "+1", but I was experiencing this "invalid byte sequence" issue with ruby 2.1.0p0 and jekyll 1.4.3 and setting encoding: utf-8 in my _config.yml fixed this.

@billwing

This comment has been minimized.

Copy link

billwing commented Mar 6, 2014

image
I can't resolve the problem.
Help me,please?
bootstrap 3.1.1
$ jekyll -v
jekyll 1.4.1
$ python
Python 2.7 (r27:82525, Jul 4 2010, 09:01:59) [MSC v.1500 32 bit (Intel)] on win32
$ ruby -v
ruby 1.9.3p194 (2012-04-20) [i386-mingw32]

@slopjong

This comment has been minimized.

Copy link

slopjong commented Mar 6, 2014

@billwing you shouldn't hijack an issue to report something else. For me it looks like you're using an incompatible version of jekyll with bootstrap 3.1.1.

@billwing

This comment has been minimized.

Copy link

billwing commented Mar 7, 2014

@slopjong Sorry,I opened the new issue #2117.
The twbs said that bootstrap If necessary, install Jekyll (requires v1.x) here,thx.

@Heliosmaster

This comment has been minimized.

Copy link

Heliosmaster commented Jun 29, 2014

Encountered the same issue again today with jekyll 1.5.x and ruby 2.1.2. Also adding the encoding manually to the _config.yml fixed it for me.

@kleinfreund

This comment has been minimized.

Copy link
Contributor

kleinfreund commented Jun 30, 2014

I agree with special characters that are allowed in domains and URLs should work, however actually using them should be avoided. There are still a lot of issues (e.g. copying an URL with an umlaut without problems) regarding this topic out there and the user will most likely run into them sooner or later.

@Heliosmaster

This comment has been minimized.

Copy link

Heliosmaster commented Jun 30, 2014

@kleinfreund yes and no. I am not entirely convinced something else is at play.
Locally i had no problem building, but then when i uploaded to the remote server i encountered this issue, and mind that both systems are en_US.UTF-8 as LC and LC_ALL.

And the file in question was: https://github.com/Heliosmaster/davide.im/blob/master/_posts/2013-10-08-incorporated.md
Maybe it was complaining about the ", but i'm not so sure at this point.

@kleinfreund

This comment has been minimized.

Copy link
Contributor

kleinfreund commented Jun 30, 2014

@Heliosmaster Your front-matter looks fine to me. :/

@Heliosmaster

This comment has been minimized.

Copy link

Heliosmaster commented Jun 30, 2014

@kleinfreund yes, that was precisely my point. It was complaining about the excerpt part. Very strange.

On another matter, setting the encoding by default to utf-8, would it be a bad design choice? Or at least document it a bit better. What do you think?

@kleinfreund

This comment has been minimized.

Copy link
Contributor

kleinfreund commented Jun 30, 2014

If I recall correctly, Jekyll is using utf-8 by default since Ruby v2.0.0. I don't have encoding set in my _config.yml at all.

@parkr

This comment has been minimized.

Copy link
Member

parkr commented Jun 30, 2014

@Heliosmaster said ~ 11hrs ago:

Encountered the same issue again today with jekyll 1.5.x and ruby 2.1.2. Also adding the encoding manually to the _config.yml fixed it for me.

@kleinfreund ☝️ Jekyll 1.5.x.

@jekyll jekyll locked and limited conversation to collaborators Feb 27, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.