Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid US-ASCII character "\xE2" #4268

Closed
dalanmiller opened this issue Dec 17, 2015 · 31 comments
Closed

Invalid US-ASCII character "\xE2" #4268

dalanmiller opened this issue Dec 17, 2015 · 31 comments
Assignees

Comments

@dalanmiller
Copy link

@dalanmiller dalanmiller commented Dec 17, 2015

There's quite a few pages on the Internet talking about this problem with Ruby but none of the solutions seem to work with Jekyll. Ultimately, I'm trying to build a Jekyll site in a Docker container and I've tried setting @charset as well as all the locale ENV variables. is there something I can set in _config.yml to solve this?

Configuration file: /www/_config.yml
            Source: /www
       Destination: /www/_site
 Incremental build: enabled
      Generating...
  Conversion error: Jekyll::Converters::Scss encountered an error while converting 'assets/css/styles.scss':
                    Invalid US-ASCII character "\xE2" on line 3
/usr/local/bundle/gems/jekyll-sass-converter-1.3.0/lib/jekyll/converters/scss.rb:97:in `rescue in convert': Invalid US-ASCII character "\xE2" on line 3 (Jekyll::Converters::Scss::SyntaxError)
    from /usr/local/bundle/gems/jekyll-sass-converter-1.3.0/lib/jekyll/converters/scss.rb:95:in `convert'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/convertible.rb:67:in `block in transform'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/convertible.rb:65:in `each'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/convertible.rb:65:in `reduce'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/convertible.rb:65:in `transform'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/convertible.rb:249:in `do_layout'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/page.rb:122:in `render'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/site.rb:302:in `block in render'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/site.rb:301:in `each'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/site.rb:301:in `render'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/site.rb:55:in `process'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/command.rb:28:in `process_site'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/commands/build.rb:58:in `build'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/commands/build.rb:34:in `process'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/lib/jekyll/commands/build.rb:18:in `block (2 levels) in init_with_program'
    from /usr/local/bundle/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `call'
    from /usr/local/bundle/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `block in execute'
    from /usr/local/bundle/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `each'
    from /usr/local/bundle/gems/mercenary-0.3.5/lib/mercenary/command.rb:220:in `execute'
    from /usr/local/bundle/gems/mercenary-0.3.5/lib/mercenary/program.rb:42:in `go'
    from /usr/local/bundle/gems/mercenary-0.3.5/lib/mercenary.rb:19:in `program'
    from /usr/local/bundle/bundler/gems/jekyll-43a28aed967b/bin/jekyll:20:in `<top (required)>'
    from /usr/local/bundle/bin/jekyll:16:in `load'
    from /usr/local/bundle/bin/jekyll:16:in `<main>'
rake aborted!
Command failed with status (1): [bundle exec jekyll build --trace...]
/www/Rakefile:295:in `jekyll'
/www/Rakefile:97:in `block in <top (required)>'
Tasks: TOP => build
(See full trace by running task with --trace)
@parkr
Copy link
Member

@parkr parkr commented Dec 17, 2015

Try adding this to your _config.yml

encoding: utf-8

This character is is â. Do you have any non-English characters in assets/css/styles.scss or any of its imports?

@parkr parkr added the support label Dec 17, 2015
@dalanmiller
Copy link
Author

@dalanmiller dalanmiller commented Dec 17, 2015

Happened again with the _config.yml change. I ack'd through my repo and couldn't find any of those and don't see why I would have any non-English characters in the imports. Also, it says it's occurring here in the styles.scss file:

Conversion error: Jekyll::Converters::Scss encountered an error while converting 'assets/css/styles.scss':
                    Invalid US-ASCII character "\xE2" on line 3
/usr/local/bundle/gems/jekyll-sass-converter-1.3.0/lib/jekyll/converters/scss.rb:97:in `rescue in convert': Invalid US-ASCII character "\xE2" on line 3 (Jekyll::Converters::Scss::SyntaxError)

assets/css/styles.scss

---
---
@charset "utf-8";

// Bourbon, Neat, Bitters, Base styles
@import "normalize";
@import "bourbon";
@import "neat";
@import "base/base";
@dalanmiller
Copy link
Author

@dalanmiller dalanmiller commented Dec 17, 2015

Also hi @parkr! I ate lunch with you at Github Universe!

@dalanmiller
Copy link
Author

@dalanmiller dalanmiller commented Dec 17, 2015

For the record, I found this great one-liner for finding non-ASCII characters:

# Linux
grep --color='auto' -P -n "[\x80-\xFF]" file.xml

# Mac OS X after brew install pcre
pcregrep --color='auto' -n "[\x80-\xFF]" file.xml
@pathawks
Copy link
Member

@pathawks pathawks commented Dec 18, 2015

If you comment out all the @imports in assets/css/styles.scss do you still get the same error message? If you comment out all but one, can you figure out which one is causing the error message?

@parkr
Copy link
Member

@parkr parkr commented Dec 18, 2015

Ok, so per the Sass documentation:

Sass follows the CSS spec to determine the encoding of a stylesheet, and falls back to the Ruby string encoding. This means that it first checks the Unicode byte order mark, then the @charset declaration, then the Ruby string encoding. If none of these are set, it will assume the document is in UTF-8.

So it should assume UTF-8, so this character should not be a problem. Interesting issue, this is. Line three would be the line which contains // Bourbon, Neat, Bitters, Base styles, which doesn't appear to contain \xE2. Try @pathawks's solution for debugging this... this is super weird. If you can isolate a given line in your CSS through this method, that would help. We delegate imports entirely to Sass, so they should be able to slurp up everything as UTF-8 by default as specified.

Also hi @parkr! I ate lunch with you at Github Universe!

Hey man, thanks for stopping by Jekyll's issues! :)

@dalanmiller
Copy link
Author

@dalanmiller dalanmiller commented Dec 18, 2015

Nice to see a friendly face on the Internet now and again @parkr 👍

So just to follow up, I used that unicode character search and found just two lines where I could just replace them with ASCII equivalents. They were both comments anyway so nothing got hurt and I could build my Jekyll docker container.

It'd probably be a lot more helpful for jekyll/jekyll-sass-converter to output the problems lines of the original source files as it traverses the import "tree" versus just the top most line where an @import occurs

@parkr
Copy link
Member

@parkr parkr commented Dec 18, 2015

It'd probably be a lot more helpful for jekyll/jekyll-sass-converter to output the problems lines of the original source files as it traverses the import "tree" versus just the top most line where an @import occurs

It may be an issue in Sass itself, as we output the line from the Sass error. If Sass passed us back the line in the imported file, then that would help a bit more, of course. Maybe #sass_backtrace would be more helpful? If you have a few minutes, would be great to have you test out the output of that in your failure scenario.

@burntcookie90
Copy link

@burntcookie90 burntcookie90 commented Dec 25, 2015

I'm seeing this issue when using octopress on one of my machines but not the other. Haven't been able to figure out which possible variable is causing the issue on the erroring machine.

edit: figured it out, the new linux machine didn't have it's locale set to utf-8 at a system level. After setting that, it works as expected

@pathawks
Copy link
Member

@pathawks pathawks commented Dec 25, 2015

Can somebody post the source of a site? I would like to investigate this issue

@parkr
Copy link
Member

@parkr parkr commented Dec 26, 2015

I believe this issue is fixed in jekyll-sass-converter v1.4.0, which I just released. https://github.com/jekyll/jekyll-sass-converter/releases/tag/v1.4.0

@parkr parkr closed this Dec 26, 2015
@AngeloAballe
Copy link

@AngeloAballe AngeloAballe commented Dec 27, 2015

I'm also having this issue. No problem if hosted via GitHub pages, but when I use docker on my own laptop or server, it has this issue.

The problem is the

---
---

above the scss file. If you remove this, there would be no error but the scss will not be converted. Still don't know how to fix this.

@parkr
Copy link
Member

@parkr parkr commented Dec 27, 2015

Did you upgrade your version of Jekyll-sass-converter?

@AngeloAballe
Copy link

@AngeloAballe AngeloAballe commented Dec 27, 2015

Imgur

I tried. Same error.

@parkr
Copy link
Member

@parkr parkr commented Dec 27, 2015

@AngeloAballe The YAML front matter (two lines of ---) are not the problem – they merely tell Jekyll to convert the SCSS into CSS. Have you tried outside of Docker? See Dalan's comment about finding these characters:

# Linux
grep --color='auto' -P -n "[\x80-\xFF]" file.xml

# Mac OS X after brew install pcre
pcregrep --color='auto' -n "[\x80-\xFF]" file.xml
@AngeloAballe
Copy link

@AngeloAballe AngeloAballe commented Dec 27, 2015

Thanks for the reply @parkr but after searching, I manage to fix it and I'm sharing my solution. I found out that the docker container is using a different charset or locale so you have to set it in you docker file.

I manage to fix mine by adding this on my Dockerfile. No need to upgrade to sass converter 1.4.0.

# Install program to configure locales
RUN apt-get install -y locales
RUN dpkg-reconfigure locales && \
  locale-gen C.UTF-8 && \
  /usr/sbin/update-locale LANG=C.UTF-8

# Install needed default locale for Makefly
RUN echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen && \
  locale-gen

# Set default locale for the environment
ENV LC_ALL C.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8

Hope this helps everyone!

@envygeeks envygeeks reopened this Dec 27, 2015
@envygeeks envygeeks closed this Dec 27, 2015
@envygeeks
Copy link
Contributor

@envygeeks envygeeks commented Dec 27, 2015

hey @AngeloAballe if you are using the official Docker image can you push a ticket to https://github.com/jekyll/docker and we'll happily add that as a default, I didn't think Alpine lacked the default language vars (but I'm not surprised, it is an embedded OS.)

@mkyung
Copy link

@mkyung mkyung commented May 13, 2016

Just for the future reference:

I originally turned off Set locale environment variables on startup in my OSX Terminal preference due to other reasons. The error is gone if I check this checkbox.

@michaellee
Copy link
Contributor

@michaellee michaellee commented Jun 14, 2016

I ran into this issue as well while trying to build my project over at GitLab. What was causing the issue was this bit of code:

...
a::before{
  content:'“';
}
a::after{
  content:'”';
}
...

The and were the culprits. Changing them to be the hex value fixed the issue.

...
a::before{
  content:'\201C';
}
a::after{
  content:'\201D';
}
...
@simonorono
Copy link

@simonorono simonorono commented Aug 8, 2016

I'm still getting this error on latest Jekyll. Check the CI log:

https://gitlab.com/simonorono/simonorono.gitlab.io/builds/2904606

@nateberkopec
Copy link
Contributor

@nateberkopec nateberkopec commented Oct 19, 2016

I get this problem when building a Jekyll site with wercker and there's Unicode in the SASS. Removing the UTF-8 characters or using @AngeloAballe's solution works.

@MildTomato
Copy link

@MildTomato MildTomato commented Nov 5, 2016

Hey guys!

I'm using Jekyll 3.3.0 and wercker. Getting a Invalid US-ASCII character "\xE2" on Wercker whenever it tries to build.
Seems to be something in foundation-sites which is causing it, but can't find which file is causing the issue.
Only getting the invalid character issue on wercker, runs fine running locally.

@MildTomato
Copy link

@MildTomato MildTomato commented Nov 5, 2016

OK, seemed to be a in foundation-sites/scss/typography/_base.scss on line 15, one of the comments.

Wercker runs fine now 😆

@Hwesta
Copy link

@Hwesta Hwesta commented Oct 23, 2017

I also ran into this problem, and found this thread very helpful but it didn't cover my case. Hopefully this helps the next person.

For me, setting LANG=en_US.UTF-8 didn't work because the locale hadn't been generated on my system and it was falling back to LANG=C which caused the error. Additionally, the problematic character I had was an en-dash (\xe2\x80\x93) not actually â (\xe2). The fix was to generate a UTF-8 locale and set it. Instructions tested on Arch Linux.

  1. Check if locale exists before you try to set it: locale -a
  2. If not, generate locale: Uncomment locale name in /etc/locale.gen and run sudo locale-gen
  3. Set locale for the system: Edit /etc/locale.conf and log in again for it to take effect
  4. Or set locale for the terminal
    • bash: export LANG=en_US.UTF-8
    • fish: set -xg LANG en_US.UTF-8
dltj added a commit to dltj/dltj-blog that referenced this issue Jan 11, 2018
@dltj
Copy link

@dltj dltj commented Jan 11, 2018

My jekyll build is being done by AWS CodeBuild, and I found I needed to add these lines to the buildspec.yml file:

  pre_build:
    commands:
      - export LC_ALL="C.UTF-8"
      - export LANG="en_US.UTF-8"
      - export LANGUAGE="en_US.UTF-8"
@IotaSpencer
Copy link

@IotaSpencer IotaSpencer commented Jan 24, 2018

If anyone is getting this on GitLab, add the list on the above ^
#4268 (comment)

just put this in a before_script: before everything else

@bsde1245
Copy link

@bsde1245 bsde1245 commented Feb 8, 2018

I get this error in gitlab, @IotaSpencer Where to add the code. I dont find buildspec.yml file
on sass converter v 1.5.0

Edit: foud it it is in the .gtlab-ci.yml

@reardestani
Copy link

@reardestani reardestani commented Mar 31, 2018

For Jekyll on Gitlab, alternative to @IotaSpencer's solution is adding them in variables, like:

variables:
  LC_ALL: "C.UTF-8"
  LANG: "en_US.UTF-8"
  LANGUAGE: "en_US.UTF-8"
@npeirson
Copy link

@npeirson npeirson commented Apr 21, 2018

Solution above looks good, I also got around it by putting an env assignment in my rake file.

@pathawks
Copy link
Member

@pathawks pathawks commented Apr 22, 2018

It seems this is an environment issue rather than a bug Jekyll could or should do something about. Maybe we could add some documentation regarding this?

Otherwise, this issue should be closed.

/cc: @jekyll/documentation

@stragu
Copy link

@stragu stragu commented Jun 11, 2018

If anyone on GitLab Pages is wondering why @reardestani 's solution does not work, there is a typo (a missing "). It should be:

variables:
  LC_ALL: "C.UTF-8"
  LANG: "en_US.UTF-8"
  LANGUAGE: "en_US.UTF-8"

Thanks everyone, I added those variables to my .gitlab-ci.yml file and it fixed it (and looked rather clean) :)

@jekyll jekyll locked as resolved and limited conversation to collaborators Jun 11, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.