Fix the codepage issue for Windows developers #1835

Closed
Haacked opened this Issue Dec 16, 2013 · 16 comments

8 participants

@Haacked

When Windows users attempt to run a Jekyll command such as jekyll generate, they run into the following error:

Incompatible character encodings: UTF-8 and IBM437...

There is a workaround for those who stick around long enough to discover it.

chcp 65001

However, I think this is important to fix. I've been promoting GitHub Pages + Jekyll to Windows users and I get a lot of responses that it doesn't work on Windows. I think it's friction like this that prevents people from using it.

Here's some Ruby code that sets the codepage only if the user is on Windows.

if (/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM) != nil
  puts '## Set the codepage to 65001 for Windows machines'
  `chcp 65001`
end

I tried to submit a PR with this, but I wasn't sure what the most proper place to put the code is. @Ivoz suggests putting it here: https://github.com/jekyll/jekyll/blob/master/bin/jekyll

I'm not sure where exactly to put it in there. I'm not much of a Ruby programmer. Sorry.

If there's a more fundamental fix we could make, that'd be helpful as well.

@benbalter

👍 to Jekyll support for windows.

:+2: for smarter Jekyll encoding support that can handle or transcode non-UT8 encodings, as we see this in production fairly often, but can't seem to recreate locally on OS X, which I would suspect means this affects Linux as well.

@parkr
Jekyll member

I definitely like the idea of making it easier to use Jekyll on Windows, despite my negative feelings about Microsoft as a company. I'm sorry if I rejected your PR too harshly – you're just trying to help and I appreciate it.

The codepage problem is definitely Windows-only (so we should be fine on the Linux front, though problems with LOCALE and LC_ALL may persist without modification -- but this is more a Ruby problem than a Jekyll-specific problem), and we know we can do something about it. We can certainly setup some tooling around platform detection and enforcement of compatibility (Jekyll.platform, Jekyll.running_on_windows?, Jekyll.compatiblize).

We setup Windows docs as a first step: http://jekyllrb.com/docs/windows/ but there's certainly more that can be done.

The biggest blocker to doing more of this has been that neither @mattr- nor I own a Windows-running computer and we can't test that these things work on Travis. I guess we just let the Windows users yell at us if it stops working :)

@Haacked

despite my negative feelings about Microsoft as a company.

Haven't you heard? Google is the evil empire now. Or is it every large technology company? 😛

Also, the folks who use Microsoft technologies aren't necessarily deserving of our scorn. We can show them a better way with tools like Jekyll. Think of it as a gateway drug to The Ruby Way™.

The biggest blocker to doing more of this has been that neither @mattr- nor I own a Windows-running computer and we can't test that these things work on Travis.

I'd be happy to test what I can on my machine. I don't know much about Travis.

I guess we just let the Windows users yell at us if it stops working :)

Sounds like a plan.

@parkr
Jekyll member

Haven't you heard? Google is the evil empire now. Or is it every large technology company? 😛

Haha, interesting! I'd say Microsoft's history of fighting against open-source development (capitalist purists, where economic gain is the only driver) would hinder my happiness about them. Your counter example, Google, is essentially turning into the next-gen Bell Labs, so I guess I would tend to like them a bit more. 😛

the folks who use Microsoft technologies aren't necessarily deserving of our scorn

Someone has to apply the pressure to divest in Microsoft. Perhaps we can do it! 😉 (totally kidding)

I'd be happy to test what I can on my machine. I don't know much about Travis.

Cool! I'll ping you when we get a Windows-related PR.

@parkr
Jekyll member

So. How often does the code page have to be set?

@Haacked

It needs to be set once per shell. But if they close the shell and re-open the shell, it needs to be set again.

aside discussion

Haha, interesting! I'd say Microsoft's history of fighting against open-source development (capitalist purists, where economic gain is the only driver) would hinder my happiness about them.

True, but that's the Microsoft of old. Microsoft has always been more like this:

There are groups that are making great strides in OSS over there such as the Azure team.

When I worked there, we released a ton of code under OSS licenses and were among the first teams to start accepting contributions back from the community. It's not a mono-culture. There are certainly groups intent on fighting OSS. But I think many progressive execs there see the light at the end of the tunnel.

I'm not trying to convince you to like them. What do I care? I don't work there anymore. Just pointing out that it's not so cut and dried anymore. The world has changed. 😄

@madhur

+1 for this. I also heavily use Jekyll for my site and whole site has been built on Windows with Notepad++. It would be great to fix the CHCP 65001 issue once for all.

@aigarsdz

@svnpenn changing the code page globally might have a negative effect on other applications or even on the operating system itself. I would put this command in the PowerShell profile file instead, so every time you open a PowerShell window the code page would be changed. Not sure how to automate it with CMD.

@XhmikosR

👍

Also see #1882 (comment)

I personally add the @chcp 65001 in jekyll.cmd and it works as long as I don't upgrade/downgrade Jekyll. In that case I just have to re-add it.

@parkr
Jekyll member

@Haacked How does the codepage work? Can it be executed from within a Ruby process? Should we ship a jekyll.cmd the way Hubot does, which sets the codepage then runs plain ol' jekyll?

@XhmikosR

@parkr: #1835 (comment)

Works perfectly fine with Ruby 1.9.3 I'm using.

@svnpenn: no need for cmd /c; plain chcp should work...

@svnpenn

But why would we need chcp anymore if default is UTF-8?

@XhmikosR

Sure, in theory it shouldn't be needed. I haven't tried it since there's no gem available yet.

@svnpenn

Yes it is

gem i jekyll --pre
@mscharley

As a Windows jekyll user who's never had an issue with this since at least 1.3 with the native versions of ruby 1.9/2.0 from RubyInstaller and even earlier versions of jekyll with cygwin, can someone explain what actually causes this issue?

@mscharley

@svnpenn If it's fixed, shouldn't it be closed then?

@parkr parkr closed this Mar 9, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment