Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

incompatible encoding regexp match (UTF-8 regexp with UTF-16LE string) (Encoding::CompatibilityError) #199

Closed
bobev18 opened this Issue Dec 9, 2013 · 13 comments

Comments

Projects
None yet
4 participants

bobev18 commented Dec 9, 2013

OS: XP 32bit
Ruby: ruby 2.1.0dev (2013-09-22) [i386-mingw32]

I get

C:\DevKit>ruby dk.rb init
dk.rb:120:in `block (3 levels) in scan_for': incompatible encoding regexp match (UTF-8 regexp with UTF-16LE string) (Encoding::CompatibilityError)
    from C:/Ruby210/lib/ruby/2.1.0/win32/registry.rb:596:in `each_key'
    from dk.rb:118:in `block (2 levels) in scan_for'
    from C:/Ruby210/lib/ruby/2.1.0/win32/registry.rb:421:in `open'
    from C:/Ruby210/lib/ruby/2.1.0/win32/registry.rb:528:in `open'
    from dk.rb:117:in `block in scan_for'
    from dk.rb:115:in `each'
    from dk.rb:115:in `scan_for'
    from dk.rb:137:in `block in installed_rubies'
    from dk.rb:137:in `collect'
    from dk.rb:137:in `installed_rubies'
    from dk.rb:145:in `init'
    from dk.rb:312:in `run'
    from dk.rb:331:in `<main>'

I edited line 118 in dk.rb from if skey =~ /\d\.\d\.\d/ to if skey.force_encoding("UTF-8") =~ /\d\.\d\.\d/u, and that worked.
I'd do a pull request but I'm not sure which is the corresponding file in the repository.

I hope this helps someone.

Owner

luislavena commented Dec 9, 2013

@bobev18 the file is this one:

https://github.com/oneclick/rubyinstaller/blob/master/resources/devkit/dk.rb.erb

Can you tell us the codepage you run the script from?

Also, have you tried setting # encoding: UTF-8 at the header of the script and test again?

That might also fix the encoding issues.

mitio commented Dec 13, 2013

@luislavena # encoding: UTF-8 at the top of the file does not solve the problem. Note that he's using Ruby 2.1 and this version of Ruby defaults ot UTF-8 encodings of all files.

I had the same problem with the DevKit, by the way and @bobev18's fix worked for me.

How can we check what codepage we run the script from?

Owner

luislavena commented Dec 24, 2013

@mitio

How can we check what codepage we run the script from?

Please use chcp in the Command Prompt.

mitio commented Dec 24, 2013

Active code page: 866
Owner

luislavena commented Dec 24, 2013

@mitio definitely the current codepage will affect the results of dk.rb

Have you tried setting chcp 65001 (which is unicode codepage) and see if the problem persist?

I'm currently attempting to solve our CI system in preparation for Ruby 2.1 release tomorrow, so probably there is not going to be a fix soon 😢

mitio commented Dec 24, 2013

No, changing the codepage does not help:

C:\Ruby_2.0_DevKit>chcp 65001
Active code page: 65001

C:\Ruby_2.0_DevKit>ruby dk.rb init
dk.rb:118:in `block (3 levels) in scan_for': incompatible encoding regexp match
(US-ASCII regexp with UTF-16LE string) (Encoding::CompatibilityError)
        from C:/Ruby210/lib/ruby/2.1.0/win32/registry.rb:596:in `each_key'
        from dk.rb:116:in `block (2 levels) in scan_for'
        from C:/Ruby210/lib/ruby/2.1.0/win32/registry.rb:421:in `open'
        from C:/Ruby210/lib/ruby/2.1.0/win32/registry.rb:528:in `open'
        from dk.rb:115:in `block in scan_for'
        from dk.rb:113:in `each'
        from dk.rb:113:in `scan_for'
        from dk.rb:135:in `block in installed_rubies'
        from dk.rb:135:in `collect'
        from dk.rb:135:in `installed_rubies'
        from dk.rb:143:in `init'
        from dk.rb:310:in `run'
        from dk.rb:329:in `<main>'

C:\Ruby_2.0_DevKit>chcp
Active code page: 65001

C:\Ruby_2.0_DevKit>

@ghost ghost assigned luislavena Dec 31, 2013

@luislavena luislavena added a commit that referenced this issue Apr 19, 2014

@luislavena luislavena Use encoding magic comment to workaround codepages
There are problems with different codepages (`chcp`) that causes `dk.rb` not to work properly.

Use a magic comment to mark the script as `UTF-8` in an attempt to workaround those issues.

Ref #199.
c61452a
Owner

luislavena commented Apr 19, 2014

@mitio have you tried adding # encoding: UTF-8 at the top of dk.rb script?

mitio commented Apr 19, 2014

@luislavena No, adding # encoding: UTF-8 at the top of the script does not solve the problem.

You have already asked that, I had forgotten, too :)

Owner

luislavena commented Apr 19, 2014

Sorry, complete facepalm. I believe there is a bug in the win32 registry
class, since similar problems have been reported.

Will take a look tomorrow again.

Sorry for top posting. Sent from mobile.
On Apr 19, 2014 5:18 PM, "Dimitar Dimitrov" notifications@github.com
wrote:

@luislavena https://github.com/luislavena No, adding # encoding: UTF-8at the top of the script does not solve the problem.

You have already asked thathttps://github.com/oneclick/rubyinstaller/issues/199#issuecomment-30538013,
I had forgotten, too :)

Reply to this email directly or view it on GitHubhttps://github.com/oneclick/rubyinstaller/issues/199#issuecomment-40879566
.

badal commented Sep 20, 2014

With a fresh Ruby21 and devkit installation on win7, I get the following error with any gem :

C:\Ruby21>bin\gem install json --platform=ruby
ERROR: While executing gem ... (Encoding::UndefinedConversionError)
U+2019 to CP850 in conversion from UTF-16LE to UTF-8 to CP850

I really do not guess where the UTF-16LE come from ...
_md

Owner

luislavena commented Sep 20, 2014

@badal your user account on Windows contains accented characters.

You will find reports of this here and also on RubyInstaller mailing list:

http://groups.google.com/group/rubyinstaller/

Please change your codebase to something that supports UTF, either 65001 (unicode) or a Latin one (1252 perhaps).

Use search functionality here or the mailing list for similar reports and how to change your settings.

badal commented Sep 20, 2014

@luislavena 'chcp 1252' did it. Thank for the hyper-fast answer !
Maybe a hint on that in the dekkit/installer page would be useful.

_md

Owner

luislavena commented Sep 20, 2014

@badal wiki pages are open to all GitHub registered users, please feel free to add it or to the Troubleshooting page instead:

@luislavena luislavena closed this Sep 24, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment