Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ruby Console fails with some non-ascii characters ("Š") #17

Closed
Nazz78 opened this issue Dec 18, 2017 · 18 comments
Closed

Ruby Console fails with some non-ascii characters ("Š") #17

Nazz78 opened this issue Dec 18, 2017 · 18 comments

Comments

@Nazz78
Copy link

@Nazz78 Nazz78 commented Dec 18, 2017

  1. SketchUp/LayOut Version: SU 2018
  2. OS Platform: Windows

When using "Š" character, SketchUp Ruby fails flat.

puts "ABCŠ "
# Error: #<SyntaxError: <main>: invalid multibyte char (UTF-8)
# <main>: unterminated string meets end of file>
# SketchUp:1:in `eval'

print "ABCŠ123456"
# Error: SketchUp:1:in `eval'

Reported also in centercode SU-0396.

@thomthom

This comment has been minimized.

Copy link
Member

@thomthom thomthom commented Dec 19, 2017

Can you attach a sample file please? I'm suspecting that the file isn't actually saved in UTF-8.

@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Dec 20, 2017

There is no file, the error pops up if you try to run it in Ruby Console. But it doesn't in SU 2017.

@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Dec 20, 2017

There is also no problem with the example under SU 2016.

@Nazz78, try the following at the console in both SU2017 and SU2018 & reports results (in a ```text block.)

puts "\n"<<'Sketchup version  : ' << Sketchup.version
puts 'External encoding : ' << Encoding.default_external.inspect
puts 'Internal encoding : ' << Encoding.default_internal.inspect

As an example for SU2016, I get:

Sketchup version  : 16.1.1449
External encoding : #<Encoding:UTF-8>
Internal encoding : nil
@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Dec 20, 2017

Hi Dan, these are my results (running on Windows 10)

Sketchup version  : 17.2.2555
External encoding : #<Encoding:UTF-8>
Internal encoding : nil
Sketchup version  : 18.0.16975
External encoding : #<Encoding:UTF-8>
Internal encoding : nil
@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Dec 21, 2017

Okay, thanks, ... looks the same as always.

(1a) For SU2018, did you cut and paste the failing text from some other application window ?
(1b) ... or did you type it directly into the console using the keyboard ?

What do you get when you type these at the console(s) ?

__ENCODING__
Encoding.find("filesystem")
Encoding.find("locale")

?


I'm also wondering if the %x execute strings are broken again ?

puts %x{dir}.gsub("\n","\n\r")

... should output the current working directory listing in the Ruby Console.

?

@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Dec 22, 2017

I get an error in any case, if I input it manually or if I copy/paste it.

In both, SU 17 and 18 I get the same result:

__ENCODING__
Encoding.find("filesystem")
Encoding.find("locale")
=> Windows-1252

Executing puts %x{dir}.gsub("\n","\n\r") just blinks the command prompt and that's it.

@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Dec 22, 2017

In both, SU 17 and 18 I get the same result:

Okay, there is a difference from SU2016 where __ENCODING__ returns "UTF-8" just after launch with only the "Tools" folder and the default stuff from "RubyStdLib" loaded.

And the %x (backtick) execute strings work fine.
(I can't believe they broke it AGAIN. It was broke for SU2014 I think.)

@thomthom

This comment has been minimized.

Copy link
Member

@thomthom thomthom commented Dec 22, 2017

I can reproduce #<SyntaxError: <main>: invalid multibyte char (UTF-8) on my machine as well when I use Norwegian characters in the Ruby Console. I'm surprised no one (not even I) ran into this before.

@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Dec 22, 2017

I guess it's because if ran in loaded script, this problem doesn't seem to appear!?

@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Dec 22, 2017

I get an error in any case, if I input it manually or if I copy/paste it.

I think the console gets it's input from STDIN.
What do you get at the console typing:

STDIN.external_encoding

?

If you get "Windows-1252", you could try ...

STDIN.set_encoding("UTF-8")

and try again.

@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Dec 22, 2017

No luck, it's UTF-8 already... :-)

@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Dec 22, 2017

Here is an Encoding Report generator script.

You drop the "enc.rb" file in the default HOME folder ("~/Documents") ,,,
and then from SketchUp's console type ...

load 'enc.rb'

enc.zip

It shows the difference when Encoding.default_internal is not set (nil) and set to "UTF-8", as well as setting STDIN encoding to "UTF-8" in the 2nd pass.

@thomthom, It should be obvious that Encoding.default_internal should be set when the Ruby interpreter is FIRST loaded, so that String#encode() can be used to convert to internal (script) encoding, INSTEAD of doing hacks like, at top of ...
https://github.com/SketchUp/htmldialog-inputbox/blob/master/src/input_box.rb

  file = __FILE__.dup
  # Account for Ruby encoding bug under Windows.
  file.force_encoding('UTF-8') if file.respond_to?(:force_encoding)

INSTEAD you just do ...

  file = __FILE__.encode
  • The duck typing is not needed for that example as it is a SU2017+ ONLY feature which is Ruby 2+ Encoding aware.

P.S. it is NOT necessary to do dup as each time __FILE__ is called it returns a NEW String instance object.

__FILE__.__id__
#=> 90762440
__FILE__.__id__
#=> 90761820
__FILE__.__id__
#=> 90761260
__FILE__.__id__ == __FILE__.__id__
#=> false
__FILE__.equal?( __FILE__ )
#=> false
@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Dec 22, 2017

No luck, it's UTF-8 already... :-)

DARN. SO much for the easy fix ... :sadface:

@thomthom

This comment has been minimized.

Copy link
Member

@thomthom thomthom commented Jan 3, 2018

Logged as SU-38424

@thomthom thomthom changed the title SU Ruby fails with some non-ascii characters ("Š") Ruby Console fails with some non-ascii characters ("Š") Jan 3, 2018
@thomthom thomthom added logged and removed need more info labels Jan 3, 2018
@AntonSynytsia

This comment has been minimized.

Copy link

@AntonSynytsia AntonSynytsia commented Jan 20, 2018

Is this bug still present in SU2018? Because when I paste this into Ruby console, the string is converted into questions marks:

str = "Здорово"
# => "?????"
str.encoding
# => "UTF-8"

I don't get any issues with this in SU2017 however

@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Jan 20, 2018

@AntonSynytsia - precisely, this is SU 2018 bug.

@DanRathbun

This comment has been minimized.

Copy link

@DanRathbun DanRathbun commented Jan 22, 2018

@thomthom Reported in the public Ruby API forums that Unicode path support is again broken in SU2018.

The APPDATA paths and Gem paths are not working. Is this part of the same regression ?

@Nazz78

This comment has been minimized.

Copy link
Author

@Nazz78 Nazz78 commented Jan 22, 2018

When testing Modelur with SketchUp 2018, we found no difference from 2017 in this respect. But we didn't get into any detailed testing...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.