Skip to content
This repository

Problem making rubies compile with readline from homebrew to avoid corrupt unicode input in IRB #1016

Closed
maxim opened this Issue · 22 comments

3 participants

Maxim Chernyak aka hakunin Michal Papis Amalrik Maia
Maxim Chernyak aka hakunin
maxim commented

If you have readline installed via homebrew on OSX Lion, and you run rvm install 1.9.3, then go to 1.9.3's irb and try to type in, say, cyrillic, you would get unicode-escaped symbols like this.

~ rvm use 1.9.3
♢ irb
>> typing cyrillic: \U+FFD1\U+FFD0\U+FFB5\U+FFD1\U+FFD1

This is because ruby did not compile with the correct readline. Even though you have readline installed via homebrew, doesn't mean rvm uses it by default. In fact, homebrew actually states that it doesn't link readline on purpose — to avoid breaking existing software.

~ brew info readline
readline 6.2.2
http://tiswww.case.edu/php/chet/readline/rltop.html

This formula is keg-only.
OS X provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only.

/usr/local/Cellar/readline/6.2.2 (30 files, 1.7M)
https://github.com/mxcl/homebrew/commits/master/Library/Formula/readline.rb

So now we just need to recompile 1.9.3 with readline. But that wouldn't work either.

~ rvm recompile 1.9.3 --with-readline-dir=/usr/local/Cellar/readline/6.2.2
[...]

~ rvm use 1.9.3
♢ irb
>> typing cyrillic: \U+FFD1\U+FFD0\U+FFB5\U+FFD1\U+FFD1

Still the same problem. That's because ruby config flags don't seem to work well. Or maybe rvm doesn't pass them correctly? To fix this problem you could bypass the flag and just modify the env variables.

~ LDFLAGS=-L/usr/local/Cellar/readline/6.2.2/lib CFLAGS=-I/usr/local/Cellar/readline/6.2.2/include rvm reinstall 1.9.3 -j 3

This actually makes everything work.

♢ irb
>> x="привет"
=> "привет"

Awesome.

Thanks to @mpapis for walking me through this on IRC.

Michal Papis
Collaborator

--with-opt-lib= is now available again, try:

rvm recompile 1.9.3 --with-readline-dir=/usr/local/Cellar/readline/6.2.2
Amalrik Maia

Hi,
I'm getting the same problem with accented characters in IRB.

typed çó:

1.9.3:ruby-1.9.2-p320
 $ irb
1.9.3-p327 :001 > \U+FFC3\U+FFA7\U+FFC3\U+FFB3

I think my ruby is using libedit instead readline:

1.9.3:~
 $ otool -L /Users/amalrik/.rvm/rubies/ruby-1.9.3-p327/bin/ruby
/Users/amalrik/.rvm/rubies/ruby-1.9.3-p327/bin/ruby:
    /Users/amalrik/.rvm/rubies/ruby-1.9.3-p327/lib/libruby.1.9.1.dylib (compatibility version 1.9.1, current version 1.9.1)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
1.9.3:~
 $ 

I tried use the same solution @maxim did but it didn't work out.
I'm on OSX 10.8.2, rvm 1.17.0 (master), xcode 4.5.2

here is how i tried to reinstall ruby with readline support:

 $ rvm reinstall 1.9.3 --disable-binary --with-readline-dir=$(brew --prefix readline) --reconfigure

the config.log and make.log are here: https://gist.github.com/4194261

I've read a ton of blogs about this issue, but i could not find a solution that works on my box.
Maybe i'm missing something terrible simple, can you please help?

Michal Papis
Collaborator

is it 1.9.2-p320 or 1.9.3-p327 ? the first code block shows both ... can you verify which one is used?

Michal Papis
Collaborator

aha the gist shows you are recompiling 1.9.2 not 1.9.3, please always use latest ruby:

rvm get head
rvm reinstall ruby --disable-binary --with-opt-dir=$(brew --prefix readline)
Amalrik Maia

Hi, thanks for the reply
i tried install another ruby version to be sure if problem wasn't in ruby itself so I put the 1.9.2 config/compile data in the gist by mistake, sorry.

Anyway i tried:

rvm get head
rvm reinstall ruby --disable-binary --with-opt-dir=$(brew --prefix readline)

but now the accented chars are getting ignored. I've updated the gist with the result.

Michal Papis
Collaborator

@amalrik please create a new gist with:

  • find /Users/amalrik/.rvm/rubies/ruby-1.9.3-p327/ -name readline.bundle | xargs otool -L
  • /Users/amalrik/.rvm/src/ruby-1.9.3-p327/ext/readline/mkmf.log
Michal Papis
Collaborator

@amalrik could be incompatibility with /usr/lib/libncurses.5.4.dylib - try also compiling ncurses in homebrew and compiling like this:

rvm reinstall ruby --disable-binary --with-opt-dir="$(brew --prefix readline):$(brew --prefix ncurses)"
Amalrik Maia

Hi
i finally got ncurses installed in homebrew, after that i run:
rvm reinstall ruby --disable-binary --with-opt-dir="$(brew --prefix readline):$(brew --prefix ncurses)"

but didn't solve the problem.
Here are the logs: https://gist.github.com/4237942

what are the files besides readline/mkmf.log, make.log and config.log i should be aware of?
thank you very much for the help.

Michal Papis
Collaborator

I'm curious why the two libs have different root paths?

  • /usr/local/opt/readline/lib/libreadline.6.2.dylib
  • /usr/local/Cellar/ncurses/5.9/lib/libncurses.5.dylib

The rest of libs should not have impact on characters ... at least I think so.

Amalrik Maia

i had problems to install ncurses via homebrew, so i did a workaround:

$ cd /Library/Caches/Homebrew/
$ tar zxvf ncurses-5.9.tar.gz
$ cd ncurses-5.9
$ ./configure --disable-debug --prefix=/usr/local/Cellar/ncurses/5.9 --mandir=/usr/local/Cellar/ncurses/5.9/share/man --with-shared --with-widec --with-manpage-format=normal --enable-symlinks 
$ make
$ make install
$ brew ln ncurses

Homebrew/homebrew-dupes#88

i think this is why the the ncurses lib has a different root path.

Michal Papis
Collaborator

@amalrik try:

$ cd /Library/Caches/Homebrew/
$ tar zxvf ncurses-5.9.tar.gz
$ cd ncurses-5.9
$ ./configure --disable-debug --prefix=/usr/local/Cellar/ncurses/5.9 --mandir=/usr/local/Cellar/ncurses/5.9/share/man --with-shared --with-manpage-format=normal --enable-symlinks 
$ make
$ make install
$ make clean
$ ./configure --disable-debug --prefix=/usr/local/Cellar/ncursesw/5.9 --mandir=/usr/local/Cellar/ncursesw/5.9/share/man --with-shared --with-widec --with-manpage-format=normal --enable-symlinks 
$ make
$ make install
$ brew ln ncursesw

and then:

rvm get head
rvm reinstall ruby --disable-binary --with-opt-dir="$(brew --prefix readline):$(brew --prefix ncurses):$(brew --prefix ncursesw)"
Amalrik Maia

after brew ln ncursesw
homebrew is missing the ncursesw formula:

$ brew --prefix ncursesw
Error: No available formula for ncursesw 

so i run:

rvm get head
$ rvm reinstall 1.9.3 --disable-binary --with-opt-dir="$(brew --prefix readline):$(brew --prefix ncurses):/usr/local/opt/ncursesw"

unfortunately i didn't work.
after install ncursesw my vim installation got broken, i guess i need to reinstall vim, but i make/install ncurses again and got vim working again.

Michal Papis
Collaborator

@amalrik please get + install osx-gcc-installer, and re run the process with:

CC=/usr/bin/gcc-4.2 rvm reinstall 1.9.3
Amalrik Maia

@mpapis
i have gcc installed in homebrew so i did:

CC=/usr/local/bin/gcc-4.2 rvm reinstall 1.9.3 --disable-binary --with-opt-dir="$(brew --prefix readline):$(brew --prefix ncurses)"

didn't work :(

Michal Papis
Collaborator

now please install osx-gcc-installer - it's a complete set of libraries and tools, it seams to be the most feature complete, using gcc from homebrew might be not enough ... also have you thought of using http://railsinstaller.org - I have build it with RVM and all the required libs included - it might be the solution (you would have to remove user rvm: rm -rf ~/.rvm*)

Amalrik Maia

I just read that install osx-gcc-installer is not recommended if you have Xcode which occurs to be my case.
I use Xcode as objective-c environment, so it seems have a tradeoff situation here.
I going to research more if there is there is a way to solve my problem and maintain Xcode, then i'll come back here to tell.

Another possible solution may be setup a linux box with vagrant.

anyway thank you very much so far.
May the Force be with you @mpapis

Michal Papis
Collaborator

@amalrik if you need Xcode then:

1) remove Xcode and command line tools
2) install osx-gcc-installer
3) install Xcode
4) install command line tools

This flow will make it all work smoothly - without any troubles - the "not recommended" applies only to overwriting Xcode with osx-gcc-installer.

Maxim Chernyak aka hakunin

I know this is an old issue, but I just wanted to add this info. What I wrote in the issue worked for me. After that I installed ruby many times, and didn't care about cyrillic. For a long time I've been using 1.9.3-p194 with the same cyrillic problem in irb, and just didn't pay attention.

Today I wanted update ruby, and fix the cyrillic in irb issue again, so I did the usual.

$ rvm get latest
$ rvm install 1.9.3
$ rvm use 1.9.3
$ irb
>> \U+FFD1\U+FFD0\U+FFB5\U+FFD1\U+FFD1

Didn't work. I thought ok - the flag is already fixed so.

$ brew info readline
readline: stable 6.2.4
$ rvm reinstall 1.9.3 --with-readline-dir=/usr/local/Cellar/readline/6.2.4
$irb
>> \U+FFD1\U+FFD0\U+FFB5\U+FFD1\U+FFD1

Same thing.

So I updated XCode, and tried installing ncurses via brew. That failed with 2 errors. I didn't spend time investigating that, and instead tried my original solution.

LDFLAGS=-L/usr/local/Cellar/readline/6.2.4/lib CFLAGS=-I/usr/local/Cellar/readline/6.2.4/include rvm reinstall 1.9.3

This worked again.

Maxim Chernyak aka hakunin

I also wanted to mention that when I ran rvm get latest it warned me that my $PATH wasn't included in my PATH=. Could this be a problem?

Michal Papis
Collaborator

@maxim open a new ticket so we can track the problem properly

Maxim Chernyak aka hakunin

Added issue #1518

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.