Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Basic Rail App fails when editing coffee file (code converter not found (UTF-8 to UTF-8)) #2427

Closed
richmolj opened this Issue · 51 comments

5 participants

@richmolj
  1. Use MRI
  2. Start up a new rails app with dummy controller
  3. Edit dummy controller's coffee file
  4. Start the server, hit the page.
  5. Kill server, switch to Rubinius
  6. Page should still load OK at this point
  7. Edit the coffee file, refresh the page

You should get the error "code converter not found (UTF-8 to UTF-8)". Full trace.

Mountain Lion
rubinius 2.0.0.rc1 (1.9.3 7058a0c yyyy-mm-dd JI) [x86_64-apple-darwin12.3.0]

Related Issues:
#2071
#2052

@dbussink
Owner

Seems like the full trace link is broken, could you get is the right link to the full trace?

@richmolj

Whoops, fixed.

@dbussink
Owner

Does the problem happen btw if you only use Rubinius and not switch halfway in the process?

@richmolj

Actually, yes. I was assuming the switch was causing the error, but it looks like Rubinius will not currently work with the default rails asset pipeline. Rails 3 or 4, FWIW.

Edited the title to reflect the issue.

@dbussink
Owner

There actually have been some encoding issues fixed since 7058a0c, so does the problem still occur on current master?

@richmolj

Yep, still occurring on 5aa808b

@dbussink
Owner

How are you setting up the dummy controller etc? Please note that while a lot of Rails terminology and commands etc. might come natural to you, they often aren't for us. Could you improve the instructions for reproducing it with step by step instructions?

Please assume we have exactly 0 knowledge of Rails and our only capabilities are exactly typing what you tell us. This helps greatly in how quick we're able to reproduce and fix the issue.

@dbussink
Owner

I've just tried this with an app just using Rubinius and it seems to work fine. I probably did something slightly different, so detailed instructions are probably necessary to reproduce this issue.

@richmolj
  1. rails new rbx_test
  2. cd rbx_test
  3. rails g controller welcome
  4. in config/routes.rb, uncomment root 'welcome#index'
  5. Add def index;end to the class in app/controllers/welcome_controller.rb
  6. touch app/views/index.html.erb
  7. rails s
  8. visit localhost:3000

You should now get the error. On rails 4 it will be on

<%= javascript_include_tag "application", "data-turbolinks-track" => true %>

On rails 3 it will be basically the same thing, with the same trace, the only exception is it tries to compile your welcome.js.coffee and fails, on rails 4 it tries to compile data-turbolinks-trace.js.coffee and fails. I only point this out since that's not a local file I'm editing, just coming from vanilla rails.

@dbussink
Owner

Just did exactly this with Rails 4, but it works fine here for me. Maybe it's an idea to see if we can set this up somewhere on a virtual server to reproduce this? Since apparently it depends on something specific in your environment, since it works fine here.

@richmolj

I just tried to reproduce on an Ubuntu VM and everything worked fine. Are we both on Mountain Lion? Is there any other introspection I could give you? I'm not doing anything whacky with my setup.

@dbussink
Owner

I'm also using Mountain Lion here as well. What is your LANG environment variable? Mine is:

LANG=en_US.UTF-8
@richmolj

Same here =/

@dbussink
Owner

I assume you already tried reinstalling Rubinius?

@dbussink
Owner

Also no other environment variables set relating to ruby? RUBYOPT etc.?

@richmolj

Yeah I did a reinstall to get the latest master. No other environment variables set, my .bashrc et al are very thin and not doing anything particular to ruby.

@dbussink
Owner

How strange, all the same here as well. What happens if you clone the Rubinius repository and run all the specs? Do they pass for you?

@richmolj

Possible, I'm just doing this wrong, but when I clone Rubinius and run ./configure, then rake, I get this error: https://gist.github.com/richmolj/5950484

@dbussink
Owner

Could you disable rvm with rvm system so it doesn't interfere with anything? After that you probably have to rerun configure and then rake

@richmolj

Same basic thing (this is with rake a--trace): https://gist.github.com/richmolj/5950625

@dbussink
Owner

You have something strange setup on your system. As you can see, it is compiling extensions for 1.8 mode, but loading the 1.9 headers. This causes the problem there. Did you specify any ./configure parameters or do you have RBXOPT set perhaps?

@richmolj

No configure parameters or RBXOPT. The only thing my bashrc and bash_profile are doing is setting up rvm and some aliases, so I'm really confused what it could be.

FWIW, installed rbx ruby 2.0 (through rvm) and same problem

@dbussink
Owner

Could you make sure to disable RVM completely when working on this? It sets up environment variables etc. and that might cause issues. We really have to remove as much components as possible when investigating this problem.

@richmolj

I disabled it (even removed from bashrc and bash_profile) when trying to compile Rubinius, and got the trace you see above. Not sure how I can disable it and still use Rubinius since I can't install directly. Does trying rbenv buy anything? Would seeing my env list help?

@dbussink
Owner

You don't need RVM in any way to install Rubinius if that is what you mean. You can compile Rubinius with the default Ruby available on Mountain Lion, that should work fine. As a matter of fact, that's what I always do and I never have any problems with that.

Can you show what config.rb looks like when building without RVM?

@richmolj

For clarification, I meant building Rubinius from source seems to fail without going through rvm (see last gist). Installs fine through rvm.

This is the config.rb: https://gist.github.com/richmolj/5950972

@dbussink
Owner

Could you also gist what env shows?

@richmolj

https://gist.github.com/richmolj/5951059

__CF_USER_TEXT_ENCODING popped out to me, but I get the same error after unsetting it.

@dbussink
Owner

Nothing weird there either. I'm really baffled as to what is causing this and honestly have no idea how to further investigate it. The problem you're having with building could be related to the encoding issues if the version RVM builds also has problems with mixing 1.8 and 1.9 modes.

As a last thing, could you gist a build without RVM, of the ./configure and rake step as well? Without cutting any terminal output like before so it's all complete?

@richmolj

https://gist.github.com/richmolj/5951176

I have no idea what it could be since my setup is pretty bare-bones. Thanks for your help anyway.

@dbussink
Owner

Oh, one thing, did you do a rake clean after the initial attempt with RVM enabled? Since it might have stored a Makefile then with the wrong settings for that attempt.

@richmolj

Ok! I did not. When I did, it built correctly and all specs passed.

@dbussink
Owner

Ah ok, that explains. What you could try is configure 1.9 as the default and add the bin directory to your path for the test app.

That way you can run it with this Rubinius build to see if the original problem still happens with this build, or that it is still a problem then.

@richmolj

Unfortunately, still a problem. I removed rvm from bashrc and bash_profile, but the newly-built rubinius in my path, use RBXOPT to set it to 1.9, installed bundler/rails, then did the usual steps. Ugh.

@dbussink
Owner

You could try to nuke all the cached compiled files, by removing the these directories:

~/.rbx
~/path/to/your/app/.rbx

Maybe removing those will help fixing the issue?

@richmolj

Same issue. FWIW I don't have a .rbx within my app.

@dbussink
Owner

That's weird there is no .rbx directory in the app, if you would run it with Rubinius it would likely create one. Also since it worked fine for you on Linux, I'm kinda out of ideas here... :(

@richmolj

Thanks anyway. The thing that gets me is that, in my real-world app, I can develop on MRI, make js edits, no problem. I can then switch to Rubinius and continue to have no problems. It's only once I make an additional edit the error starts. This seems nuts since I believe the coffee files are compiled each request in development mode.

@dbussink
Owner

Yeah, also that I've been running the steps you described and didn't have any issue with those and could just edit stuff and run it without any issues.

@richmolj

Ok, so I completely wiped my Mac and reinstalled OSX. The only things I've done:

  1. Xcode (+ command-line tools)
  2. Xquartz
  3. Homebrew (to install libyaml)
  4. Install Rubinius from source
  5. Install rails
  6. The prior steps noted in this.

So, no edits to bash files, no RVM, etc etc. This was a complete wipe and reinstall.

@fxn
fxn commented

:heart:

@richmolj

So I have to think Rubinius just does not work with Rails on OSX, considering a clean install gets this error. I imagine @dbussink has something set that will get it to work, but we don't know what that is or how to track it down.

Is there anything else I can provide you? I am probably going to give up on Rubinius and start setting my Mac back up unless there's anything I can give you while in this clean state.

@dbussink
Owner

Well, the problem is that there are a lot of people using Rails, Rubinius and OS X who don't have this problem. I'm going to try this on another machine with OS X I have to see if it works there.

I'll also ask some other people to try and / or use machines from others I can ask to try. Honestly I don't think they will experience the same issue since we have never encountered it so far.

One thing maybe is to upload your app to github so we can run the exact same code locally. One thing that might be of influence is the JavaScript engine used to generate the coffeescript files.

Really want to thank you for all the effort so far, hopefully we can find a way isolate the problem.

@richmolj

My only thought re: other users is maybe this is a recent regression. I'll try resetting Rubinius back a few hundred commits, building again, and see if the problem still occurs.

I'll also upload my exact rails app, though it's literally rails new foo at this point.

Thanks!

@richmolj

Just tried this on an iMac, OSX 10.8.3. Not a totally clean install, but fairly close. Had no problems. So, FML.

@jc00ke
Owner

@richmolj I'd like to echo @dbussink's praise on trying to troubleshoot this. I don't remember anyone ever wiping their machine and starting from scratch to track down an issue like this. :+1: and thank you!

@richmolj

Amazingly, I have a fix. I haven't had time to look into the exact cause but if I take line 171 of execjs-1.4.0/lib/execjs/external_runtime.rb, and change it:

options[:external_encoding] = 'UTF-8' # @encoding if @encoding

It works. So I saw I was using the Node runtime for execjs (was my default):

Node = ExternalRuntime.new(
  :name        => "Node.js (V8)",
  :command     => ["nodejs", "node"],
  :runner_path => ExecJS.root + "/support/node_runner.js",
  :encoding    => 'UTF-8'
)

This correctly set @encoding on initialization but dropped it by the time sh was called (I don't know exactly where).

So I tried setting a new runtime for execjs via gem 'therubyracer'. I rebundled and no longer have problems.

I then took the second example from execjs's Readme (compiling coffee file) to try and get an isolated bit of code you could reproduce. I executed this in irb outside of my rails app and it still used therubyracer, not Node, as the runtime. So I tried to force it, and now I get this error: https://gist.github.com/richmolj/5963164

So a) use therubyracer to fix b) there is some kind of false-positive Node runtime on a fresh OSX install (I assume this is only true with Rubinius since MRI works, but I still haven't set up anything else) c) once you've installed therubyracer you'll no longer run against b. I assume you probably won't get the error if you've actually installed Node, as well.

@dbussink
Owner

Ah, that is great! I'm going to try tonight or tomorrow with another machine I have and make sure to uninstall any Javascript runtimes etc. to see if I can reproduce it that way on that machine then. Thanks for persisting on this!

@dbussink
Owner

Managed to reproduce the issue here with the suggestions you made. Had to make sure to not have therubyracer gem installed and also had to remove node.js to reproduce it. Thanks for all the help so far!

@dbussink
Owner

So I extracted the issue to the following script:

IO.popen("ls", :internal_encoding => Encoding::UTF_8) { |f| puts f.read }
@leocassarani leocassarani was assigned
@leocassarani
Collaborator

@dbussink: I've fixed the issue with IO.popen. I don't know if you have a repro handy to test the overall Rails problem, but it would be good to know if that's also fixed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.