Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Encoding::CompatibilityError with Ruby 1.9 + ERB when multi-byte string is used in both template and variable #48

Merged
1 commit merged into from

5 participants

@gimite

Here's script to reproduce the issue:
https://gist.github.com/728823

Here's output of the script with tilt 1.1:

/tmp/template20101205-22819-1ljrpwf:2:in `concat': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
    from /tmp/template20101205-22819-1ljrpwf:2:in `evaluate_source'
    from /usr/local/lib/ruby/gems/1.9.1/gems/tilt-1.1/lib/tilt.rb:254:in `instance_eval'
    from /usr/local/lib/ruby/gems/1.9.1/gems/tilt-1.1/lib/tilt.rb:254:in `evaluate_source'
    from /usr/local/lib/ruby/gems/1.9.1/gems/tilt-1.1/lib/tilt.rb:195:in `evaluate'
    from /usr/local/lib/ruby/gems/1.9.1/gems/tilt-1.1/lib/tilt.rb:128:in `render'
    from test.rb:15:in `<main>'

Issue here is that:

  • ERB#src correctly outputs magic comment e.g. "# coding: utf-8" based on either magic comment in ERB template or encoding of the template string
  • eval() correctly handles magic comment, but it must be in the first line
  • But Tilt add some script (preamble) before the output of ERB#src, so the magic comment no longer works

Solution in this change is to copy the magic comment to the first line.
It looks like #45 solves the same issue, but this patch might be slightly simpler and more general (because it should also work for other template engines which output magic comment if any).

@rkh
Collaborator
rkh commented

Hiroshi: Does this solve the Sinatra issue on Ubuntu?
Ryan: If so, +1 an can we have a new Tilt release some time soon?

@gimite

Yes, I verified it solves the issue of Encoding::CompatibilityError when multi-byte character is used in both controller and view template (if magic comment is correctly written in both file). I haven't checked original issue reported by Quintus (where one template renders another template), but I guess it's fixed too.

@kasperbn

When will this fix be released in a gem?

@alminde

Marvelous! Gem soon, please!

@brockgr

Was just playing with this patch and Sinatra, and it only seems to work for the first render of the ERB. The second time render() is called, the template seems to have reverted to ASCII-8BIT.

I created a patched script to reproduce the issue:
https://gist.github.com/749712

@rkh
Collaborator

Fixed in PR #50.

@alexch alexch referenced this pull request from a commit in alexch/tilt
@rkh rkh Add magic comments for encoding to method compilation, too.
Also, add test for encoding issues with erb.

Fixes #48.
59e04a7
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 4, 2010
  1. @gimite

    Fixing Encoding::CompatibilityError with Ruby 1.9 + ERB, by copying m…

    gimite authored
    …agic comment to the first line so that it works.
This page is out of date. Refresh to see the latest.
Showing with 12 additions and 1 deletion.
  1. +12 −1 lib/tilt.rb
View
13 lib/tilt.rb
@@ -189,9 +189,16 @@ def self.evaluate(scope, locals, &block)
# easier and more appropriate.
def precompiled(locals)
preamble = precompiled_preamble(locals)
+ template = precompiled_template(locals)
+ magic_comment = extract_magic_comment(template)
+ if magic_comment
+ # Magic comment e.g. "# coding: utf-8" has to be in the first line.
+ # So we copy the magic comment to the first line.
+ preamble = magic_comment + "\n" + preamble
+ end
parts = [
preamble,
- precompiled_template(locals),
+ template,
precompiled_postamble(locals)
]
[parts.join("\n"), preamble.count("\n") + 1]
@@ -282,6 +289,10 @@ def unbind_compiled_method(method_name)
method
end
+ def extract_magic_comment(script)
+ script.slice(/\A[ \t]*\#.*coding\s*[=:]\s*([[:alnum:]\-_]+).*$/)
+ end
+
# Special case Ruby 1.9.1's broken yield.
#
# http://github.com/rtomayko/tilt/commit/20c01a5
Something went wrong with that request. Please try again.