Skip to content
This repository

Force form params into UTF-8 encoding when using Ruby 1.9.x #326

Closed
wants to merge 1 commit into from

3 participants

Johannes Fahrenkrug Jim Gay Sam Whited
Johannes Fahrenkrug
Collaborator

This is basically a back port of Rails 3 built-in behavior.

Background: When entering special chars like umlauts in Radiant forms (like the admin page forms), these strings enter Rails as ASCII-8BIT-encoded strings. That causes lots of problems, especially with some DB-backends like DB2.

The solution is to force the encoding of incoming params to UTF-8. While that is not a perfect solution, it's much better than the current situation.

More background:

Rails discussion about this: https://rails.lighthouseapp.com/projects/8994/tickets/4807
@wycats' commit to Rails 3: rails/rails@25215d7
SO discussion: http://stackoverflow.com/questions/8268778/rails-2-3-9-encoding-of-query-parameters
Blog post: http://jasoncodes.com/posts/ruby19-rails2-encodings (thanks for the following code, Jason!)

Let me know what you guys think.

  • Johannes
Jim Gay
Owner

awesome. :+1:

Johannes Fahrenkrug
Collaborator

@saturnflyer You think I should merge it?

Jim Gay
Owner

yup. push it into the version1 branch

Sam Whited
Collaborator

Great idea! This will make a fantastic addition to 1.x

Johannes Fahrenkrug
Collaborator

Thanks' @SamWhited :)

I've merged it into the version1 branch: 667c4dc

Closing the ticket now.

Johannes Fahrenkrug
Collaborator

Maybe it's just taking longer than normal, but Travis doesn't seem to pick up the new commit on the version1 branch: http://travis-ci.org/#!/radiant/radiant/branch_summary

Sam Whited
Collaborator

I've been having a lot of issues with it too lately. Probably just under load.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Mar 14, 2012
Johannes Fahrenkrug jfahrenkrug Form params are forced into UTF-8 encoding when using Ruby 1.9.x (bas…
…ically a back-port of Rails 3 behavior)
e3f15d4
This page is out of date. Refresh to see the latest.
26 app/controllers/application_controller.rb
@@ -11,6 +11,7 @@ class ApplicationController < ActionController::Base
11 11 before_filter :set_timezone
12 12 before_filter :set_user_locale
13 13 before_filter :set_javascripts_and_stylesheets
  14 + before_filter :force_utf8_params if RUBY_VERSION =~ /1\.9/
14 15 before_filter :set_standard_body_style, :only => [:new, :edit, :update, :create]
15 16
16 17 attr_accessor :config, :cache
@@ -84,4 +85,29 @@ def set_standard_body_style
84 85 @body_classes.concat(%w(reversed))
85 86 end
86 87
  88 + # When using Radiant with Ruby 1.9, the strings that come in from forms are ASCII-8BIT encoded.
  89 + # That causes problems, especially when using special chars and with certain DBs, like DB2
  90 + # That's why we force the encoding of the params to UTF-8
  91 + # That's what's happening in Rails 3, too: https://github.com/rails/rails/commit/25215d7285db10e2c04d903f251b791342e4dd6a
  92 + #
  93 + # See http://stackoverflow.com/questions/8268778/rails-2-3-9-encoding-of-query-parameters
  94 + # See https://rails.lighthouseapp.com/projects/8994/tickets/4807
  95 + # See http://jasoncodes.com/posts/ruby19-rails2-encodings (thanks for the following code, Jason!)
  96 + def force_utf8_params
  97 + traverse = lambda do |object, block|
  98 + if object.kind_of?(Hash)
  99 + object.each_value { |o| traverse.call(o, block) }
  100 + elsif object.kind_of?(Array)
  101 + object.each { |o| traverse.call(o, block) }
  102 + else
  103 + block.call(object)
  104 + end
  105 + object
  106 + end
  107 + force_encoding = lambda do |o|
  108 + o.force_encoding(Encoding::UTF_8) if o.respond_to?(:force_encoding)
  109 + end
  110 + traverse.call(params, force_encoding)
  111 + end
  112 +
87 113 end
15 spec/controllers/admin/pages_controller_spec.rb
... ... @@ -1,3 +1,5 @@
  1 +#encoding: utf-8
  2 +
1 3 require File.dirname(__FILE__) + '/../../spec_helper'
2 4
3 5 describe Admin::PagesController do
@@ -291,6 +293,19 @@
291 293 final_updated_at = pages(:home).updated_at
292 294 lambda{ next_updated_at <=> final_updated_at }.should be_true
293 295 end
  296 +
  297 + it 'should convert form input to UTF-8' do
  298 + # When using Radiant with Ruby 1.9, the strings that come in from forms are ASCII-8BIT encoded.
  299 + # That causes problems, especially when using special chars and with certain DBs, like DB2
  300 + #
  301 + # See http://stackoverflow.com/questions/8268778/rails-2-3-9-encoding-of-query-parameters
  302 + # See https://rails.lighthouseapp.com/projects/8994/tickets/4807
  303 + # See http://jasoncodes.com/posts/ruby19-rails2-encodings
  304 +
  305 + put :update, :id => page_id(:home), :page => {:breadcrumb => 'Homepage', :parts_attributes => {'0' => {:id => pages(:home).parts[0].id, :content => 'Ümlautö'.force_encoding('ASCII-8BIT')}}} and sleep(1)
  306 + params['page']['parts_attributes']['0']['content'].encoding.to_s.should == 'UTF-8'
  307 + params['page']['parts_attributes']['0']['content'].should == 'Ümlautö'
  308 + end
294 309 end
295 310
296 311 it "should initialize meta and buttons_partials in edit action" do

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.