Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

[3.1.0.rc1] MultiJson::DecodeError with alternative I18n Backends #1499

Closed
nledez opened this Issue Jun 5, 2011 · 23 comments

Comments

Projects
None yet

nledez commented Jun 5, 2011

Hello,

I try to apply episode 256 from Ryan Bates :
http://railscasts.com/episodes/256-i18n-backends

But with Rails 3.1.0 rc1 but I have this error :
MultiJson::DecodeError in Tickets#index
737: unexpected token at '"Welcome from redis"'

The error expected with Redis but also with :
I18n.backend = I18n::Backend::KeyValue.new({})

Thanks for your help.

for f in Gemfile* config/initializers/i18n_backend.rb app/controllers/translations_controller.rb ; do echo "    ==== $f" ; cat $f | sed 's/^/    /' ; done
==== Gemfile
source 'http://rubygems.org'

gem 'rails', '3.1.0.rc1'

# Bundle edge Rails instead:
# gem 'rails',     :git => 'git://github.com/rails/rails.git'

gem 'sqlite3'

# Asset template engines
gem 'sass'
gem 'coffee-script'
gem 'uglifier'

gem 'jquery-rails'

# Add mongo mapper
gem 'mongo_mapper'
gem 'bson_ext'

# Add redis
gem 'redis'

# Use unicorn as the web server
# gem 'unicorn'

# Deploy with Capistrano
# gem 'capistrano'

# To use debugger
# gem 'ruby-debug19', :require => 'ruby-debug'

group :test do
  # Pretty printed test output
  gem 'turn', :require => false
  gem 'rspec-rails'
  gem 'rb-fsevent', :require => false if RUBY_PLATFORM =~ /darwin/i
  gem 'growl', :require => false if RUBY_PLATFORM =~ /darwin/i
  gem 'guard-rspec'
  gem 'guard-livereload'
  gem 'capybara', :git => 'git://github.com/jnicklas/capybara.git'
  gem 'launchy'
  gem 'database_cleaner'
end
==== Gemfile.lock
GIT
  remote: git://github.com/jnicklas/capybara.git
  revision: e1091daced2513a2d2ce15ae4033d9da1ced9f7e
  specs:
    capybara (1.0.0.beta1)
      mime-types (>= 1.16)
      nokogiri (>= 1.3.3)
      rack (>= 1.0.0)
      rack-test (>= 0.5.4)
      selenium-webdriver (~> 0.2.0)
      xpath (~> 0.1.4)

GEM
  remote: http://rubygems.org/
  specs:
    actionmailer (3.1.0.rc1)
      actionpack (= 3.1.0.rc1)
      mail (~> 2.3.0)
    actionpack (3.1.0.rc1)
      activemodel (= 3.1.0.rc1)
      activesupport (= 3.1.0.rc1)
      builder (~> 3.0.0)
      erubis (~> 2.7.0)
      i18n (~> 0.6.0beta1)
      rack (~> 1.3.0.beta2)
      rack-cache (~> 1.0.1)
      rack-mount (~> 0.8.1)
      rack-test (~> 0.6.0)
      sprockets (~> 2.0.0.beta.5)
      tzinfo (~> 0.3.27)
    activemodel (3.1.0.rc1)
      activesupport (= 3.1.0.rc1)
      bcrypt-ruby (~> 2.1.4)
      builder (~> 3.0.0)
      i18n (~> 0.6.0beta1)
    activerecord (3.1.0.rc1)
      activemodel (= 3.1.0.rc1)
      activesupport (= 3.1.0.rc1)
      arel (~> 2.1.1)
      tzinfo (~> 0.3.27)
    activeresource (3.1.0.rc1)
      activemodel (= 3.1.0.rc1)
      activesupport (= 3.1.0.rc1)
    activesupport (3.1.0.rc1)
      multi_json (~> 1.0)
    addressable (2.2.6)
    ansi (1.2.5)
    arel (2.1.1)
    bcrypt-ruby (2.1.4)
    bson (1.3.1)
    bson_ext (1.3.1)
    builder (3.0.0)
    childprocess (0.1.9)
      ffi (~> 1.0.6)
    coffee-script (2.2.0)
      coffee-script-source
      execjs
    coffee-script-source (1.1.1)
    configuration (1.2.0)
    database_cleaner (0.6.7)
    diff-lcs (1.1.2)
    em-websocket (0.2.1)
      addressable (>= 2.1.1)
      eventmachine (>= 0.12.9)
    erubis (2.7.0)
    eventmachine (0.12.10)
    execjs (1.1.0)
      multi_json (~> 1.0)
    ffi (1.0.9)
    growl (1.0.3)
    guard (0.3.4)
      thor (~> 0.14.6)
    guard-livereload (0.1.11)
      em-websocket (~> 0.2.0)
      guard (>= 0.2.2)
      json (~> 1.5.1)
    guard-rspec (0.3.1)
      guard (>= 0.2.2)
    hike (1.0.0)
    i18n (0.6.0)
    jquery-rails (1.0.9)
      railties (~> 3.0)
      thor (~> 0.14)
    json (1.5.1)
    json_pure (1.5.1)
    launchy (0.4.0)
      configuration (>= 0.0.5)
      rake (>= 0.8.1)
    mail (2.3.0)
      i18n (>= 0.4.0)
      mime-types (~> 1.16)
      treetop (~> 1.4.8)
    mime-types (1.16)
    mongo (1.3.1)
      bson (>= 1.3.1)
    mongo_mapper (0.9.1)
      activemodel (~> 3.0)
      activesupport (~> 3.0)
      plucky (~> 0.3.8)
    multi_json (1.0.3)
    nokogiri (1.4.4)
    plucky (0.3.8)
      mongo (~> 1.3)
    polyglot (0.3.1)
    rack (1.3.0)
    rack-cache (1.0.2)
      rack (>= 0.4)
    rack-mount (0.8.1)
      rack (>= 1.0.0)
    rack-ssl (1.3.2)
      rack
    rack-test (0.6.0)
      rack (>= 1.0)
    rails (3.1.0.rc1)
      actionmailer (= 3.1.0.rc1)
      actionpack (= 3.1.0.rc1)
      activerecord (= 3.1.0.rc1)
      activeresource (= 3.1.0.rc1)
      activesupport (= 3.1.0.rc1)
      bundler (~> 1.0)
      railties (= 3.1.0.rc1)
    railties (3.1.0.rc1)
      actionpack (= 3.1.0.rc1)
      activesupport (= 3.1.0.rc1)
      rack-ssl (~> 1.3.2)
      rake (>= 0.8.7)
      thor (~> 0.14.6)
    rake (0.9.1)
    rb-fsevent (0.4.0)
    redis (2.2.0)
    rspec (2.6.0)
      rspec-core (~> 2.6.0)
      rspec-expectations (~> 2.6.0)
      rspec-mocks (~> 2.6.0)
    rspec-core (2.6.3)
    rspec-expectations (2.6.0)
      diff-lcs (~> 1.1.2)
    rspec-mocks (2.6.0)
    rspec-rails (2.6.1)
      actionpack (~> 3.0)
      activesupport (~> 3.0)
      railties (~> 3.0)
      rspec (~> 2.6.0)
    rubyzip (0.9.4)
    sass (3.1.2)
    selenium-webdriver (0.2.1)
      childprocess (>= 0.1.7)
      ffi (>= 1.0.7)
      json_pure
      rubyzip
    sprockets (2.0.0.beta.9)
      hike (~> 1.0)
      rack (~> 1.0)
      tilt (!= 1.3.0, ~> 1.1)
    sqlite3 (1.3.3)
    thor (0.14.6)
    tilt (1.3.2)
    treetop (1.4.9)
      polyglot (>= 0.3.1)
    turn (0.8.2)
      ansi (>= 1.2.2)
    tzinfo (0.3.27)
    uglifier (0.5.4)
      execjs (>= 0.3.0)
      multi_json (>= 1.0.2)
    xpath (0.1.4)
      nokogiri (~> 1.3)

PLATFORMS
  ruby

DEPENDENCIES
  bson_ext
  capybara!
  coffee-script
  database_cleaner
  growl
  guard-livereload
  guard-rspec
  jquery-rails
  launchy
  mongo_mapper
  rails (= 3.1.0.rc1)
  rb-fsevent
  redis
  rspec-rails
  sass
  sqlite3
  turn
  uglifier
==== config/initializers/i18n_backend.rb
TRANSLATION_STORE = Redis.new  
I18n.backend = I18n::Backend::Chain.new(I18n::Backend::KeyValue.new(TRANSLATION_STORE), I18n.backend)
==== app/controllers/translations_controller.rb
class TranslationsController < ApplicationController
  def index
    @translations = TRANSLATION_STORE
  end

  def create
    I18n.backend.store_translations(params[:locale], {params[:key] => params[:value]}, :escape => false)
    redirect_to translations_url, :notice => "Added translations"
  end
end
Contributor

slbug commented Jun 6, 2011

I had to change this line https://github.com/svenfuchs/i18n/blob/master/lib/i18n/backend/key_value.rb#L76 to

@store[key] = ActiveSupport::JSON.encode([value]) unless value.is_a?(Symbol)

And https://github.com/svenfuchs/i18n/blob/master/lib/i18n/backend/key_value.rb#L93 to

value = ActiveSupport::JSON.decode(value)[0] if value

I don't know why, but there is somethong wrong with decoding plain text.

nledez commented Jun 11, 2011

Yes ! Many thanks.

Bellow the monkey patch :

diff --git a/config/environment.rb b/config/environment.rb
index bb2ed67..025e04f 100644
--- a/config/environment.rb
+++ b/config/environment.rb
@@ -1,5 +1,43 @@
 # Load the rails application
 require File.expand_path('../application', __FILE__)

+module I18n
+  module Backend
+    class KeyValue
+      module Implementation
+        def store_translations(locale, data, options = {})
+          escape = options.fetch(:escape, true)
+          flatten_translations(locale, data, escape, @subtrees).each do |key, value|
+            key = "#{locale}.#{key}"
+
+            case value
+            when Hash
+              if @subtrees && (old_value = @store[key])
+                old_value = ActiveSupport::JSON.decode(old_value)
+                value = old_value.deep_symbolize_keys.deep_merge!(value) if old_value.is_a?(Hash)
+              end
+            when Proc
+              raise "Key-value stores cannot handle procs"
+            end
+
+            #@store[key] = ActiveSupport::JSON.encode(value) unless value.is_a?(Symbol)
+            @store[key] = ActiveSupport::JSON.encode([value]) unless value.is_a?(Symbol)
+          end
+        end
+
+      protected
+
+        def lookup(locale, key, scope = [], options = {})
+          key   = normalize_flat_keys(locale, key, scope, options[:separator])
+          value = @store["#{locale}.#{key}"]
+          #value = ActiveSupport::JSON.decode(value) if value
+          value = ActiveSupport::JSON.decode(value)[0] if value
+          value.is_a?(Hash) ? value.deep_symbolize_keys : value
+        end
+      end
+    end
+  end
+end
+
 # Initialize the rails application
 Tickets::Application.initialize!
Contributor

dmathieu commented Jun 11, 2011

Could you make a pull request ?

Owner

guilleiguaran commented Jun 11, 2011

Looks like an error in i18n, @svenfuchs can you take a look to this?

Contributor

svenfuchs commented Jun 11, 2011

folks, thanks for tracking this down!

i'll ask @josevalim to look into this as i have no clue about the keyvalue backend :)

Contributor

slbug commented Jun 11, 2011

My thoughts:

JSON should decode simple text value, but some person says that this is incorrect (e.g. JSON should work only with objects). So there s 2 ways to fix it:

  1. Make json library to work correctly with strings
  2. Transparently convert string to object (e.g. array).

first looks way better for me.

Contributor

josevalim commented Jun 11, 2011

I believe simply the string is not valid JSON, right? So wrapping it in an array in the backend seems like the proper solution.

Contributor

slbug commented Jun 11, 2011

@josevalim, some json implementations works correctly with encoded strings (e.g "foo bar"). it decodes to simple string. just drops quotes

Contributor

slbug commented Jun 11, 2011

Even multi_json right now encodes string (for example "foo bar") to ""foo bar"", but can't decode back.

nledez commented Jun 11, 2011

If put data in a array is a good solution I made a pull request :
svenfuchs/i18n#97

jake3030 pushed a commit to jake3030/rails that referenced this issue Jun 28, 2011

Ensure Model#last doesn't affects order for another finders inside th…
…e same scope [#1499 state:resolved]

Signed-off-by: Pratik Naik <pratiknaik@gmail.com>
Contributor

Mik-die commented Jul 18, 2011

It's json gem problem, with other engines all works fine. I install gem 'yajl-ruby' instead 'json' and have no more trouble with MultiJson::DecodeError

Contributor

carlesso commented Aug 31, 2011

It's related to flori/json#84, and yajl-ruby seems not working as workaround with rails >= 3.1.0.rc6 (no, not even with fresh new 3.1)

rweng commented Sep 14, 2011

is there any solution to this available?

@rweng
temporary solution: see nledez's monkey patch above.
long term solution: JSON.parse("a".to_json) needs to be addressed (see carlesso's entry on flori/json#84)

Pajk commented Oct 1, 2011

Install yajl-ruby and put in your config/application.rb:

require 'yajl/json_gem'

And everything is working again, no monkey patching needed:).
Tested with rails 3.1 and Redis backend.

houen commented Oct 20, 2011

That fixed it! Wonderful Pajk!

Pajk!

You just made my day! Wasted close to 10hrs trying to work around this MultiJson::DecodeError and being close to going back to good ol' yml-files, you came miraculously into my "google-radar"

Perfect!

ralibi commented Apr 11, 2012

thx pajk. your solution works like a charm. well done.

Sorry, can't get this to work, I must be missing something obvious here...

ruby 1.9.3p125 (2012-02-16) [i386-mingw32], rails 3.2.1

From a simple tutorial:

rails new demo
cd demo
rails g controller Hello index
rails s
0.0.0.0:3000/Hello

MultiJson::DecodeError in Hello#index
C:/Sites/dem/app/views/layouts/application.html.erb where line #6 raised:
<%= javascript_include_tag "application" %>

So... installed yajl-ruby;
changed gemfile to include: gem 'yajl-ruby'
changed config\application.rb to include: require 'yajl/json_gem'
removed json from gemfile.lock

New bundle install shows using yajl and not json gem.

Repeat steps as above:
rails s
0.0.0.0:3000/Hello

Gives the same error message as before, at the same location:
<%= javascript_include_tag "application" %>

Do I need to remove other dependencies?

Contributor

bmaland commented Apr 23, 2012

Are there any quick solutions for this on JRuby, where yajl-ruby isn't an option?

Contributor

ankit8898 commented Apr 28, 2012

@Pajk it woked like a charm.. thanks!

Owner

rafaelfranca commented Apr 28, 2012

@svenfuchs @josevalim can we close this one?

Member

arunagw commented Aug 21, 2012

Closing this now. Please reopen if issue still exists.

Cheers,
Arun

@arunagw arunagw closed this Aug 21, 2012

@houen houen referenced this issue in flori/json Sep 28, 2012

Closed

JSON.parse("a".to_json) fails! #84

jay3126 pushed a commit to jay3126/i18n-custom that referenced this issue Mar 10, 2014

Merge pull request #97 from nledez/master
Fix to close issue rails/rails#1499 i18n with key/value backend

javierv added a commit to javierv/41cero10asesores that referenced this issue Nov 11, 2017

Rodeo para hacer funcionar Redis con Rails 3.1.
Fallo debido al nuevo manejo de JSON. Ver: rails/rails#1499
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment