New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

padrino-cache ArgumentError: invalid byte sequence in UTF-8 #821

Closed
solon opened this Issue Apr 2, 2012 · 14 comments

Comments

Projects
None yet
7 participants
@solon

solon commented Apr 2, 2012

I'm trying to get a Redis cache working with Padrino 0.10.6 and Ruby 1.9.3-p125 on OS X 10.7.3.

I've been getting ArgumentError: invalid byte sequence in UTF-8 errors when the cache is read.

Here is a stripped-down gist to demonstrate the error: https://gist.github.com/2279844

In the above gist, there are two examples of UTF-8 strings (differing by a single character) being assigned to the :response_buffer key in the hash that is passed to Marshal.dump.

When the second example is retrieved from Redis, it contains a non-UTF-8 character that then causes a regexp in ActiveSupport to choke on the input.

I don't even know if this is a Padrino bug, so I'd welcome any pointers.

solon added a commit to solon/padrino-framework that referenced this issue Apr 2, 2012

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Apr 9, 2012

Member

@brian9 thanks so much for this, my only thought is to check performances, since caching (generally) used to scale performances and decrease memory/cpu usage we need to think a bit better about this.

@nesquena, @skade suggestion?

Member

DAddYE commented Apr 9, 2012

@brian9 thanks so much for this, my only thought is to check performances, since caching (generally) used to scale performances and decrease memory/cpu usage we need to think a bit better about this.

@nesquena, @skade suggestion?

@snowyu

This comment has been minimized.

Show comment
Hide comment
@snowyu

snowyu Apr 10, 2012

#563

Try setting in boot.rb

Encoding.default_internal = nil
Encoding.default_external = 'ASCII-8BIT'

But why not the Marshl.dump is UTF-8 always?

snowyu commented Apr 10, 2012

#563

Try setting in boot.rb

Encoding.default_internal = nil
Encoding.default_external = 'ASCII-8BIT'

But why not the Marshl.dump is UTF-8 always?

@dariocravero

This comment has been minimized.

Show comment
Hide comment
@dariocravero

dariocravero Sep 8, 2012

Contributor

Hi guys, is there any resolution on this? @snowyu proposed solution from issue #563 doesn't seem to be working. Thanks :)

Contributor

dariocravero commented Sep 8, 2012

Hi guys, is there any resolution on this? @snowyu proposed solution from issue #563 doesn't seem to be working. Thanks :)

@ethnt

This comment has been minimized.

Show comment
Hide comment
@ethnt

ethnt Oct 16, 2012

Contributor

👍 I'm having trouble with this as well. Is there anything I can do right now to fix my application?

Contributor

ethnt commented Oct 16, 2012

👍 I'm having trouble with this as well. Is there anything I can do right now to fix my application?

@shipstar

This comment has been minimized.

Show comment
Hide comment
@shipstar

shipstar Dec 13, 2012

Are there performance concerns around changing

if code.present?

to

unless code.nil? || code.empty?

Marshal.dump always returns a string, so code should be a string, right? The nil?/empty? check seems to sidestep the UTF-8 issues associated with calling regexes on the marshaled string.

Am I missing something?

shipstar commented Dec 13, 2012

Are there performance concerns around changing

if code.present?

to

unless code.nil? || code.empty?

Marshal.dump always returns a string, so code should be a string, right? The nil?/empty? check seems to sidestep the UTF-8 issues associated with calling regexes on the marshaled string.

Am I missing something?

@ghost ghost assigned DAddYE Dec 15, 2012

@shipstar

This comment has been minimized.

Show comment
Hide comment
@shipstar

shipstar Dec 31, 2012

Another thing worth mentioning: at least for me, this seemed to be specifically related to padrino-cache's use of marshalling. Since I was just storing JSON data, I didn't need object serialization. I used Redis directly and am not experiencing this issue.

shipstar commented Dec 31, 2012

Another thing worth mentioning: at least for me, this seemed to be specifically related to padrino-cache's use of marshalling. Since I was just storing JSON data, I didn't need object serialization. I used Redis directly and am not experiencing this issue.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Mar 10, 2013

Member

Shall we move this off of 0.11.0? I'd like to release soon. Please advise if want to aim to do this before release @DAddYE @dariocravero

Member

nesquena commented Mar 10, 2013

Shall we move this off of 0.11.0? I'd like to release soon. Please advise if want to aim to do this before release @DAddYE @dariocravero

@dariocravero

This comment has been minimized.

Show comment
Hide comment
@dariocravero

dariocravero Mar 11, 2013

Contributor

@DAddYE would know better...

Contributor

dariocravero commented Mar 11, 2013

@DAddYE would know better...

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Mar 16, 2013

Member

@DAddYE Is this still an issue in edge?

Member

nesquena commented Mar 16, 2013

@DAddYE Is this still an issue in edge?

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Mar 17, 2013

Member

@nesquena yesse and also on ruby 2.0

Member

DAddYE commented Mar 17, 2013

@nesquena yesse and also on ruby 2.0

DAddYE added a commit that referenced this issue Mar 17, 2013

Added Cache Parser Strategy. Fix #821
Now different than before the we store data as plain text, otherwise
we bundle also the marshal serializer or you can simply provide yours.

Example:

    Padrino.cache = Padrino::Cache::File.new(...)
    Padrino.cache.parser # => Padrino::Cache::Parser:Plain
    Padrino.cache.parser = :marshal

Or you can write your own:

    require 'oj'
    module PadrinoJSON
      def self.encode(code)
        Oj.dump(code)
      end

      def self.decode(code)
        Oj.load(code)
      end
    end

Finally you can load your strategy:

    Padrino.cache.parser = PadrinoJSON

Or if you prefer do something on the fight you can:

    Padrino.cache.parser = Module.new do
      def self.encode
      end
      def self.decode
      end
    end

//cc @padrino/core-members
@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Mar 17, 2013

Member

Sorry for spam 🙉

Member

DAddYE commented Mar 17, 2013

Sorry for spam 🙉

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Mar 17, 2013

Member

@nesquena, @skade feel free to check and merge

Member

DAddYE commented Mar 17, 2013

@nesquena, @skade feel free to check and merge

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Mar 17, 2013

Member

and @dariocravero and @padrino/core-members that had work on cache 👯

Member

DAddYE commented Mar 17, 2013

and @dariocravero and @padrino/core-members that had work on cache 👯

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Mar 17, 2013

Member

Closed in favor of #1128

Member

nesquena commented Mar 17, 2013

Closed in favor of #1128

@nesquena nesquena closed this Mar 17, 2013

DAddYE added a commit that referenced this issue Mar 18, 2013

Added Cache Parser Strategy. Fix #821
Now different than before the we store data as plain text, otherwise
we bundle also the marshal serializer or you can simply provide yours.

Example:

    Padrino.cache = Padrino::Cache::File.new(...)
    Padrino.cache.parser # => Padrino::Cache::Parser:Plain
    Padrino.cache.parser = :marshal

Or you can write your own:

    require 'oj'
    module PadrinoJSON
      def self.encode(code)
        Oj.dump(code)
      end

      def self.decode(code)
        Oj.load(code)
      end
    end

Finally you can load your strategy:

    Padrino.cache.parser = PadrinoJSON

Or if you prefer do something on the fight you can:

    Padrino.cache.parser = Module.new do
      def self.encode
      end
      def self.decode
      end
    end

//cc @padrino/core-members

WaYdotNET added a commit to WaYdotNET/padrino-framework that referenced this issue Jul 4, 2013

Merge remote-tracking branch 'upstream/master' into super-admin
* upstream/master:
  Reflect slim changes on tests
  Switch to slim as default engine
  Setup correctly all tasks
  Use a consistent name also for minirecord.
  Added bundler/setup to the generated Rakefile.
  [cache] Cleanup old code, add comments and fix test
  Added Cache Parser Strategy. Fix #821
  [WIP] Added a cache parser for #821 with a default now to Plain.
  [helpers] Fix all capture_concat templates to be equal in tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment