Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

padrino-cache ArgumentError: invalid byte sequence in UTF-8 #821

Closed
solon opened this issue Apr 2, 2012 · 14 comments
Closed

padrino-cache ArgumentError: invalid byte sequence in UTF-8 #821

solon opened this issue Apr 2, 2012 · 14 comments
Assignees
Milestone

Comments

@solon
Copy link

solon commented Apr 2, 2012

I'm trying to get a Redis cache working with Padrino 0.10.6 and Ruby 1.9.3-p125 on OS X 10.7.3.

I've been getting ArgumentError: invalid byte sequence in UTF-8 errors when the cache is read.

Here is a stripped-down gist to demonstrate the error: https://gist.github.com/2279844

In the above gist, there are two examples of UTF-8 strings (differing by a single character) being assigned to the :response_buffer key in the hash that is passed to Marshal.dump.

When the second example is retrieved from Redis, it contains a non-UTF-8 character that then causes a regexp in ActiveSupport to choke on the input.

I don't even know if this is a Padrino bug, so I'd welcome any pointers.

solon added a commit to solon/padrino-framework that referenced this issue Apr 2, 2012
@DAddYE
Copy link
Member

DAddYE commented Apr 9, 2012

@brian9 thanks so much for this, my only thought is to check performances, since caching (generally) used to scale performances and decrease memory/cpu usage we need to think a bit better about this.

@nesquena, @skade suggestion?

@snowyu
Copy link

snowyu commented Apr 10, 2012

#563

Try setting in boot.rb

Encoding.default_internal = nil
Encoding.default_external = 'ASCII-8BIT'

But why not the Marshl.dump is UTF-8 always?

@dariocravero
Copy link

Hi guys, is there any resolution on this? @snowyu proposed solution from issue #563 doesn't seem to be working. Thanks :)

@ethnt
Copy link
Contributor

ethnt commented Oct 16, 2012

👍 I'm having trouble with this as well. Is there anything I can do right now to fix my application?

@shipstar
Copy link

Are there performance concerns around changing

if code.present?

to

unless code.nil? || code.empty?

Marshal.dump always returns a string, so code should be a string, right? The nil?/empty? check seems to sidestep the UTF-8 issues associated with calling regexes on the marshaled string.

Am I missing something?

@ghost ghost assigned DAddYE Dec 15, 2012
@shipstar
Copy link

Another thing worth mentioning: at least for me, this seemed to be specifically related to padrino-cache's use of marshalling. Since I was just storing JSON data, I didn't need object serialization. I used Redis directly and am not experiencing this issue.

@nesquena
Copy link
Member

Shall we move this off of 0.11.0? I'd like to release soon. Please advise if want to aim to do this before release @DAddYE @dariocravero

@dariocravero
Copy link

@DAddYE would know better...

@nesquena
Copy link
Member

@DAddYE Is this still an issue in edge?

@DAddYE
Copy link
Member

DAddYE commented Mar 17, 2013

@nesquena yesse and also on ruby 2.0

DAddYE added a commit that referenced this issue Mar 17, 2013
Now different than before the we store data as plain text, otherwise
we bundle also the marshal serializer or you can simply provide yours.

Example:

    Padrino.cache = Padrino::Cache::File.new(...)
    Padrino.cache.parser # => Padrino::Cache::Parser:Plain
    Padrino.cache.parser = :marshal

Or you can write your own:

    require 'oj'
    module PadrinoJSON
      def self.encode(code)
        Oj.dump(code)
      end

      def self.decode(code)
        Oj.load(code)
      end
    end

Finally you can load your strategy:

    Padrino.cache.parser = PadrinoJSON

Or if you prefer do something on the fight you can:

    Padrino.cache.parser = Module.new do
      def self.encode
      end
      def self.decode
      end
    end

//cc @padrino/core-members
@DAddYE
Copy link
Member

DAddYE commented Mar 17, 2013

Sorry for spam 🙉

@DAddYE
Copy link
Member

DAddYE commented Mar 17, 2013

@nesquena, @skade feel free to check and merge

@DAddYE
Copy link
Member

DAddYE commented Mar 17, 2013

and @dariocravero and @padrino/core-members that had work on cache 👯

@nesquena
Copy link
Member

Closed in favor of #1128

DAddYE added a commit that referenced this issue Mar 18, 2013
Now different than before the we store data as plain text, otherwise
we bundle also the marshal serializer or you can simply provide yours.

Example:

    Padrino.cache = Padrino::Cache::File.new(...)
    Padrino.cache.parser # => Padrino::Cache::Parser:Plain
    Padrino.cache.parser = :marshal

Or you can write your own:

    require 'oj'
    module PadrinoJSON
      def self.encode(code)
        Oj.dump(code)
      end

      def self.decode(code)
        Oj.load(code)
      end
    end

Finally you can load your strategy:

    Padrino.cache.parser = PadrinoJSON

Or if you prefer do something on the fight you can:

    Padrino.cache.parser = Module.new do
      def self.encode
      end
      def self.decode
      end
    end

//cc @padrino/core-members
WaYdotNET added a commit to WaYdotNET/padrino-framework that referenced this issue Jul 4, 2013
* upstream/master:
  Reflect slim changes on tests
  Switch to slim as default engine
  Setup correctly all tasks
  Use a consistent name also for minirecord.
  Added bundler/setup to the generated Rakefile.
  [cache] Cleanup old code, add comments and fix test
  Added Cache Parser Strategy. Fix padrino#821
  [WIP] Added a cache parser for padrino#821 with a default now to Plain.
  [helpers] Fix all capture_concat templates to be equal in tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants