Cookies serializer improvements

[chancancode]

• Since the option actually changes how encrypted cookies are serialized, we should name it properly and include the signed cookie jar as well.
• Better tests. Right now the test coverage is pretty sparse. Need to better test the existing cookie features and make sure they work with both of the built-in serializers.
• Fix flash. (Return hash with_indifferent_access on Session::JsonSerializer #13910, JSON session serializer broke flash notice #13944)
• Find out what else is broken by swapping the default serializers, then fix those.
• Add HybridSerializer (need a better name): subclass JsonSerializer to transparently migrate existing marshalled cookies into JSON cookies (we can detect the marshal signiture "\x04\x08" and branch accordingly).
• Update docs, guides, changelogs to reflect this. Also mention the restrictions on what can be reliably stored in the serialized jars.
• Upgrade guides on migrating existing cookies.

[chancancode]

cc @guilleiguaran @NZKoz @rafaelfranca

[huoxito]

Hey guys just a reminder, I don't think it's only flash messages that are broken (not accessible via symbols). But everything on the session object we call on controllers for example. Turning that into hwia should fix it I guess? Another place I thought we could call it is here

rails/activesupport/lib/active_support/message_encryptor.rb

Line 93 in f92142d

@serializer.load(decrypted_data)

[chancancode]

@huoxito I think the session object is already behaving like a HWIA by calling to_s on its [] and []= method, but the overall caveat of this change is that you should not expect to be able to put arbitrary objects into the signed/encrypted cookie jars and expect them to come out exactly the same.

We should obviously fix the parts that Rails uses internally (flash, etc), but if you do session[:today] = Date.today, it will come out as a String unless you use the Marshal serializer.

We'll make sure this is documented by the end of this.

[lukesarnacki]

Thanks @chancancode, this is awesome insight.

When it comes to naming, maybe use special name for current JSON serializer, something like :safe_json (or other name other name connected to being safe / simple etc.).

I will try to find out other things that could be affected by serializer changes.

[NZKoz]

I like the idea of a hybrid serializer, 4.2 could default to that then 4.NEXT could switch to the pure json serializer. The hybrid serializer could take extra care to raise informative exceptions when people put in non-roundtrippable data structures.

[chancancode]

I thought about that, but that would still be a (subtle) breaking change, wouldn't it? (Because people might depends on non-primitives type in their apps.)

I like the idea of more aggressively pushing this, but I am not sure if we can make it the default for existing apps. Once you switched to the hybrid serializer, it's kind of a one way street. 
—
Sent from Mailbox for iPhone

On Tue, Feb 4, 2014 at 12:29 PM, Michael Koziarski
notifications@github.com wrote:

I like the idea of a hybrid serializer, 4.2 could default to that then 4.NEXT could switch to the pure json serializer. The hybrid serializer could take extra care to raise informative exceptions when people put in non-roundtrippable data structures.

Reply to this email directly or view it on GitHub:
#13945 (comment)

[vajrasky]

s/signiture/signature

[chancancode]

Alright, I think this is pretty much done 😄 The "Find out what else is broken by swapping the default serializers, then fix those." TODO is still open, because a lot of the tests are testing against the marshal serializer with hardcoded Marshal dumps =/

We should improve that going forward (because it means that a lot of tests aren't covering the JSON serializer – which became the new default now), but maybe we can probably do it post 4.1?

[NZKoz]

Why not have an explicit hybrid serializer and keep the \x04 stuff in there? have deserialize support both and serialize write json?

[chancancode]

It was changed in ead947a to enable automatica write-back.

Previously, I was passing a HybridSerializer downstream to the Message{Encryptor,Verifier}, however, if we want to actually migrate the cookie value on read (i.e. if I access cookies.signed[:some_key] it will re-write it as json), then it has to be done on the cookie jar level, because it needs access to []= on the cookie jar.

That is mainly to mirror the behaviour on the legacy cookie jar migration thing we did from 3.2 -> 4.0, and to more aggressively migrate legacy cookies.

If we don't care for that, it could be much simpler.

[chancancode]

For reference, this is the before code:

rails/actionpack/lib/action_dispatch/middleware/cookies.rb

Lines 387 to 397 in fafe8ec

	class HybridSerializer < JsonSerializer
	MARSHAL_SIGNATURE = "\x04\x08".freeze
	def self.load(value)
	if value.start_with?(MARSHAL_SIGNATURE)
	Marshal.load(value)
	else
	super
	end
	end
	end

[NZKoz]

I see, if you can't piggyback on @set_cookies in the cookie jar then I guess this is the best way to handle it, does seem a bit janky is all

[chancancode]

Yeah, it's definitely a bit of a hack. I tried to come up with better ways to implement this, but I couldn't =/ The main problem is that if the deserialization is happening downstream, the cookie jar has no insight on whether a migration is necessary or not. If there are any suggestions over this I'd love to hear them 😄

[guilleiguaran]

This is in 4-1-stable branch already and 4.1.0.rc1 was created from 4-1-stable branch 😁

what problem are you seeing?

[jcoyne]

@guilleiguaran Nevermind. I removed my earlier comment. I was storing a HWIA in the session. Apparently I can no longer do that with the default JSON serializer.