Add expiry metadata to Cookies and freshen expires option to support duration #30121

assain · 2017-08-07T19:11:16Z

This PR adds:

Expiry meta data to signed/encrypted cookies.
Duration support to :expires option, i.e. you can specify when the cookie should expire relatively.
e.g. cookies.signed[:user_name] = { value: "bob", expires: 2.hours }

rails-bot · 2017-08-07T19:11:19Z

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @sgrif (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

assain · 2017-08-07T19:16:40Z

actionpack/lib/action_dispatch/middleware/cookies.rb

+          elsif expires_in
+            options[:expires] = expires_in.from_now
+          end
+        end


@kaspth

Rack::Utils.add_cookie_to_header uses the :expires option to set the expiration date, so I decided to set expiration using it.

This makes sense to me. The expiration set for a cookie can match the expiration for the underlying Verified/Encrypted message. This would definitely mitigate the "valid forever" cookie problem as the messages will now have their own expiration builtin. 👍

assain · 2017-08-07T19:18:27Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -569,7 +580,7 @@ def parse(name, signed_message)
        end

        def commit(options)
-          options[:value] = @verifier.generate(serialize(options[:value]))
+          options[:value] = @verifier.generate(serialize(options[:value]), expires_at: options[:expires_at], expires_in: options[:expires_in])



@kaspth

Currently, just the value gets serialized not the wrapped message with metadata since it uses NullSerailizer

@kaspth 😄
Does the following sound reasonable?
Set the verifier to use the serializer used by cookies instead of NullSerializer, this way the wrapped message could be serialized with the cookie_serializer.

To preserve backwards compatibility we could have a switch for it.

The verifier/encryptor is unaware of the serialization for cookies since custom serializers could be used for cookies. This method here is where a completely custom serializer could come into play: https://github.com/assain/rails/blob/931257defaecd16212d3b6853d5ca2177f8d0700/actionpack/lib/action_dispatch/middleware/cookies.rb#L547-L557

mjc-gh · 2017-08-09T14:50:44Z

actionpack/test/dispatch/cookies_test.rb

+  def test_cookie_expiration_using_expires_in
+    freeze_time do
+      get :set_expiration_using_expires_in
+      assert_cookie_header "user_name=assain; path=/; expires=#{1.hour.from_now.rfc2822}"


What about timezones here in this test case? Should the 1.hour.from_now be converted to UTC first?

Not liking the dynamic interpolated time here. We should use a fixed time like the other tests do. E.g. travel_to Time.new(2017, …) do

matthewd · 2017-08-09T16:41:20Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -458,6 +460,15 @@ def make_set_cookie_header(header)
        def write_cookie?(cookie)
          request.ssl? || !cookie[:secure] || always_write_cookie
        end
+
+        def set_expires_attribute(options)
+          expires_at, expires_in = options[:expires_at], options[:expires_in]


These should probably delete.

Might be worth asserting that only one of :expires, :expires_at, :expires_in is set?

I do think that, even if we add these friendlier aliases, :expires should remain fully supported. In fact, alternative API suggestion: only use :expires, but make both expires: DateTime and expires: Duration DWIM.

only use :expires, but make both expires: DateTime and expires: Duration DWIM.

Ah yeah, let's just go with that here 👍

Then we only have to convert expires: 1.hour into an absolute time via options[:expires] = options[:expires].from_now if options[:expires].respond_to?(:from_now).

And pass it in to the verifier/encryptor via expires_at: options[:expires].

kaspth · 2017-08-10T19:27:49Z

actionpack/test/dispatch/cookies_test.rb

+    end
+
+    def set_expiration_using_expires_at
+      cookies["user_name"] = { value: "assain", expires_at: Time.now.advance(years: 1)}


Needs a space before }

kaspth · 2017-08-10T19:27:49Z

actionpack/test/dispatch/cookies_test.rb

+
+    def set_signed_cookie_expiration_using_expires_in
+      cookies.signed["user_name"] = { value: "assain", expires_in: 1.hour }
+      head :ok


Don't controllers respond with head :no_content by default? So that these head lines can be omitted?

When switching to expires supporting both durations and time we only need to test signed cookies and how their expiry is enforced.

E.g. do as this test but add travel_to <after_expiry> and assert_nil on the signed cookie.

kaspth · 2017-08-10T19:27:49Z

actionpack/test/dispatch/cookies_test.rb

+  def test_cookie_expiration_using_expires_in
+    freeze_time do
+      get :set_expiration_using_expires_in
+      assert_cookie_header "user_name=assain; path=/; expires=#{1.hour.from_now.rfc2822}"


Not liking the dynamic interpolated time here. We should use a fixed time like the other tests do. E.g. travel_to Time.new(2017, …) do

assain · 2017-08-14T14:57:19Z

actionpack/test/dispatch/cookies_test.rb

+
+      expected_time = 2.hours.from_now.utc.rfc2822
+      assert_cookie_header "user_name=assain; path=/; expires=#{expected_time}"
+    end


@kaspth 😄
Previously you'd admonished that interpolating was a bad idea. However, I couldn't figure out the best way to test this yet 😅

Don't use freeze_time then. Maybe travel_to Time.parse("some httpdate formatted time with timezone"), then put that into the httpdate time into the expires= like some of the other tests do.

I think this test can be written like the two above it. That is, we can travel within the expiration window, assert we can read it, then travel to outside the expiration window and assert it's nil.

assain · 2017-08-14T16:20:16Z

actionpack/lib/action_dispatch/middleware/cookies.rb

-          options[:value] = @encryptor.encrypt_and_sign(serialize(options[:value]))
+          if options[:expires].respond_to?(:from_now)
+            options[:expires] = options[:expires].from_now
+          end


@kaspth 😄
Should we extract this into a method?

We can't share that implementation between the general cookie jar and the abstract cookie jar. So maybe we should put a handle_options before commit is called.

Something like:

private delegate :handle_options, to: :@parent_jar

Then we'll eventually hit the CookieJar handle_options. Though it might be better to define a preprocess_options that specifically does the from_now handling.

Shows another fault in the cookies code, that we still have duplicated and split options processing, but I guess that's for later.

kaspth

There's a failing test that now has it's signed data changed because we're verifying the integrity of the expiry.

kaspth · 2017-08-14T18:35:30Z

actionpack/test/dispatch/cookies_test.rb

+
+    def cookie_expires_in_two_hours
+      cookies[:user_name] = { value: "assain", expires: 2.hours }
+      head :ok


Why do we need the head :ok don't these default to head :no_content?

@kaspth 😄
But, deleting the line throws ActionController::UnknownFormat:
ActionController::UnknownFormat: CookiesTest::TestController#cookie_expires_in_two_hours is missing a template for this request format and variant.

kaspth · 2017-08-14T18:35:30Z

actionpack/test/dispatch/cookies_test.rb

+
+      expected_time = 2.hours.from_now.utc.rfc2822
+      assert_cookie_header "user_name=assain; path=/; expires=#{expected_time}"
+    end


Don't use freeze_time then. Maybe travel_to Time.parse("some httpdate formatted time with timezone"), then put that into the httpdate time into the expires= like some of the other tests do.

kaspth · 2017-08-14T18:35:30Z

actionpack/lib/action_dispatch/middleware/cookies.rb

-          options[:value] = @encryptor.encrypt_and_sign(serialize(options[:value]))
+          if options[:expires].respond_to?(:from_now)
+            options[:expires] = options[:expires].from_now
+          end


We can't share that implementation between the general cookie jar and the abstract cookie jar. So maybe we should put a handle_options before commit is called.

Something like:

private delegate :handle_options, to: :@parent_jar

Then we'll eventually hit the CookieJar handle_options. Though it might be better to define a preprocess_options that specifically does the from_now handling.

Shows another fault in the cookies code, that we still have duplicated and split options processing, but I guess that's for later.

assain · 2017-08-15T10:06:54Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -488,6 +497,8 @@ def []=(name, options)
        def request; @parent_jar.request; end

      private
+        delegate :preprocess_options, to: :@parent_jar


@kaspth 😄
This way handle_options work is not duplicated.

mjc-gh · 2017-08-16T14:07:09Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -378,6 +378,12 @@ def handle_options(options) #:nodoc:
        end
      end

+      def preprocess_options(options)


Could this method's logic just be moved to the handle_options method?

mjc-gh · 2017-08-16T14:08:29Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -480,6 +487,8 @@ def []=(name, options)
          options = { value: options }
        end

+        preprocess_options(options)


Two lines after this the call @parent_jar[name] will lead to preprocess_options being invoked in the @parent_jar. Thus, I think this call maybe unnecessary...

@mikeycgto 😄
Correct me if I'm wrong:
We now want :expires to support duration, i.e. we want to specify the duration for which the cookie is valid like this:
cookies.signed[:user_name] = { value: "bob", expires: 2.hours }
And since we're tacking on the value of options[:expires] to :expires_at in case of signed / encrypted cookies, isn't it necessary to do preprocess_options before commit method is called, since @parent_jar[name] = options is only called later on.

If I'm following this code correctly, I think calling @parent_jar[name] = options here will lead to preprocess_options being called again on line 398 in the parent jar. Thus, maybe we only need to call it once in the parent jar itself?

You are right that we do need the options to be "converted" before the cookie is actually written.

The tough part is that we need preprocess_options to happen before commit(options) (so options[:expires] can be passed to :expires_at. When @parent_jar[name] = options comes along we'll hit preprocess_options again, yes.

Perhaps there's two things going on here that happen to look similar.

E.g. there's the duration support in :expires, but then there's the conversion that Messages::Metadata handles.

We could stash the from_now thing in handle_options and then have a separate method to handle the chained cookie jar cases.

def expiry_options if options[:expires].is_a?(ActiveSupport::Duration) { expires_in: options[:expires] } else { expires_at: options[:expires] } end end

That would spare us the complex delegation up the chained jars but cost us a type check. We could perhaps use respond_to?(:from_now) in both cases.

"We could perhaps use respond_to?(:from_now) in both cases"

@kaspth are you suggesting to stick onto delegation? Or make the changes using ActiveSupport::Duration? 😄

No, I meant respond_to?(:from_now) in lieu of is_a? in my proposed expiry_options.

kaspth · 2017-08-16T18:58:17Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -378,6 +378,12 @@ def handle_options(options) #:nodoc:
        end
      end

+      def preprocess_options(options)


Remember to # :nodoc: this.

kaspth · 2017-08-16T19:10:25Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -389,6 +395,7 @@ def []=(name, options)
          options = { value: value }
        end

+        preprocess_options(options)
        handle_options(options)


We should probably just call preprocess_options in handle_options as well.

Of course, if you know why it isn't necessary for delete and deleted? do explain and we'll just leave this as is.

"We should probably just call preprocess_options in handle_options as well." Did you mean to say: "delete the function call here and add it to handle_options?" 😄 Or did I get the message wrong

That was what I meant then… but now I'm more interested in #30121 (comment) and #30121 (comment) 😊

kaspth · 2017-08-16T19:20:32Z

actionpack/test/dispatch/cookies_test.rb

+  def test_vanilla_cookie_with_expires_set_relatively
+    travel_to Time.utc(2017, 8, 15) do
+      get :cookie_expires_in_two_hours
+      assert_cookie_header "user_name=assain; path=/; expires=Tue, 15 Aug 2017 02:00:00 -0000"


Won't this return a different time zone depending on where you run it?

Should I change this? 😅

Only if it needs to change, which is what I'm asking you about 😊

I used Time.utc method, since there were other tests using it while setting the :expires option. Was that method the root of your concern? 😅

Ah, read utc as new, which isn't the same. But now I'm a little curious why the time is "02:00" and not "00:00".

In get :cookie_expires_in_two_hours the cookie is being set to expire in two hours and since:
Time.utc(2017, 8, 15) => 2017-08-15 00:00:00 UTC
Two hours from_now in rfc 2822:
Tue, 15 Aug 2017 02:00:00 -0000
Hope, I got your question right 😄

Ah right, all good 👍

assain · 2017-08-17T11:53:45Z

actionpack/lib/action_dispatch/middleware/cookies.rb

+            { expires_at: options[:expires] }
+          end
+        end
+


@kaspth 😄
Here's the suggested changes!

assain · 2017-08-17T11:59:18Z

actionpack/test/dispatch/session/cookie_store_test.rb

@@ -292,7 +297,7 @@ def test_session_store_with_expire_after
      end

      # Second request does not access the session
-      time = Time.local(2008, 4, 25)
+      time = Time.local(2008, 4, 24)
      Time.stub :now, time do


Are these changes to the test okay?😄

kaspth

One final comment, otherwise I think it's there codewise.

We should update the documentation as well.

We'll also need entries in the Action Pack changelog:

That :expires supports ActiveSupport::Durations
That cookie expiry integrity is now enforced for signed/encrypted cookies

Write a title and short description for each.

Later we'll add something to the upgrading guide.

kaspth · 2017-08-17T19:00:57Z

actionpack/test/dispatch/session/cookie_store_test.rb

@@ -292,7 +297,7 @@ def test_session_store_with_expire_after
      end

      # Second request does not access the session
-      time = Time.local(2008, 4, 25)
+      time = Time.local(2008, 4, 24)


Why do we have to change the date here?

cookie_body at line 294 contains the message with expiry metadata time = Time.local(2008, 4, 24) five hours from this time.
At line 307, the cookie_body contains the value from 294, but its expected expiry is changed by 1 day. Consequently, the assertion fails because the time is stubbed at: Time.local(2008, 4, 25).
Hope I didn't goof up again 😅

But then why did it pass before?

Just being frank: I'm not convinced you have a sufficient understanding of what's going on here, to allow this change to happen. It's your job to prove you know why changes are required. 😊

mjc-gh · 2017-08-17T19:17:41Z

I think it is also worth mentioning the expiry integrity in the security guide. Definitely a major step forward for Rails' security overall 🤘

… to cookies

assain · 2017-08-20T17:40:55Z

actionpack/test/dispatch/session/cookie_store_test.rb

@@ -299,7 +304,7 @@ def test_session_store_with_expire_after
        get "/no_session_access"
        assert_response :success

-        assert_equal "_myapp_session=#{cookie_body}; path=/; expires=#{expected_expiry}; HttpOnly",
+        assert_equal "_myapp_session=#{cookies[SessionKey]}; path=/; expires=#{expected_expiry}; HttpOnly",
          headers["Set-Cookie"]
      end


Here's the suggested changes @kaspth 😄

assain · 2017-08-20T18:22:45Z

@kaspth, The checks have passed 😄

kaspth · 2017-08-20T18:23:39Z

Indeed they have! 😊

jfine · 2017-08-28T18:45:47Z

Great addition!

rails-bot assigned sgrif Aug 7, 2017

assain changed the title ~~Set Cookie Expiration Using :expires_in & :expires_at~~ Set Cookie Expiration Using :expires_in & :expires_at Aug 7, 2017

assain commented Aug 7, 2017

View changes

mjc-gh reviewed Aug 9, 2017

View changes

matthewd reviewed Aug 9, 2017

View changes

kaspth reviewed Aug 10, 2017

View changes

assain commented Aug 14, 2017

View changes

assain changed the title ~~Set Cookie Expiration Using :expires_in & :expires_at~~ Add expires_at metadata to Cookies and freshen expires option to support duration Aug 14, 2017

kaspth reviewed Aug 14, 2017

View changes

assain commented Aug 15, 2017

View changes

mjc-gh reviewed Aug 16, 2017

View changes

kaspth reviewed Aug 16, 2017

View changes

assain commented Aug 17, 2017

View changes

assain changed the title ~~Add expires_at metadata to Cookies and freshen expires option to support duration~~ Add expiry metadata to Cookies and freshen expires option to support duration Aug 17, 2017

kaspth reviewed Aug 17, 2017

View changes

kaspth assigned kaspth and unassigned sgrif Aug 19, 2017

freshen :expires option with duration support and add expiry metadata…

5f37e16

… to cookies

assain commented Aug 20, 2017

View changes

kaspth merged commit cdcd6c0 into rails:master Aug 20, 2017

assain mentioned this pull request Aug 25, 2017

Add Documentation For Duration Support & Expiry Meta Data Added to Signed / Encrypted Cookies #30407

Merged

Add expiry metadata to Cookies and freshen expires option to support duration #30121

Add expiry metadata to Cookies and freshen expires option to support duration #30121

Conversation

assain commented Aug 7, 2017 • edited

rails-bot commented Aug 7, 2017

assain Aug 7, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain Aug 15, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain Aug 16, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain Aug 17, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjc-gh commented Aug 17, 2017 via email • edited by kaspth

Choose a reason for hiding this comment

assain commented Aug 20, 2017

kaspth commented Aug 20, 2017

jfine commented Aug 28, 2017

assain commented Aug 7, 2017 •

edited

assain Aug 7, 2017 •

edited

assain Aug 15, 2017 •

edited

assain Aug 16, 2017 •

edited

assain Aug 17, 2017 •

edited

mjc-gh commented Aug 17, 2017 via email •

edited by kaspth