Add Purpose Metadata to Cookies #32937

assain · 2018-05-20T18:50:05Z

Summary
This PR adds purpose metadata to cookies so that the value of one cookie cannot be copied and used as the value of another cookie.

Other Information
Turn on config.action_dispatch.use_cookies_with_metadata to embed expiry and purpose metadata inside signed/encrypted cookies, and to verify if signed/encrypted cookie values have been copy-pasted. Previously set cookies (i.e. without purpose or expiry metadata) will continue to be honored.

rails-bot · 2018-05-20T18:50:07Z

r? @georgeclaghorn

(@rails-bot has picked a reviewer for you, use r? to override)

assain · 2018-05-20T18:51:16Z

r? @kaspth
@guilleiguaran

Edouard-chin · 2018-05-20T19:52:30Z

actionpack/test/dispatch/cookies_test.rb

@@ -1300,6 +1300,22 @@ def test_vanilla_cookie_with_expires_set_relatively
    end
  end

+  def test_purpose_metadata_for_signed_cookies


We probably want the two new tests to set the ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata config to true

The tests related to purpose and expiry metadata are failing because I can't switch on - ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata properly, which is set to false by default.

I tried setting ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata config to true within the new tests, but don't know why it breaks a lot of other tests. Would it be because of the way the tests are run ?

I tried switching it to true in the metadata tests like this -

def test_purpose_metadata_for_encrypted_cookies ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = true cookies.encrypted[:user_id] = 50 cookies.encrypted[:discount_percentage] = 10 cookies[:discount_percentage] = cookies[:user_id] assert_nil cookies.encrypted[:discount_percentage] # ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = false end

I also tried creating a subclass of CookiesTest with setup and teardown methods. But it screws up everything and other kinds of experimentation.

Could you suggest me the right way to do this? 😕
@Edouard-chin @kaspth @guilleiguaran

It's probably because ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = false was never run since the execution stoped when the assertion assert_nil cookies.encrypted[:discount_percentage] failed.

I had tried both. Are tests run parallelly ?

They are not, I tried running the tests and they work fine with something like

Diff

diff --git a/actionpack/test/dispatch/cookies_test.rb b/actionpack/test/dispatch/cookies_test.rb index aa9e4dac42..9a822c4f44 100644 --- a/actionpack/test/dispatch/cookies_test.rb +++ b/actionpack/test/dispatch/cookies_test.rb @@ -1301,19 +1301,27 @@ def test_vanilla_cookie_with_expires_set_relatively end def test_purpose_metadata_for_signed_cookies + previous_config = ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata + ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = true cookies.signed[:user_id] = 50 cookies.signed[:discount_percentage] = 10 cookies[:discount_percentage] = cookies[:user_id] assert_nil cookies.signed[:discount_percentage] + ensure + ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = previous_config end def test_purpose_metadata_for_encrypted_cookies + previous_config = ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata + ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = true cookies.encrypted[:user_id] = 50 cookies.encrypted[:discount_percentage] = 10 cookies[:discount_percentage] = cookies[:user_id] assert_nil cookies.encrypted[:discount_percentage] + ensure + ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = previous_config end

Only two tests are failing related to expiry, but it makes sense since the expiry option is now set solely when use_cookies_with_metadata is true.

Thanks, @Edouard-chin for helping me out with your expertise. Got to make a few more tweaks to the PR 😄

Edouard-chin · 2018-05-20T19:52:50Z

actionpack/test/dispatch/cookies_test.rb

+   cookies.signed[:discount_percentage] = 10
+
+   cookies[:discount] = cookies[:user_id]
+   assert_nil cookies.signed[:discount_percentage]


Shouldn't this be assert_nil cookies.signed[:discount] instead?

@Edouard-chin Yeah, my bad 😅

Edouard-chin · 2018-05-20T19:52:55Z

actionpack/test/dispatch/cookies_test.rb

@@ -1318,4 +1334,4 @@ def assert_not_cookie_header(expected)
        assert_not_equal expected.split("\n"), header
      end
    end
-end
+end


@Edouard-chin I've made some more changes 😄

assain · 2018-05-20T20:03:16Z

actionpack/test/dispatch/cookies_test.rb

+    cookies[:discount] = cookies[:user_id]
+    assert_nil cookies.encrypted[:discount_percentage]
+  end
+


The tests related to purpose and expiry metadata are failing, because I can't switch on - ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata which is set to false by default.
I tried turning it to true in the metadata tests like this -

def test_purpose_metadata_for_encrypted_cookies ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = true cookies.encrypted[:user_id] = 50 cookies.encrypted[:discount_percentage] = 10 cookies[:discount] = cookies[:user_id] assert_nil cookies.encrypted[:discount_percentage] # ActionDispatch::Cookies::AbstractCookieJar.use_cookies_with_metadata = false end

I also tried creating a subclass of CookiesTest with setup and teardown methods. But it screws up everything and other kinds of experimentation.

Could you suggest me the right way to do this? 😕

mjc-gh

Looks great overall! Nice work!!

mjc-gh · 2018-06-10T21:54:47Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -489,11 +497,30 @@ def []=(name, options)
        def request; @parent_jar.request; end

      private
-        def expiry_options(options)
+        def fetch_metadata(options)
+          metadata = Hash.new


Could make this a bit more concise with tap and a Hash literal:

{ purpose: options[:cookie_name] }.tap do |metadata| # set expires_in or expires_at... end

assain · 2018-06-10T23:51:16Z

actionpack/lib/action_dispatch/middleware/cookies.rb

+              metadata[:expires_at] = options[:expires]
+            end
+          end
+        end


@mikeycgto Was this what you had suggested? 😄

Yup, exactly!

assain · 2018-06-11T00:17:52Z

...ils/generators/rails/app/templates/config/initializers/handle_cookies_without_metadata.rb.tt

+# for how long those cookies should be honored as a Time or ActiveSupport::Duration object.
+
+# Also, setting this value to nil or to a date in the past rejects the existing cookies upfront.
+Rails.application.config.action_dispatch.fail_cookies_without_metadata_after = <%= 1.month.from_now %>


@mikeycgto @kaspth @guilleiguaran
Travis CI's GEM = aj:integration fails with a syntax error because of the output of <%= 1.month.from_now %>. Should I convert <%= 1.month.from_now %> into a string or any other format? Or am I missing anything?

guilleiguaran · 2018-06-11T17:56:52Z

...ils/generators/rails/app/templates/config/initializers/handle_cookies_without_metadata.rb.tt

+# for how long those cookies should be honored as a Time or ActiveSupport::Duration object.
+
+# Also, setting this value to nil or to a date in the past rejects the existing cookies upfront.
+Rails.application.config.action_dispatch.fail_cookies_without_metadata_after = <%= ActiveSupport::JSON.encode 1.month.from_now %>


Maybe https://github.com/rails/rails/blob/master/railties/lib/rails/generators/rails/app/templates/config/initializers/new_framework_defaults_6_0.rb.tt is a better place for this?

@kaspth wdyt?

@guilleiguaran Yes, that's the place we should put this.

kaspth

Initial pass! Looks like this mostly gets the job done, but the flow and readability isn't quite where we need it. But that's good! Now we have a base to expand on.

Left lots of questions to hopefully guide you in your further decision making.

kaspth · 2018-06-12T14:07:26Z

actionpack/test/dispatch/cookies_test.rb

    cookies.signed[:user_name] = { value: "assain", expires: 2.hours }

    travel 1.hour
    assert_equal "assain", cookies.signed[:user_name]

    travel 2.hours
    assert_nil cookies.signed[:user_name]
+
+  ensure
+    switch_off_cookies_with_metadata


Having these ensures spread throughout is a bad practice.

Switch to:

with_cookie_metadata_enabled do # … end

Remember to capture the old value in a local variable too and reassign it back.

kaspth · 2018-06-12T14:08:00Z

...ils/generators/rails/app/templates/config/initializers/handle_cookies_without_metadata.rb.tt

+# for how long those cookies should be honored as a Time or ActiveSupport::Duration object.
+
+# Also, setting this value to nil or to a date in the past rejects the existing cookies upfront.
+Rails.application.config.action_dispatch.fail_cookies_without_metadata_after = <%= ActiveSupport::JSON.encode 1.month.from_now %>


@guilleiguaran Yes, that's the place we should put this.

kaspth · 2018-06-12T14:09:58Z

...ils/generators/rails/app/templates/config/initializers/handle_cookies_without_metadata.rb.tt

+# for how long those cookies should be honored as a Time or ActiveSupport::Duration object.
+
+# Also, setting this value to nil or to a date in the past rejects the existing cookies upfront.
+Rails.application.config.action_dispatch.fail_cookies_without_metadata_after = <%= ActiveSupport::JSON.encode 1.month.from_now %>


Why did you make this a JSON value here? What's the benefit of this way, as opposed to deferring it? How does this choice mesh with our goal of programmer happiness and readability?

Where is use_cookies_with_metadata set to true? Seems like we'd ship something that would never actually use metadata? And is it the best approach to split this over 2 options? E.g. is the mental overhead of 2 toggles worth it for end users?

kaspth · 2018-06-12T14:11:35Z

...ils/generators/rails/app/templates/config/initializers/handle_cookies_without_metadata.rb.tt

+# If you don't want your existing cookies to be rejected immediately you need to specify a window
+# for how long those cookies should be honored as a Time or ActiveSupport::Duration object.
+
+# Also, setting this value to nil or to a date in the past rejects the existing cookies upfront.


Put yourself in the shoes of a person who's upgrading their Rails app. What would they want to know here?

kaspth · 2018-06-12T14:15:22Z

actionpack/lib/action_dispatch/middleware/cookies.rb

-        def expiry_options(options)
-          if options[:expires].respond_to?(:from_now)
-            { expires_in: options[:expires] }
+        def fetch_metadata(options)


Why is this named fetch? How are we fetching the metadata? And is it the input that's the most important thing here? Do we regularly call methods to pass something in or to get something out?

kaspth · 2018-06-12T14:17:46Z

actionpack/lib/action_dispatch/middleware/cookies.rb

          end
        rescue ActiveSupport::MessageEncryptor::InvalidMessage, ActiveSupport::MessageVerifier::InvalidSignature
          parse_legacy_signed_message(name, encrypted_message)
        end

        def commit(options)
-          options[:value] = @encryptor.encrypt_and_sign(serialize(options[:value]), expiry_options(options))
+          if cookies_with_metadata?
+            options[:value] = @encryptor.encrypt_and_sign(serialize(options[:value]), fetch_metadata(options))


Why is this method concerned with cookies_with_metadata?? Especially when there's a method that's called fetch_metadata? Same for the other instances.

kaspth · 2018-06-12T14:20:35Z

actionpack/lib/action_dispatch/middleware/cookies.rb

@@ -481,6 +487,8 @@ def []=(name, options)
          options = { value: options }
        end

+        options[:cookie_name] = name


Won't this break old Rails apps as we're always writing the cookie_name to the cookie? E.g. would a cookie from 5.2 (e.g. no metadata) be readable on 6.0? And would a metadata attached cookie from 6.0 be readable on 5.2? This is one of the most important things we have to keep in mind, so do our tests test these scenarios sufficiently and do they do so in a sufficiently realistic manner?

I still don't like assigning straight to the options here. They'll be committed in the cookie body after all. Have you tried passing the name in to commit or moving the metadata construction up to here?

kaspth · 2018-06-12T14:23:21Z

actionpack/lib/action_dispatch/middleware/cookies.rb

+              end
+              value
+            else
+              @encryptor.decrypt_and_verify(encrypted_message, on_rotation: rotate)


Calling @encryptor.decrypt_and_verify(encrypted_message, on_rotation: rotate) 3 times here is a smell.

kaspth · 2018-06-12T14:23:44Z

actionpack/lib/action_dispatch/middleware/cookies.rb

+              if value.nil? && read_cookies_without_metadata?
+                value = @verifier.verified(signed_message, on_rotation: rotate)
+              end
+              value


Also smelly.

kaspth · 2018-06-12T14:25:34Z

actionpack/lib/action_dispatch/middleware/cookies.rb

-          if options[:expires].respond_to?(:from_now)
-            { expires_in: options[:expires] }
+        def fetch_metadata(options)
+          { purpose: options[:cookie_name] }.tap do |metadata|


Think the tap here blurs the flow. Is building the metadata at the same level of abstraction as normalizing the expiry? If not, how can we compose this code better?

assain · 2018-06-18T16:15:40Z

@kaspth Here's the suggested changes 😄

assain · 2018-06-26T15:08:49Z

actionpack/lib/action_dispatch/railtie.rb

+    config.action_dispatch.cookies_with_purpose_metadata = {
+      switch_on: false,
+      honor_cookies_wihout_metadata_till: nil
+    }


There's a typo here, I'll fix that...

sikachu

I do feel weird about having a hash ({ switch_on: true }) as a way to enable the feature. Is there a particular reason why we can't make falsy as an off state and any object as an on state?

i.e.

.cookies_with_purpose_metadata = false # off
.cookies_with_purpose_metadata = true # on, `honor_cookies_without_metadata_till` is nil
.cookies_with_purpose_metadata = { honor_cookies_without_metadata_till: 30.days.from_now }

Also, I'm curious about the wording of honor_cookies_without_metadata_till. I think expanding till to until would make it reads better, and maybe we can change honor to allow to, just to make it more inline of other flags that available in Rails (assuming it still conveys the same message):

config.action_dispatch.cookies_with_purpose_metadata = {
  allow_cookies_without_metadata_until: 30.days.from_now
}

Please let me know what you think. Thank you for your patch.

sikachu · 2018-06-27T00:41:27Z

...ib/rails/generators/rails/app/templates/config/initializers/new_framework_defaults_6_0.rb.tt

+# before they're rejected so that your application behaviour is not broken immediately.
+#
+# Rails.application.config.action_dispatch.cookies_with_purpose_metadata = {
+# switch_on: true,


Do you mind adding some indentation for line 21 and 22 too?

Purpose metadata prevents cookie values from being copy-pasted and ensures that the cookie is used only for its originally intended purpose. The Purpose and Expiry metadata are embedded inside signed/encrypted cookies and will not be readable on previous versions of Rails. We can switch off purpose and expiry metadata embedded in signed and encrypted cookies using config.action_dispatch.use_cookies_with_metadata = false if you want your cookies to be readable on older versions of Rails.

kaspth

❤️👏

Let's clean up the test things and add a changelog in a separate PR. I don't want to hold this up any longer 😄

kaspth · 2018-08-12T17:00:35Z

actionpack/test/dispatch/cookies_test.rb

+    assert_equal "5-2-Stable Chocolate Cookies", cookies.encrypted[:favorite]
+
+    freeze_time do
+      travel 1001.years


freeze_time just does travel_to Time.now, so you could just do travel 1001.years do.

…"Configuring Rails Applications" guide [ci skip] Related to rails#32937, rails#33605.

…[ci skip] Context rails#33605 (comment) Related to rails#32937, rails#33605

Follow up #32937 [ci skip]

- There is a regression in 6.0 introduced by rails#32937 where cookie doesn't expire anymore unless the new `use_cookies_with_metadata` configuration is set to `true`. This causes issue for app migration from 5.2 to 6.0 because the `use_cookies_with_metadata` flag can't be set to true until all servers are running on 6.0. Here is a small reproduction script that you can run in the console ```ruby ActionDispatch::Cookies request = ActionDispatch::Request.empty request.env["action_dispatch.key_generator"] = ActiveSupport::KeyGenerator.new('1234567890') request.env["action_dispatch.signed_cookie_salt"] = 'signed cookie' request.env["action_dispatch.cookies_rotations"] = ActiveSupport::Messages::RotationConfiguration.new request.env["action_dispatch.use_authenticated_cookie_encryption"] = true signed_cookie = request.cookie_jar.signed signed_cookie[:foobar] = { value: '123', expires: 1.day.ago } p signed_cookie[:foobar] ```

The use_cookies_with_metadata option was created in #32937 and documented in #33757. At the time, the option controlled both purpose and expiry metadata. Later #35134 changed the option to control only the purpose metadata, but the documentation was not updated. I've done that here.

rails-bot assigned georgeclaghorn May 20, 2018

rails-bot assigned kaspth and unassigned georgeclaghorn May 20, 2018

Edouard-chin reviewed May 20, 2018

View reviewed changes

assain commented May 20, 2018

View reviewed changes

assain force-pushed the add-purpose-to-cookies branch from 1153258 to 11ae805 Compare May 20, 2018 20:38

assain force-pushed the add-purpose-to-cookies branch 2 times, most recently from 290e675 to d898409 Compare June 8, 2018 18:31

assain changed the title ~~Add purpose metadata to cookies.~~ Add Purpose Metadata to Cookies Jun 8, 2018

assain force-pushed the add-purpose-to-cookies branch 3 times, most recently from 6819834 to a8c88cd Compare June 10, 2018 07:04

mjc-gh approved these changes Jun 10, 2018

View reviewed changes

assain force-pushed the add-purpose-to-cookies branch from a8c88cd to 803ab5c Compare June 10, 2018 23:47

assain commented Jun 10, 2018

View reviewed changes

assain commented Jun 11, 2018

View reviewed changes

assain force-pushed the add-purpose-to-cookies branch 3 times, most recently from aa4cd7e to cf88f3b Compare June 11, 2018 10:12

guilleiguaran reviewed Jun 11, 2018

View reviewed changes

kaspth reviewed Jun 12, 2018

View reviewed changes

assain force-pushed the add-purpose-to-cookies branch 3 times, most recently from 9e9aff9 to 490cfcb Compare June 18, 2018 16:08

assain commented Jun 26, 2018

View reviewed changes

sikachu requested changes Jun 27, 2018

View reviewed changes

assain force-pushed the add-purpose-to-cookies branch from 490cfcb to 6d4fd58 Compare July 1, 2018 13:33

assain force-pushed the add-purpose-to-cookies branch 8 times, most recently from ae121b9 to 5d03ffc Compare August 11, 2018 18:22

assain force-pushed the add-purpose-to-cookies branch from 5d03ffc to 1cda4fb Compare August 12, 2018 16:21

kaspth approved these changes Aug 12, 2018

View reviewed changes

kaspth merged commit 14d3c7c into rails:master Aug 12, 2018

bogdanvlviv mentioned this pull request Aug 30, 2018

Follow up #32937 [ci skip] #33757

Merged

bogdanvlviv added a commit to bogdanvlviv/rails that referenced this pull request Aug 30, 2018

Add info about config.action_dispatch.use_cookies_with_metadata to …

35740ab

…"Configuring Rails Applications" guide [ci skip] Related to rails#32937, rails#33605.

bogdanvlviv added a commit to bogdanvlviv/rails that referenced this pull request Aug 30, 2018

Add info about purpose in cookies to "Upgrading Ruby on Rails" guide …

03ba74c

…[ci skip] Context rails#33605 (comment) Related to rails#32937, rails#33605

kamipo added a commit that referenced this pull request Aug 30, 2018

Merge pull request #33757 from bogdanvlviv/follow-up-32937

9297af5

Follow up #32937 [ci skip]

Edouard-chin mentioned this pull request Feb 1, 2019

Cookie doesn't expire anymore unless a flag is set: #35134

Merged

kaspth mentioned this pull request Apr 25, 2019

Default cookies_serializer to :json #33967

Closed

rhymes mentioned this pull request May 23, 2020

Upgrade to Rails 6.0 forem/forem#7658

Merged

12 tasks

rolandasb mentioned this pull request May 27, 2020

Fix nil encrypted/signed cookie value when value is stored as false #39452

Merged

clayton-shopify mentioned this pull request Dec 2, 2020

Fix the documentation for use_cookies_with_metadata #40724

Merged

Add Purpose Metadata to Cookies #32937

Add Purpose Metadata to Cookies #32937

Conversation

assain commented May 20, 2018 • edited Loading

rails-bot commented May 20, 2018

assain commented May 20, 2018

Choose a reason for hiding this comment

assain May 20, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain May 20, 2018 • edited Loading

Choose a reason for hiding this comment

mjc-gh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain Jun 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain commented Jun 18, 2018

Choose a reason for hiding this comment

sikachu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

assain commented May 20, 2018 •

edited

Loading

assain May 20, 2018 •

edited

Loading

assain May 20, 2018 •

edited

Loading

assain Jun 10, 2018 •

edited

Loading