Support :message_pack as message serializer

activesupport/CHANGELOG.md

@@ -1,3 +1,40 @@
+*   `MessageEncryptor`, `MessageVerifier`, and `config.active_support.message_serializer`
+    now accept `:message_pack` and `:message_pack_allow_marshal` as serializers.
+    These serializers require the [`msgpack` gem](https://rubygems.org/gems/msgpack)
+    (>= 1.7.0).
+
+    The Message Pack format can provide improved performance and smaller payload
+    sizes. It also supports roundtripping some Ruby types that are not supported
+    by JSON. For example:
+
+      ```ruby
+      verifier = ActiveSupport::MessageVerifier.new("secret")
+      data = [{ a: 1 }, { b: 2 }.with_indifferent_access, 1.to_d, Time.at(0, 123)]
+      message = verifier.generate(data)
+
+      # BEFORE with config.active_support.message_serializer = :json
+      verifier.verified(message)
+      # => [{"a"=>1}, {"b"=>2}, "1.0", "1969-12-31T18:00:00.000-06:00"]
+      verifier.verified(message).map(&:class)
+      # => [Hash, Hash, String, String]
+
+      # AFTER with config.active_support.message_serializer = :message_pack
+      verifier.verified(message)
+      # => [{:a=>1}, {"b"=>2}, 0.1e1, 1969-12-31 18:00:00.000123 -0600]
+      verifier.verified(message).map(&:class)
+      # => [Hash, ActiveSupport::HashWithIndifferentAccess, BigDecimal, Time]
+      ```
+
+    The `:message_pack` serializer can fall back to deserializing with
+    `ActiveSupport::JSON` when necessary, and the `:message_pack_allow_marshal`
+    serializer can fall back to deserializing with `Marshal` as well as
+    `ActiveSupport::JSON`. Additionally, the `:marshal`, `:json`, and
+    `:json_allow_marshal` serializers can now fall back to deserializing with
+    `ActiveSupport::MessagePack` when necessary. These behaviors ensure old
+    messages can still be read so that migration is easier.
+
+    *Jonathan Hefner*
+
 *   A new `7.1` cache format is available which includes an optimization for
     bare string values such as view fragments. The `:message_pack` cache format
     has also been modified to include this optimization.

activesupport/lib/active_support/message_encryptor.rb

@@ -144,18 +144,24 @@ class InvalidMessage < StandardError; end
     #   The serializer used to serialize message data. You can specify any
     #   object that responds to +dump+ and +load+, or you can choose from
     #   several preconfigured serializers: +:marshal+, +:json_allow_marshal+,
-    #   +:json+.
+    #   +:json+, +:message_pack_allow_marshal+, +:message_pack+.
     #
     #   The preconfigured serializers include a fallback mechanism to support
     #   multiple deserialization formats. For example, the +:marshal+ serializer
-    #   will serialize using +Marshal+, but can deserialize using +Marshal+ or
-    #   ActiveSupport::JSON. This makes it easy to migrate between serializers.
+    #   will serialize using +Marshal+, but can deserialize using +Marshal+,
+    #   ActiveSupport::JSON, or ActiveSupport::MessagePack. This makes it easy
+    #   to migrate between serializers.
     #
-    #   The +:marshal+ and +:json_allow_marshal+ serializers support
-    #   deserializing using +Marshal+, but :+json+ does not. Beware that
-    #   +Marshal+ is a potential vector for deserialization attacks in cases
-    #   where a message signing secret has been leaked. <em>If possible, choose
-    #   a serializer that does not support +Marshal+.</em>
+    #   The +:marshal+, +:json_allow_marshal+, and +:message_pack_allow_marshal+
+    #   serializers support deserializing using +Marshal+, but the others do
+    #   not. Beware that +Marshal+ is a potential vector for deserialization
+    #   attacks in cases where a message signing secret has been leaked. <em>If
+    #   possible, choose a serializer that does not support +Marshal+.</em>
+    #
+    #   The +:message_pack+ and +:message_pack_allow_marshal+ serializers use
+    #   ActiveSupport::MessagePack, which can roundtrip some Ruby types that are
+    #   not supported by JSON, and may provide improved performance. However,
+    #   these require the +msgpack+ gem.
     #
     #   When using \Rails, the default depends on +config.active_support.message_serializer+.
     #   Otherwise, the default is +:marshal+.

activesupport/lib/active_support/message_pack/serializer.rb

@@ -5,7 +5,7 @@
 module ActiveSupport
   module MessagePack
     module Serializer # :nodoc:
-      SIGNATURE = (+"\xCC\x80").force_encoding("ASCII-8BIT").freeze # == 128.to_msgpack
+      SIGNATURE = "\xCC\x80".b.freeze # == 128.to_msgpack
       SIGNATURE_INT = 128
 
       def dump(object)

activesupport/lib/active_support/message_verifier.rb

@@ -115,18 +115,24 @@ class InvalidSignature < StandardError; end
     #   The serializer used to serialize message data. You can specify any
     #   object that responds to +dump+ and +load+, or you can choose from
     #   several preconfigured serializers: +:marshal+, +:json_allow_marshal+,
-    #   +:json+.
+    #   +:json+, +:message_pack_allow_marshal+, +:message_pack+.
     #
     #   The preconfigured serializers include a fallback mechanism to support
     #   multiple deserialization formats. For example, the +:marshal+ serializer
-    #   will serialize using +Marshal+, but can deserialize using +Marshal+ or
-    #   ActiveSupport::JSON. This makes it easy to migrate between serializers.
-    #
-    #   The +:marshal+ and +:json_allow_marshal+ serializers support
-    #   deserializing using +Marshal+, but :+json+ does not. Beware that
-    #   +Marshal+ is a potential vector for deserialization attacks in cases
-    #   where a message signing secret has been leaked. <em>If possible, choose
-    #   a serializer that does not support +Marshal+.</em>
+    #   will serialize using +Marshal+, but can deserialize using +Marshal+,
+    #   ActiveSupport::JSON, or ActiveSupport::MessagePack. This makes it easy
+    #   to migrate between serializers.
+    #
+    #   The +:marshal+, +:json_allow_marshal+, and +:message_pack_allow_marshal+
+    #   serializers support deserializing using +Marshal+, but the others do
+    #   not. Beware that +Marshal+ is a potential vector for deserialization
+    #   attacks in cases where a message signing secret has been leaked. <em>If
+    #   possible, choose a serializer that does not support +Marshal+.</em>
+    #
+    #   The +:message_pack+ and +:message_pack_allow_marshal+ serializers use
+    #   ActiveSupport::MessagePack, which can roundtrip some Ruby types that are
+    #   not supported by JSON, and may provide improved performance. However,
+    #   these require the +msgpack+ gem.
     #
     #   When using \Rails, the default depends on +config.active_support.message_serializer+.
     #   Otherwise, the default is +:marshal+.

activesupport/lib/active_support/messages/metadata.rb

@@ -16,11 +16,14 @@ module Metadata # :nodoc:
         Marshal,
       ]
 
-      TIMESTAMP_SERIALIZERS = []
+      TIMESTAMP_SERIALIZERS = [
+        SerializerWithFallback::SERIALIZERS.fetch(:message_pack),
+        SerializerWithFallback::SERIALIZERS.fetch(:message_pack_allow_marshal),
+      ]
 
       ActiveSupport.on_load(:message_pack) do
-        ENVELOPE_SERIALIZERS.unshift ActiveSupport::MessagePack
-        TIMESTAMP_SERIALIZERS.unshift ActiveSupport::MessagePack
+        ENVELOPE_SERIALIZERS << ActiveSupport::MessagePack
+        TIMESTAMP_SERIALIZERS << ActiveSupport::MessagePack
       end
 
       private

activesupport/lib/active_support/messages/serializer_with_fallback.rb

@@ -1,11 +1,16 @@
 # frozen_string_literal: true
 
+require "active_support/core_ext/kernel/reporting"
 require "active_support/notifications"
 
 module ActiveSupport
   module Messages # :nodoc:
     module SerializerWithFallback # :nodoc:
       def self.[](format)
+        if format.to_s.include?("message_pack") && !defined?(ActiveSupport::MessagePack)
+          require "active_support/message_pack"
+        end
+
         SERIALIZERS.fetch(format)
       end
 
@@ -27,6 +32,8 @@ def load(dumped)
       private
         def detect_format(dumped)
           case
+          when MessagePackWithFallback.dumped?(dumped)
+            :message_pack
           when MarshalWithFallback.dumped?(dumped)
             :marshal
           when JsonWithFallback.dumped?(dumped)
@@ -103,10 +110,48 @@ module JsonWithFallbackAllowMarshal
           extend self
         end
 
+        module MessagePackWithFallback
+          include SerializerWithFallback
+          extend self
+
+          def format
+            :message_pack
+          end
+
+          def dump(object)
+            ActiveSupport::MessagePack.dump(object)
+          end
+
+          def _load(dumped)
+            ActiveSupport::MessagePack.load(dumped)
+          end
+
+          def dumped?(dumped)
+            available? && ActiveSupport::MessagePack.signature?(dumped)
+          end
+
+          private
+            def available?
+              return @available if defined?(@available)
+              silence_warnings { require "active_support/message_pack" }
+              @available = true
+            rescue LoadError
+              @available = false
+            end
+        end
+
+        module MessagePackWithFallbackAllowMarshal
+          include MessagePackWithFallback
+          include AllowMarshal
+          extend self
+        end
+
         SERIALIZERS = {
           marshal: MarshalWithFallback,
           json: JsonWithFallback,
           json_allow_marshal: JsonWithFallbackAllowMarshal,
+          message_pack: MessagePackWithFallback,
+          message_pack_allow_marshal: MessagePackWithFallbackAllowMarshal,
         }
     end
   end

activesupport/test/messages/serializer_with_fallback_test.rb

@@ -13,6 +13,11 @@ class MessagesSerializerWithFallbackTest < ActiveSupport::TestCase
     assert_roundtrip serializer(:json_allow_marshal), ActiveSupport::JSON
   end
 
+  test ":message_pack serializer dumps objects using MessagePack format" do
+    assert_roundtrip serializer(:message_pack), ActiveSupport::MessagePack
+    assert_roundtrip serializer(:message_pack_allow_marshal), ActiveSupport::MessagePack
+  end
+
   test "every serializer can load every non-Marshal format" do
     (FORMATS - [:marshal]).product(FORMATS) do |dumping, loading|
       assert_roundtrip serializer(dumping), serializer(loading)

guides/source/configuring.md

@@ -2257,14 +2257,21 @@ support multiple deserialization formats:
 
 | Serializer | Serialize and deserialize | Fallback deserialize |
 | ---------- | ------------------------- | -------------------- |
-| `:marshal` | `Marshal` | `ActiveSupport::JSON` |
-| `:json` | `ActiveSupport::JSON` | |
-| `:json_allow_marshal` | `ActiveSupport::JSON` | `Marshal` |
+| `:marshal` | `Marshal` | `ActiveSupport::JSON`, `ActiveSupport::MessagePack` |
+| `:json` | `ActiveSupport::JSON` | `ActiveSupport::MessagePack` |
+| `:json_allow_marshal` | `ActiveSupport::JSON` | `ActiveSupport::MessagePack`, `Marshal` |
+| `:message_pack` | `ActiveSupport::MessagePack` | `ActiveSupport::JSON` |
+| `:message_pack_allow_marshal` | `ActiveSupport::MessagePack` | `ActiveSupport::JSON`, `Marshal` |
 
 WARNING: `Marshal` is a potential vector for deserialization attacks in cases
 where a message signing secret has been leaked. _If possible, choose a
 serializer that does not support `Marshal`._
 
+INFO: The `:message_pack` and `:message_pack_allow_marshal` serializers support
+roundtripping some Ruby types that are not supported by JSON, such as `Symbol`.
+They can also provide improved performance and smaller payload sizes. However,
+they require the [`msgpack` gem](https://rubygems.org/gems/msgpack).
+
 Each of the above serializers will emit a [`message_serializer_fallback.active_support`][]
 event notification when they fall back to an alternate deserialization format,
 allowing you to track how often such fallbacks occur.

railties/lib/rails/generators/rails/app/templates/config/initializers/new_framework_defaults_7_1.rb.tt

@@ -92,6 +92,11 @@
 # In Rails 7.2, the default will become `:json` which serializes and
 # deserializes with `ActiveSupport::JSON` only.
 #
+# Alternatively, you can choose `:message_pack` or `:message_pack_allow_marshal`,
+# which serialize with `ActiveSupport::MessagePack`. `ActiveSupport::MessagePack`
+# can roundtrip some Ruby types that are not supported by JSON, and may provide
+# improved performance, but it requires the `msgpack` gem.
+#
 # For more information, see
 # https://guides.rubyonrails.org/v7.1/configuring.html#config-active-support-message-serializer
 #