fix: resolve, check and parse manifests more carefully in the Jackson serializers - #3503
Open
pjfanning wants to merge 1 commit into
Open
fix: resolve, check and parse manifests more carefully in the Jackson serializers#3503pjfanning wants to merge 1 commit into
pjfanning wants to merge 1 commit into
Conversation
… serializers Motivation: Four issues in JacksonSerializer, all reachable from a wire manifest: - The case object branch called getObjectFor before checkAllowedClass. getObjectFor reads the MODULE$ field, which initializes the class, so a manifest naming a class the allow list would reject ran that class's initializer on the way to being rejected. - LZ4Meta.get checked for 4 remaining bytes and then read 8, so a 4 to 7 byte payload beginning with the LZ4 magic raised BufferUnderflowException. - parseManifest called toInt on whatever followed the last '#', so a non-numeric version raised NumberFormatException. - isInAllowList evaluated isBoundToJacksonSerializer first, which calls serializerFor and raises (filling in a stack trace) for a class that is allowed only by prefix. checkAllowedClass runs on every fromBinary. Modification: Resolve the case object's class with getClassFor, which does not initialize, run checkAllowedClass on it, and only then read the module field. Require 8 remaining bytes for an LZ4 header. Parse the manifest version with toIntOption and report a bad one as NotSerializableException. Test the prefix before the binding, which cannot throw. Applied to both serialization-jackson and serialization-jackson3. Result: A rejected manifest no longer initializes the class it names, two malformed manifests are reported as serialization failures rather than as unrelated runtime exceptions, and the allow list check no longer builds an exception per message on the prefix path. No change for manifests that were accepted before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Four issues in
JacksonSerializer, all reachable from the manifest string on the wire.The case object branch resolved the class in a way that initializes it, before checking
the allow list. When the manifest ends in
$,fromBinarycalledsystem.dynamicAccess.getObjectFor(className), which reads theMODULE$field through aVarHandleand so runs the class's initializer;checkAllowedClassran only afterwards.The only gate before that point is
checkAllowedClassName, which consults Jackson's gadgetdeny list rather than
serialization-bindingsorallowed-class-prefix. So a manifestnaming any Scala
objecton the classpath ran that object's body on the way to beingrejected. The sibling branch already does the right thing —
getClassForisClass.forName(fqcn, false, cl)and does not initialize. Same shape as #3495.LZ4Meta.getchecked for four remaining bytes and then read eight — the magic plus thedeclared length. A 4-to-7 byte payload beginning with
0x87D96DF6raisedBufferUnderflowExceptionout ofdecompress.parseManifestcalledtoInton whatever followed the last#, soFoo#abcor atrailing
#raisedNumberFormatExceptionrather than a serialization error.isInAllowListevaluated the throwing operand first.isBoundToJacksonSerializercallsserializerFor, which raisesNotSerializableException— stack trace and all — for a classthat is not bound, and that is exactly the case for a class allowed only by
allowed-class-prefix.checkAllowedClassruns on everyfromBinary, so this built anddiscarded one exception per message on that path.
Modification
getClassFor, runcheckAllowedClasson it, and readthe module field only after that.
toIntOptionand report a non-numeric one asNotSerializableExceptionnaming the manifest.hasAllowedClassPrefixbeforeisBoundToJacksonSerializer.Applied identically to
serialization-jacksonandserialization-jackson3.The reordering in
isInAllowListis the only change with no behaviour difference — bothoperands are pure predicates and
isBoundToJacksonSerializeralready swallows its exception —so it is covered by the existing
allowed-class-prefixtests rather than a new one.Result
A rejected manifest no longer initializes the class it names; two malformed manifests are
reported as serialization failures instead of unrelated runtime exceptions; the allow list
check no longer constructs an exception per message on the prefix path. No change for
manifests that were accepted before.
Tests
New tests in
JacksonSerializerSpecin both modules (so each runs under both the JSON andCBOR serializers):
not allow deserialization of a case object that is not in serialization-bindings— theallow list still rejects it
not initialize a case object class it goes on to reject— a test-only object whose bodysets a flag; the flag must still be false after the rejection
reject a manifest whose version is not a numbernot underflow on a payload that is only as long as the LZ4 magicI checked these discriminate by reverting the production change and re-running: the last
three fail without it (
true did not equal false,NumberFormatException was thrown, andthe underflow respectively). The first passes either way — it documents that the allow list
decision itself is unchanged, which is the point.
sbt "serialization-jackson/testOnly org.apache.pekko.serialization.jackson.*"— 128 passedsbt "serialization-jackson3/testOnly org.apache.pekko.serialization.jackson3.*"— 126 passedsbt "serialization-jackson/mimaReportBinaryIssues"— no issues (serialization-jackson3disables MimaPlugin)
sbt scalafmtAll headerCreateAll— no changesReferences
The case object issue is the same shape as #3495, which stopped the serializer resolving a
wire-supplied manifest class it would not use.
ProtobufSerializer.isInAllowListhas the same throwing-operand-first ordering, but itcaches its method handle after the check so it pays the cost once per class rather than once
per message; it is in
pekko-remoteand is left for a separate change.