file.serialize state: fix corruption when using msgpack serializer#57858
Merged
Conversation
09fe8ba to
fa38484
Compare
rallytime
previously approved these changes
Jul 2, 2020
e225154 to
e9b99c4
Compare
The `file.serialize` state adds a newline to the end of the serialized contents before writing them to disk. This works fine with some serializers, but not so well with others. MacOS plist files are invalid with a trailing newline, so a recent release added a check for that. But that still leaves msgpack, which is also corrupted by the addition of a trailing newline (resulting in a "received extra bytes" error when one attempts to unpack the file). This changeset keeps the state from adding the newline to the end of the file when using the msgpack serializer. In doing so, it simplifies the logic that determines whether or not to add the newline. It makes the assumption that if the serialized contents are a bytestring (as in the plist and msgpack serializers), a newline should not be added, as the results are likely in a binary format that will not work well with unexpected bytes added. Additionally, it introduces a new argument to the state: `serializer`. When the `file.serialize` state was added to Salt nearly 7 years ago, it for some reason was written to use `formatter` to specify which serializer to use. Which is fine, in its own quirky way, but later on the `serializer_opts` and `deserializer_opts` arguments were added to allow the user to pass through additional options to the serializer. And this leaves you with `serializer_opts`/`deserializer_opts`, and `formatter` instead of `serializer`. It's weird and asymmetric. So this changeset also adds `serializer` (updating docs to prefer this usage), while maintaining backwards compatibility with `formatter`.
DmitryKuzmenko
approved these changes
Jul 7, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
file.serializestate adds a newline to the end of the serialized contents before writing them to disk. This works fine with some serializers, but not so well with others. MacOS plist files are invalid with a trailing newline, so a recent release added a check for that. But that still leaves msgpack, which is also corrupted by the addition of a trailing newline (resulting in a "received extra bytes" error when one attempts to unpack the file).This changeset keeps the state from adding the newline to the end of the file when using the msgpack serializer. In doing so, it simplifies the logic that determines whether or not to add the newline. It makes the assumption that if the serialized contents are a bytestring (as in the plist and msgpack serializers), a newline should not be added, as the results are likely in a binary format that will not work well with unexpected bytes added.
Additionally, it introduces a new argument to the state:
serializer. When thefile.serializestate was added to Salt nearly 7 years ago, it for some reason was written to useformatterto specify which serializer to use. Which is fine, in its own quirky way, but later on theserializer_optsanddeserializer_optsarguments were added to allow the user to pass through additional options to the serializer. And this leaves you withserializer_opts/deserializer_opts, andformatterinstead ofserializer. It's weird and asymmetric. So this changeset also addsserializer(updating docs to prefer this usage), while maintaining backwards compatibility withformatter.