Use YAML document terminators and skip empty documents in parser #268
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YAML supports a document terminator (...) when reading files in a stream. This allows for the reader to indicate that the current document has ended without indicating that the stream is finished. Anything after the document terminator is ignored by the consumer. Therefore, we write the terminator at the end of a read file, then immediately follow with a YAML document separator (---), which is mandatory according to the YAML spec when using document terminators. Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
If a document contains only whitespace and we are not able to decode it into one of the supported schemas, we ignore it and continue. We want to avoid skipping a type that does not have a registered schema so that we do not silently ignore errors when package building, but we consider it safe to assume that a YAML document with no content is safe to skip. Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
negz
approved these changes
Jun 11, 2021
This was referenced Jun 14, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of your changes
YAML supports a document terminator (...) when reading files in a
stream. This allows for the reader to indicate that the current document
has ended without indicating that the stream is finished. Anything after
the document terminator is ignored by the consumer. Therefore, we write
the terminator at the end of a read file, then immediately follow with a
YAML document separator (---), which is mandatory according to the YAML
spec when using document terminators.
Signed-off-by: hasheddan georgedanielmangum@gmail.com
If a document contains only whitespace and we are not able to decode it
into one of the supported schemas, we ignore it and continue. We want to
avoid skipping a type that does not have a registered schema so that we
do not silently ignore errors when package building, but we consider it
safe to assume that a YAML document with no content is safe to skip.
Signed-off-by: hasheddan georgedanielmangum@gmail.com
Reference: https://yaml.org/spec/1.2/spec.html#id2801681
I have:
make reviewable testto ensure this PR is ready for review.How has this code been tested
I have updated unit tests to demonstrate this functionality, as well as testing on a variety of existing packages. One important distinction here is that consecutive separators (
---) or trailing separators in a YAML document can still cause an error as they are interpreted as document begin / end. So multiple consecutive separators can produce a document with literal---as content and a trailing separator can produce a document with literal...as content.Overall, this expands the range of valid YAML that the Crossplane CLI and package manager will accept.