Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Rollup's metadata parser #36791

Merged
merged 1 commit into from Dec 18, 2018

Conversation

Projects
None yet
3 participants
@polyfractal
Copy link
Member

commented Dec 18, 2018

The parser used for rollup configs in _meta fields was not able to handle unrelated data in the meta field. If an unrelated object was encountered, it would half-consume the JSON object, realize it wasn't a rollup config, then stop parsing. This would leave the object halfway consumed and the parsing framework would throw an exception.

This commit replaces the parsing logic with a set of minimal static parsers, each for the specific component we care about (_doc, _meta, _rollup) and configured to ignore unknown fields where applicable.

More verbose, but less hacky than before and should be more robust.

Also adds tests (randomized and explicit) to make sure this doesn't break in the future.

Closes #36726

Fix Rollup's metadata parser
The parser used for rollup configs in _meta fields was not able to
handle unrelated data in the meta field.  If an unrelated object
was encountered, it would half-consume the JSON object, realize it
wasn'ta rollup config, then stop parsing.  This would leave the object
halfway consumed and the parsing framework would throw an exception.

This commit replaces the parsing logic with a set of minimal parsers,
each for the specific component we care about (`_doc`, `_meta`,
`_rollup`) and configured to ignore unknown fields where applicable.

Not quite as clean, but less hacky than before and more robust.

Also adds tests (randomized and explicit) to make sure this doesn't
break in the future.
@elasticmachine

This comment has been minimized.

Copy link

commented Dec 18, 2018

@polyfractal polyfractal requested a review from colings86 Dec 18, 2018

@polyfractal polyfractal merged commit 6d49873 into elastic:master Dec 18, 2018

7 checks passed

CLA Commit author is a member of Elasticsearch
Details
elasticsearch-ci-1 Build finished.
Details
elasticsearch-ci-2 Build finished.
Details
elasticsearch-ci/default-distro Build finished.
Details
elasticsearch-ci/docbldesx Build finished.
Details
elasticsearch-ci/oss-distro-docs Build finished.
Details
elasticsearch-ci/packaging-sample Build finished.
Details

polyfractal added a commit that referenced this pull request Dec 18, 2018

Fix Rollup's metadata parser (#36791)
The parser used for rollup configs in _meta fields was not able to
handle unrelated data in the meta field.  If an unrelated object
was encountered, it would half-consume the JSON object, realize it
wasn'ta rollup config, then stop parsing.  This would leave the object
halfway consumed and the parsing framework would throw an exception.

This commit replaces the parsing logic with a set of minimal parsers,
each for the specific component we care about (`_doc`, `_meta`,
`_rollup`) and configured to ignore unknown fields where applicable.

More verbose, but less hacky than before and should be more robust.

Also adds tests (randomized and explicit) to make sure this doesn't
break in the future.

polyfractal added a commit that referenced this pull request Dec 18, 2018

Fix Rollup's metadata parser (#36791)
The parser used for rollup configs in _meta fields was not able to
handle unrelated data in the meta field.  If an unrelated object
was encountered, it would half-consume the JSON object, realize it
wasn'ta rollup config, then stop parsing.  This would leave the object
halfway consumed and the parsing framework would throw an exception.

This commit replaces the parsing logic with a set of minimal parsers,
each for the specific component we care about (`_doc`, `_meta`,
`_rollup`) and configured to ignore unknown fields where applicable.

More verbose, but less hacky than before and should be more robust.

Also adds tests (randomized and explicit) to make sure this doesn't
break in the future.

@colings86 colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.