-
Notifications
You must be signed in to change notification settings - Fork 512
METRON-449 JSONMapParser should unfold maps to arbitrary depths #271
Conversation
@@ -56,7 +56,7 @@ public void testHappyPath() { | |||
|
|||
/** | |||
{ | |||
"collection" : { "blah" : 7, "blah2" : "foo" } | |||
"collection" : { "blah" : 7, "blah2" : "foo", "bigblah" : { "innerBlah" : "baz", "reallyInnerBlah" : { "color" : "grey" }}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like it! Would you make one more test case that involves just a single map?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
like
{
"collection" : {
"key" : "value"
},
"key" : "value"
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I'll do that
@cestella , from looking at the ES templates etc, it seems that ':' is used as the name separator and not '.', although the documented JSON Object uses '.'. Is there some conversion happening? Should the the unfolding use ':'? |
No, ES uses ':' and modifies the keys upon write converting '.' to ':'. On Mon, Sep 26, 2016 at 11:59 AM, ottobackwards notifications@github.com
|
Never mind - just found the field name converter |
+1, tested in single node vagrant. Did the following:
Thanks a lot for your contribution, Otto! |
# By cstella (4) and others # Via cstella * 'master' of https://github.com/apache/incubator-metron: METRON-457 Correct GrokParser logging spelling error (mmiklavc via cestella) closes apache#274 METRON-449 JSONMapParser should unfold maps to arbitrary depths closes apache#271 METRON-453: Add a stellar shell function to open an external editor and return the editor's contents closes apache#272 METRON-452: Add rudimentary configuration management functions to Stellar closes apache#269 METRON-374: Add appropriate bundled 3rd party licenses to NOTICE and LICENSE where appropriate closes apache#229 flush METRON-427 Create Ambari Management Pack for Metron Installation closes apache#266
The JSON Parser currently only unfolds maps at the root level. The parser would support a wider range of sources is it supported unfolding maps at any depth of the document.
METRON-449 adds a recursive function to unfold Maps as they are encountered when parsing. A new function was added because the enum lambdas cannot reference themselves.