Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement custom function for unrolling list of Name/Value pair objects #5

Closed
lowell80 opened this issue Nov 9, 2018 · 1 comment
Closed
Labels
custom function enhancement New feature or request
Milestone

Comments

@lowell80
Copy link
Member

lowell80 commented Nov 9, 2018

Unroll lists of Name/Value pairs and turn them back into dicts/objects. (Rather than using xyseries key Name Value" or eval {Name}=Value tricks.)

Take data that looks like this:

{
    "ExtendedProperties": [
        {
            "Name": "resultType",
            "Value": "Success"
        },
        {
            "Name": "auditEventCategory",
            "Value": "ApplicationManagement"
        },
        {
            "Name": "nCloud",
            "Value": "<null>"
        },
        {
            "Name": "actorContextId",
            "Value": "acaxxxxx-bb0e-42c4-xxxx-xxxxxxxxxxxx"
        },
        {
            "Name": "actorObjectId",
            "Value": "81bxxx08-123e-4a45-xxxx-xxxxxxxxxxxx"
        },
        {
            "Name": "actorObjectClass",
            "Value": "User"
        },
        {
            "Name": "actorUPN",
            "Value": "admin@splunk0365.onmicrosoft.com"
        }, 
   ]
}

And we want to instead map that to an object that looks like this:

{ 
    "resultType" : "Success",
    "auditEventCategory":  "ApplicationManagement",
    "nCloud" : "<null>",
    "actorContextId" : "acaxxxxx-bb0e-42c4-xxxx-xxxxxxxxxxxx",
    "actorObjectId": "81bxxx08-123e-4a45-xxxx-xxxxxxxxxxxx",
    "actorObjectClass": "User",
    "actorUPN" : "admin@splunk0365.onmicrosoft.com"}
}

And from a splunk side of things, it would be helpful if we could assign these all to a common prefix.
So for example, if the prefix given was ExtProp. then the field names in Splunk would be ExtProp.resultType, ExtProp.auditEventCategory, ExtProp.nCloud, ... and so on.

May need to add some kind of field name sanitization to this as well, to prevent spaces and other weird characters from slipping though.

This entire approach would hide values if the name Name was given twice, but I think that's an acceptable risk. Know your data.

@lowell80 lowell80 modified the milestones: Post 2.0, Release 2.0 Nov 9, 2018
@lowell80 lowell80 added enhancement New feature or request custom function labels Nov 9, 2018
@lowell80
Copy link
Member Author

lowell80 commented Nov 9, 2018

Proof-of-conept implementation:

from jmespath import functions

class JmesPathSplunkExtraFunctions(functions.Functions):

    @functions.signature({'types': ['array']}, {'types':['string']}, {'types':['string']})
    def _func_unroll(self, objlist, key, value):
        d = dict()
        for item in objlist:
            try:
                # Todo sanitize key names
                k = None
                v = None
                k = item[key]
                v = item[value]
                d[k] = v
            except Exception as e:
                return "Couldn't find key={} ({}={})value={} in {}".format(key, value, k, v, item) # "IT FAILED:  %s" % (str(e),)
        return d

@lowell80 lowell80 added this to To do in Make it awesome Nov 9, 2018
lowell80 added a commit that referenced this issue Nov 9, 2018
- Added an initial implementation of the unroll(hash,'key','value') function
  for JMESPath.  Still not sure what it should be called, but the logic should
  word.  Closes #5.
- Add an online appinspect checking script.
@lowell80 lowell80 moved this from To do to Finished in Make it awesome Nov 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
custom function enhancement New feature or request
Projects
Make it awesome
  
Finished
Development

No branches or pull requests

1 participant