Parameters in Filter part of Manual Question

[superponible]

This is a valid question in the console:

Get Computer Name from all machines with Index Query File Exists[, , , b32189bdff6e577a92baa61ad49264e6, , , ] containing "Yes"

But when I try to run it as a manual question using PyTan, the parameters are stripped off the filter:

./ask_manual.py -s 'Computer Name' -f 'Index Query File Exists{fileName=b32189bdff6e577a92baa61ad49264e6}, that contains:Yes'
++ Asked Question 'Get Computer Name from all machines with Index Query File Exists containing "Yes"' ID: 2371224

I tried modifying get_filter_obj() in utils.py to add the parameters in the same way that build_selectlist_obj() does it, but that doesn't seem to work.

Below is the added_obj value just before the question is asked, as well as a diff of the code I tried to use to make it work. Since it's a valid question in the console, it seems like this should work. Is this a limitation with PyTan and I just haven't added the parameters correctly, or is it a limitation in the API?

added_obj JSON
{
"_type": "question",
"group": {
"_type": "group",
"filters": {
"_type": "filters",
"filter": [
{
"_type": "filter",
"not_flag": 0,
"operator": "RegexMatch",
"sensor": {
"_type": "sensor",
"id": 1295,
"parameters": {
"_type": "parameters",
"parameter": [
{
"_type": "parameter",
"key": "||fileName||",
"value": "b32189bdff6e577a92baa61ad49264e6"
}
]
},
"source_id": 1295
},
"value": ".Yes."
}
]
}
},
"selects": {
"_type": "selects",
"select": [
{
"_type": "select",
"filter": {
"_type": "filter",
"sensor": {
"_type": "sensor",
"hash": 3409330187
}
},
"sensor": {
"_type": "sensor",
"hash": 3409330187
}
}
]
}
}

diff --git a/lib/pytan/handler.py b/lib/pytan/handler.py
index 9b396c7..dddf302 100755
--- a/lib/pytan/handler.py
+++ b/lib/pytan/handler.py
@@ -3285,6 +3285,7 @@ class Handler(object):
'pytan_help',
'handler',
'sse',

•        'params',
   ]
   clean_kwargs = pytan.utils.clean_kwargs(kwargs=kwargs, keys=clean_keys)
  

@@ -3350,6 +3351,8 @@ class Handler(object):
# add our Question and get a Question ID back
h = "Issue an AddObject to add a Question object"
added_obj = self._add(obj=add_obj, pytan_help=h, **clean_kwargs)

•    print "added_obj JSON"
  

•    print added_obj.to_json(add_obj)
  
   m = "Question Added, ID: {}, query text: {!r}, expires: {}".format
   self.mylog.debug(m(added_obj.id, added_obj.query_text, added_obj.expiration))
  

  diff --git a/lib/pytan/utils.py b/lib/pytan/utils.py
  index 7cb521b..0b59a95 100644
  --- a/lib/pytan/utils.py
  +++ b/lib/pytan/utils.py
  @@ -604,6 +604,7 @@ def dehumanize_question_filters(question_filters):
  question_filter_defs = []
  for question_filter in question_filters:
  s, parsed_selector = extract_selector(question_filter)

•    s, parsed_params = extract_params(s)
   s, parsed_filter = extract_filter(s)
   if not parsed_filter:
       err = "Filter {!r} is not a valid filter!".format
  

  @@ -611,6 +612,7 @@ def dehumanize_question_filters(question_filters):

   question_filter_def = {}
   question_filter_def[parsed_selector] = s
  

•    question_filter_def['params'] = parsed_params
   question_filter_def['filter'] = parsed_filter
  
   dbg = (
  

  @@ -1235,6 +1237,7 @@ def build_group_obj(q_filter_defs, q_option_defs):
  for d in q_filter_defs:
  # validate/map question filter into a Filter()
  filter_obj = get_filter_obj(d)

•    print "filter_obj JSON:", filter_obj.to_json(filter_obj)
  
   # update filter_obj with any options
   filter_obj = apply_options_obj(q_option_defs, filter_obj, 'filter')
  

  @@ -1514,7 +1517,23 @@ def get_filter_obj(sensor_def):

  create our basic filter that is needed no matter what

  filter_obj = taniumpy.Filter()
  filter_obj.sensor = taniumpy.Sensor()

• filter_obj.sensor.hash = sensor_obj.hash

• user_params = sensor_def.get('params', {})
• param_objlist = build_param_objlist(

•    obj=filter_obj.sensor,
  

•    user_params=user_params,
  

•    delim='||',
  

•    derive_def=True,
  

•    empty_ok=True
  

• )
• print "PARAMS:", param_objlist.to_json(param_objlist)

• if param_objlist:

•    filter_obj.sensor.source_id = sensor_obj.id
  

•    filter_obj.sensor.parameters = param_objlist
  

•    filter_obj.sensor.id = sensor_obj.id
  

• else:

•    filter_obj.sensor.hash = sensor_obj.hash
  

  get the filter the user supplied

  filter_def = sensor_def.get('filter', {})

[lifehackjim]

I'll look into this further next week, although I'll be travelling so will have limited time to do so. Something that would help me when I can look into this fully is the sensor object for the sensor "Index Query File Exists". A JSON export via get_sensor.py would be ideal.

[superponible]

I can probably get that if needed, but I think the issue applies to any sensor's parameters when used as a filter.

./ask_manual.py -s 'Computer Name' -f 'Installed Application Version{application=Google}, that contains:40'
++ Asked Question 'Get Computer Name from all machines with Installed Application Version containing "40"' ID: 2378233

I think the changes I made above show that I got it to include the parameters all the way up until it sends the request to the server, but I didn't look at what it actually POSTs over the network yet. In the Tanium console, if I view the question history, it also shows up there without the parameters. So at some point it's getting stripped, but I'm not sure where.

[lotekdan]

Might be worth following it through shell mode and reviewing the body at each point.

[superponible]

Looks like the parameter in the SOAP request. This is from printing request_body in pytan/session.py's add() method.

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SOAP-ENV:Body>
  <typens:tanium_soap_request xmlns:typens="urn:TaniumSOAP">
    <command>AddObject</command>
    <object_list><question><group><filters><filter><not_flag>0</not_flag><value>.*Yes.*</value><operator>RegexMatch</operator><sensor><source_id>1295</source_id><id>1295</id><parameters><parameter><value>b32189bdff6e577a92baa61ad49264e6</value><key>||fileMD5Hash||</key></parameter></parameters></sensor></filter></filters></group><selects><select><filter><sensor><hash>3409330187</hash></sensor></filter><sensor><hash>3409330187</hash></sensor></select></selects></question></object_list>
    <options><suppress_object_list>1</suppress_object_list></options>
  </typens:tanium_soap_request>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

[superponible]

As an alternative, I tried to see if ask_parsed.py would work. It's in the request, but the suggested questions aren't using the parameters:

./ask_parsed.py -q 'GET Computer Name FROM machines WITH Index Query File Exists[, , , b32189bdff6e577a92baa61ad49264e6, , , ] containing "Yes"'


<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SOAP-ENV:Body>
  <typens:tanium_soap_request xmlns:typens="urn:TaniumSOAP">
    <command>AddObject</command>
    <object_list><parse_job><parser_version>2</parser_version><question_text>GET Computer Name FROM machines WITH Index Query File Exists[, , , b32189bdff6e577a92baa61ad49264e6, , , ] containing "Yes"</question_text></parse_job></object_list>
    <options><export_format>csv</export_format></options>
  </typens:tanium_soap_request>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

2016-08-05 18:32:39,246 CRITICAL pytan.handler: You must supply an index as picker=$index to choose one of the parse responses -- re-run ask_parsed with picker set to one of these indexes!!
2016-08-05 18:32:39,247 CRITICAL pytan.handler: Index 1, Score: 6231, Query: 'Get Computer Name from machines with Index Query File Exists containing "b32189bdff6e577a92baa61ad49264e6,"'
2016-08-05 18:32:39,247 CRITICAL pytan.handler: Index 2, Score: 1852, Query: 'Get Computer Name from machines with Index Query File Exists containing "b32189bdff6e577a92baa61ad49264e6"'
2016-08-05 18:32:39,247 CRITICAL pytan.handler: Index 3, Score: 550, Query: 'Get Computer Name from machines with Index Query File Exists containing "b32189bdff6e577a92baa61ad49264e"'
2016-08-05 18:32:39,247 CRITICAL pytan.handler: Index 4, Score: 339, Query: 'Get BIOS Name from machines with Index Query File Exists containing "b32189bdff6e577a92baa61ad49264e6,"'
2016-08-05 18:32:39,247 CRITICAL pytan.handler: Index 5, Score: 239, Query: 'Get Domain Name from machines with Index Query File Exists containing "b32189bdff6e577a92baa61ad49264e6,"'

<snip>

I also printed the servers response to this and the question_text still had the parameters:

<question_text>GET Computer Name FROM machines WITH Index Query File Exists[, , , b32189bdff6e577a92baa61ad49264e6, , , ] containing &quot;Yes&quot;</question_text>

[lotekdan]

This is a known issue with ask_parsed and how the parser API handles request bodies. A work around has been added to the next release already. Jim and I will look into the original issue as soon as time permits.

[lotekdan]

Parameters work in filters now with manual questions. I've not ported the ask_parsed functionality.