Applying the output of one query to the input of another is broken

[scudette]

This might be a design problem with the efilter implementation. What is happening is that when Apply() is called it assigns a list if there is more that one result and a scalar if there only one result. This creates a situation where a query will fail depending on the number of results returned.

For example:

[1] Live (API) 00:50:29> select Hashes, Path from hash(paths: (select path from glob("/home/*/.ssh/*")).path.filename)
2017-08-06 00:50:29,880:DEBUG:rekall.1:Running plugin (search) with args (()) kwargs ({'query': u'select Hashes, Path from hash(paths: (select path from glob("/home/*/.ssh/*")).path.filename)'})
                    Hashes                                     Path                                                                                                                     
---------------------------------------------- ------------------------------------                                                                                                     
sha1 41b5e39b28e3c39ec9e376624b40a0c3544b6958  /home/scudette/.ssh/authorized_keys                                                                                                      
sha1 188e0f24996ee87b6082e786aacd48bb565fc997  /home/scudette/.ssh/id_rsa.pub                                                                                                           
sha1 ced1cfb8e52b7fc0a75df8bd13ac0466e8bad2a6  /home/scudette/.ssh/id_rsa                          

This works fine because the glob expands to three files and this produces a list which is passed to the paths arg of the hash plugin (which expects a list).

However the following query breaks:

[1] Live (API) 00:51:44> select Hashes, Path from hash(paths: (select path from glob("/home/*/.ssh/authorized_keys")).path.filename)
2017-08-06 00:51:44,181:DEBUG:rekall.1:Running plugin (search) with args (()) kwargs ({'query': u'select Hashes, Path from hash(paths: (select path from glob("/home/*/.ssh/authorized_k
eys")).path.filename)'})
2017-08-06 00:51:44,382:CRITICAL:rekall.1:Traceback (most recent call last):
  File "/home/scudette/rekall/rekall-core/rekall/session.py", line 862, in RunPlugin
    result = plugin_obj.render(ui_renderer) or plugin_obj
  File "/home/scudette/rekall/rekall-core/rekall/plugins/common/efilter_plugins/search.py", line 777, in render
    rows = self.collect() or []
  File "/home/scudette/rekall/rekall-core/rekall/plugins/common/efilter_plugins/search.py", line 677, in collect
    result = self.solve()
....
  File "/home/scudette/rekall/rekall-core/rekall/plugin.py", line 608, in __init__
    value, session=kwargs.get("session"))
  File "/home/scudette/rekall/rekall-core/rekall/plugin.py", line 156, in parse
    raise TypeError("Arg %s must be a list of strings" % self.name)
TypeError: Arg paths must be a list of strings

This happens because this time, glob returns one result and so hashes gets a single entry not a list (with one entry):

> /home/scudette/projects/dotty/efilter/transforms/solve.py(348)solve_apply()
    346             args.append(solve(arg, vars).value)
    347 
--> 348     result = applicative.apply(func, args, kwargs)
    349 
    350     return Result(result, ())

ipdb> p func
<CommandWrapper: 'IRHash'>
ipdb> p kwargs
{u'paths': <rekall.plugins.response.common.FileSpec object at 0x7f0994a70710>}
ipdb> p args
[]
ipdb> expr.args
(Pair(
 Var(u'paths')
 Resolve(
  Resolve(
   Map(
    Apply(
     Var(u'glob')
     Literal(u'/home/*/.ssh/authorized_keys'))
    Bind(Pair(
     Literal(u'path')
     Var(u'path'))))
   Literal(u'path'))
  Literal(u'filename'))),)

[scudette]

After more debugging I narrowed the issue down to:

https://github.com/google/dotty/blob/master/efilter/protocols/repeated.py#L152

def isrepeating(x):
    """Optional: Is x a repeated var AND does it have more than one value?"""
    return isinstance(x, IRepeated) and counted.count(x) > 1

If I comment out the counted.count part this fixes this issue but it causes a lot of breakage everywhere since it seems that everything becomes repeated now.