Unable to traverse sorted and filtered results as expected

[cdent]

I was working on adding jsonpath-rw-ext in gabbi (to use sorts and filters) and during my testing discovered that thigngs weren't working as expected. I want to be able to traverse a sorted or filtered result to then use those results.

So I made a branch to demo things but I'm not sure what the correct way things out to work.

The goal is to know how to articulate tests like this:

    - name: test sorted list
      url: /foobar
      method: POST
      request_headers:
        content-type: application/json
      data:
          - alpha: one
            beta: horse
          - alpha: two
            beta: cow
      response_json_paths:
          $[/alpha].[0].beta: horse
          $[\alpha].[0].beta: cow

    - name: test sorted dict
      url: /foobar
      method: POST
      request_headers:
        content-type: application/json
      data:
          objects:
              - alpha: one
                beta: horse
              - alpha: two
                beta: cow
      response_json_paths:
          $.objects[/alpha].[0].beta: horse
          $.objects[\alpha].[0].beta: cow

Having _iterable.SortedThis.find be:

    def find(self, datum):
        """Return sorted value of This if list or dict."""
        if isinstance(datum.value, dict) and self.expressions:
            return datum

        import sys
        sys.stderr.write('datam: %s\n' % datum)
        sys.stderr.write('exp: %s\n' % self.expressions)
        if isinstance(datum.value, dict) or isinstance(datum.value, list):
            key = (functools.cmp_to_key(self._compare)
                   if self.expressions else None)
            return_value = [jsonpath_rw.DatumInContext(value, context=datum)
                    for value in sorted(datum.value, key=key)]

            return_value = [value.value for value in return_value]
            sys.stderr.write('return: %s\n' % return_value)
            return [jsonpath_rw.DatumInContext(return_value)]
        return datum

gets desired results but breaks the tests here. Adjusting the tests to try and work with this breaks filter and other extensions.

From what I can tell the basic underlying bug is that the thing being returned from find() in the extensions is not properly structured to be used for continued processing. Making it a list containing a datum which itself has the expected values sort of does the right thing but I'm insufficiently familiar with the parse tree (at least in tonight's exploration) to know what to do. Halp!

[cdent]

Here's a simple test case in gabbi language:

    - name: test non atomic
      POST: /foobar
      request_headers:
        content-type: application/json
      data:
          - two
          - one
      response_json_paths:
          $: ['two', 'one']
          $.`sorted`: ['one', 'two']
          $[0]: 'two'
          # Above works this fails
          $.`sorted`[0]: 'one'

Error message is AssertionError: Unable to match $.sorted.[0] as one, got [u'o', u't'], which is the first character of both, so we've skipped a step in the context.

[sileht]

$.[*][0] and $[0:2][0] have the same issue... That infortunatly an deeper issue of how jsonpath_rw work.

When you use $ jsonpath_rw return [DatumInContext(value=['two', 'one'])]
Then when [0:2], [*], sorted is used that returns [DatumInContext(value='two'), DatumInContext(value='one')]

So this is logic that next [0] returns the first char in each string returned by [*], sorted, ...

If I change Sorted to returns the same format as '$' that will break existing gabbi tests.
Because $.sorted will returns: [['one', 'two']] instead ['one', 'two'].

[cdent]

I've got a change pending in gabbi to adjust how it deals with multiple matches from a find that addresses Sorted's find() returning things in a list context (which is required for continued traversal). When I get on the machine that has it (I forgot to commit it anywhere) I'll show it here (sometime today).

It sounds like, however, that what you're saying is that what I want isn't possible? I'm not clear.

[cdent]

Here's the diff to gabbi that seems to allow my sorted changes to work as desired:

--- a/gabbi/case.py
+++ b/gabbi/case.py
@@ -198,9 +198,16 @@ class HTTPTestCase(unittest.TestCase):
         """
         path_expr = json_parser.parse(path)
         matches = [match.value for match in path_expr.find(data)]
-        try:
-            return matches[0]
-        except IndexError:
+        import sys
+        sys.stderr.write('matches: %s\n' % matches)
+
+        match_length = len(matches)
+        if match_length:
+            if match_length > 1:
+                return matches
+            else:
+                return matches[0]
+        else:
             raise ValueError(
                 "JSONPath '%s' failed to match on data: '%s'" % (path, data))

[cdent]

With regard to $[*][0], that's a conundrum too but I could survive without that working since $[0] gets the same result...