Improve query interpretation format

[plajjan]

This changes the query interpretation format to a new and improved format that is able to correctly model a query. The old format is a list with an entry for each element, which doesn't allow correctly grouping a more complex query.

For example, the query 'a b c' means we have three elements; a, b and c. There is an implicit boolean AND operator between them and so the query interpretation could be correctly represented with just a list and an implicit AND operator. However, as we wish to support a query like 'a AND (b or c)' we now have a different operator and it is (explicitly) grouped in a way that isn't possible to express with a flat list. The new format follows the query format, ie it is a hierarchy and the new query interpretation format can thus express all possible queries.

[plajjan]

@garberg I'm not sure we should actually merge this just yet but I want your feedback on the look of it at least. I have updated the web UI to display the new format but it isn't very visually pleasing so before merging to master we might want to revisit that and perhaps write something with AngularJS.

[plajjan]

I realize I should perhaps comment on the structure, because while the interpretation format can closely follow the query hierarchy it doesn't have to. In fact, in many cases it is desirable that it does not.

Performing a smart_search_pool on the string 'foo' will result in a query that looks like this:

'operator': 'or',
'val1': {
   'operator': 'regex',
   'val1': 'name',
   'val2': 'foo'
},
'val2': {
   'operator': 'regex',
   'val1': 'description',
   'val2': 'foo'
}

And inserting the query interpretation in this, it could look like this:

'operator': 'or',
'val1': {
    'interpretation': {
        'attribute': 'name',
        'interpretation': 'text',
        'operator': 'regex',
        'string': 'foo'
    },
    'operator': 'regex',
    'val1': 'name',
    'val2': 'foo'
},
'val2': {
    'interpretation': {
        'attribute': 'name',
        'interpretation': 'text',
        'operator': 'regex',
        'string': 'foo'
    },
    'operator': 'regex',
    'val1': 'description',
    'val2': 'foo'
}

Which in turn would generate two interpretation lines in the CLI or web UI.

The alternative is this:

'interpretation': {
    'attribute': 'name or description',
    'interpretation': 'text',
    'operator': 'regex',
    'string': 'foo'
},
'operator': 'or',
'val1': {
    'operator': 'regex',
    'val1': 'name',
    'val2': 'foo'
},
'val2': {
    'operator': 'regex',
    'val1': 'description',
    'val2': 'foo'
}

Ie, we let one interpretation node describe a deeper structure so that it a bit briefer. This is often the preferred way.

[garberg]

The only comment I have is regarding the different keys in the interpretation-dict which varies a bit in the smart_search_prefix-function. Depending on the input the interpretation sometimes has an expanded-key and/or a strict_prefix-key, but as this is not introduced in this pull request it's not really relevant here.

As soon as the interpretation is displayed prettier in the web UI I'm happy with merging this pull request.

[plajjan]

What do you think of this format for the CLI:

kll@lingloi320 ~/kod/NIPAP/nipap-cli $ ./nipap address list --show-interpretation a b c d e
Searching for prefixes in any VRF...
Query interpretation:
      AND-- e: text matching e
        `-- AND-- d: text matching d
              `-- AND-- c: text matching c
                    `-- AND-- b: text matching b
                          `-- a: text matching a
No addresses matching 'a b c d e' found.

As compared to what the PR currently renders:

kll@lingloi320 ~/kod/NIPAP/nipap-cli $ ./nipap address list --show-interpretation a b c d e
Searching for prefixes in any VRF...
Query interpretation:
 - AND
   - e: text matching e
   - AND
     - d: text matching d
     - AND
       - c: text matching c
       - AND
         - b: text matching b
         - a: text matching a
No addresses matching 'a b c d e' found.

I quite like it. It's more compact (not width-wise though) and is easier to read.

[plajjan]

Okay, here's a new version of the interpretation in the web UI

Compared to the "old" way (ie, not git master, but as currently suggested in this patch).

I don't think there's any doubt on that the new format is way better. Does it need further improvement or is this enough?

I know alignment is a little bit off but this is a hack. I really don't want to spend more time on it right now. It should be rewritten as AngularJS thingy anyway.

@garberg feedback plxz. This is the last thing for this PR. Please comment on looks of interpretation in CLI (previous comment) and web UI. If we are happy, I'll update the PR and we should be good to merge.

[garberg]

Looks great! The hierarchies are quite deep, but that's just motivation to build the support for multi-parameter operators (so only one AND-operator is needed for multiple values) we've spoken about for so long time 😄

[plajjan]

Okay, did a slight rebase so had to force push.

@garberg This should now be ready to go. Have a last read if you want and push the magic button ;)

[garberg]

k'POW!