Better filtering

[Ninjani]

• Add quote and text filter options
• Add an option to filter page notes.

See some comments in #89 here

[Ninjani]

Also add

• a way to filter annotations not containing a tag
• a way to filter annotations containing any of the given tags (instead of all)

[ngirard]

• a way to filter annotations not containing a tag
• a way to filter annotations containing any of the given tags (instead of all)

These ones might be tricky to implement on the CLI side. I'll try to come up with source code to take inspiration from.
Tonight I'm clueless. Fselect does that, but it leverages its own parser/lexer and it's perhaps too much.
I'll report back when I find something simpler.

[ngirard]

Oh yeah, and there's obviously Ripgrep, but as its author said recently on r/rust, the source code isn't always straightforward.
Still, it might be worth taking a look to the code behind the --type and --type-not options.

[ngirard]

Aha. Got one. Watchexec does this in a very straightforward way:

OPTIONS:
    (...)
    -f, --filter <pattern>...     Ignore all modifications except those matching the pattern
    -i, --ignore <pattern>...     Ignore modifications to paths matching the pattern

[Ninjani]

-i, --ignore <pattern>...     Ignore modifications to paths matching the pattern

Yeah, I like this one too! I'm thinking something like --not instead of ignore since -i is already taken by --include-updated. This would easily solve the first point (a way to filter annotations not containing a tag), and in general allow filtering annotations not matching a certain critera.

For the second point (a way to filter annotations containing any of the given tags (instead of all)) I'm thinking of an --or flag that flips this behaviour. Any other ideas?

[ngirard]

Using --and / --or / --not would allow for maximum flexibility for constructing complex queries, which I think the problem at hand deserves, and it would be great.

If you're willing to take that route, you'll probably also need to add parentheses, like GNU find (see the OPERATORS section) or Git-annex do.

I just cannot think of relevant code in Rust at the moment.

[ngirard]

Jeez... that was fast !

I'll test this as soon as I can, by tomorrow in any case.

[ngirard]

I couldn't resist and compiled the branch.

I guess you're aware of this, but just in case:
searching --tags x --not --tags y seems to return the union of the sets of 1) annotations tagged x ; 2) annotations not tagged y.
I'd expect the result to be the set of annotations tagged x and not tagged y.

Cheers

[Ninjani]

Ooh I didn't actually think of that, right now the --not applies to the entire query, and it'll return annotations not tagged with both x and y. Will dive a bit deeper into clap to see how to parse multiple occurrences of an argument separately and also make use of the argument order.

[ngirard]

Hence the parentheses...
Yeah, taking a look to an existing implementation with clap would help.

[Ninjani]

Looks like clap doesn't really have support for things like this. The only example I could find is here where they write their own parsing code for filter arguments. I'm considering if it's worth the effort or if there's a simpler workaround on top of the current PR.

One option is to add an exclude_tags option, which would make it easy to select annotations containing a certain (set of) tag(s) x and not containing another (set of) tag(s) y, using gooseberry search --tags x --exclude-tags y. Using this with --or would return annotations with ANY of x and none of y, otherwise it'd be ALL of x and none of y. With --not you'd get ANY of y and not ALL of x (not sure what the use case for that would be but still nice to have)

For everything else I think the current PR and the search box should help with more complex queries - to make this smoother I'll add another keyboard shortcut to search (probably Shift-Down) which runs make on the filtered annotations (this way search would pretty much be the only command needed to run everything), and I'll add a --search option to the make command as well.

Do you have any other queries/situations in mind that need extra handling? The only one I can think of that can't be handled by search is to find annotations within a certain time period but not on a certain day within that time period or something like that, which can be solved with two searches instead of one (not ideal of course but not a dealbreaker I'd say)

[ngirard]

The only example I could find is here where they write their own parsing code for filter arguments.

Ha ! Good catch, I didn't dig that deep.

One option is to add an exclude_tags option (...) to make this smoother I'll add another keyboard shortcut to search (...)

This would indeed be the most simple to implement, and it seems reasonable.

Do you have any other queries/situations in mind that need extra handling?

I feel like I need to be using the tool to be able to confront it with different situations and answer your question properly; but I'm sure that simply adding such --exclude-tags semantics would be enough to cover plenty of them -- if not all.

[Ninjani]

Great! On it then 😄

[ngirard]

Thanks, and godspeed to you !