fix: allow filters with ( ) and " delimiters in Boolean filters

[claremacrae]

Description

Major fix of Boolean filters, to allow use of (, ) and " in filters within Boolean expressions.

Note: more improvements are coming so I am not updating the documentation yet.

Caveat

The one current caveat with this implementation is that it still does not work for filters the end with any of the characters ()", as they are swallowed up and treated as delimiters.

So for example, this filter:

(description includes "maybe")

is interpreted as this simplified line - note the stray ":

(f1")

where f1 is this - note the missing ":

description includes "maybe

Workarounds

filter by function instructions are very likely to end with ), due to the expression often ending with a function call.

Available workarounds:

• add a trailing ; at the end of any filter by function that ends in ) or ".
• convert Boolean combinations of multiple filter by function to be a single filter by function that uses JavaScript &&, || and !.

Motivation and Context

• Fixes Allow parenthesis to be used in search strings in boolean filters #1308
• Fixes malformed boolean query for valid regex search that includes () #1500

How has this been tested?

• Automated tests
• Exploratory testing

Screenshots (if appropriate)

Examples of searches that now work, followed by their explain output:

Example 1 - containing a regular expression with ()

( description regex matches /(buy|order|voucher|lakeland|purchase|\spresent)/i ) OR ( path includes Home/Shopping )

explain output:

  ( description regex matches /(buy|order|voucher|lakeland|purchase|\spresent)/i ) OR ( path includes Home/Shopping ) =>
    OR (At least one of):
      description regex matches /(buy|order|voucher|lakeland|purchase|\spresent)/i =>
        using regex:            '(buy|order|voucher|lakeland|purchase|\spresent)' with flag 'i'
      path includes Home/Shopping

Example 2

(path includes (some example) OR (path includes )some example()

explain output:

  (path includes (some example) OR (path includes )some example() =>
    OR (At least one of):
      path includes (some example
      path includes )some example(

Example 3 - complex combination of custom filters

Important: Note that the following only works because every filter expression that ends with ) is followed by a ;, to prevent the ) being wrongly treated as part of the Boolean logic.

( filter by function ! 'NON_TASK,CANCELLED'.includes(task.status.type); ) OR ( filter by function const date = task.due.moment; return date ? !date.isValid() : false; ) OR ( filter by function task.due.moment?.isSameOrBefore(moment(), 'day') || false; ) OR ( filter by function task.urgency.toFixed(2) === 1.95.toFixed(2); ) OR ( filter by function (!task.isRecurring) && task.originalMarkdown.includes('🔁'); ) OR ( filter by function task.file.path.toLocaleLowerCase() === 'TASKS RELEASES/4.1.0 RELEASE.MD'.toLocaleLowerCase(); ) OR ( filter by function const taskDate = task.due.moment; const now = moment(); return taskDate?.isSame(now, 'day') || ( !taskDate && task.heading?.includes(now.format('YYYY-MM-DD')) ) || false; ) OR ( filter by function const wanted = '#context/home'; return task.heading?.includes(wanted) || task.tags.find( (tag) => tag === wanted ) && true || false; )

explain output:

  ( filter by function ! 'NON_TASK,CANCELLED'.includes(task.status.type); ) OR ( filter by function const date = task.due.moment; return date ? !date.isValid() : false; ) OR ( filter by function task.due.moment?.isSameOrBefore(moment(), 'day') || false; ) OR ( filter by function task.urgency.toFixed(2) === 1.95.toFixed(2); ) OR ( filter by function (!task.isRecurring) && task.originalMarkdown.includes('🔁'); ) OR ( filter by function task.file.path.toLocaleLowerCase() === 'TASKS RELEASES/4.1.0 RELEASE.MD'.toLocaleLowerCase(); ) OR ( filter by function const taskDate = task.due.moment; const now = moment(); return taskDate?.isSame(now, 'day') || ( !taskDate && task.heading?.includes(now.format('YYYY-MM-DD')) ) || false; ) OR ( filter by function const wanted = '#context/home'; return task.heading?.includes(wanted) || task.tags.find( (tag) => tag === wanted ) && true || false; ) =>
    OR (At least one of):
      filter by function ! 'NON_TASK,CANCELLED'.includes(task.status.type);
      filter by function const date = task.due.moment; return date ? !date.isValid() : false;
      filter by function task.due.moment?.isSameOrBefore(moment(), 'day') || false;
      filter by function task.urgency.toFixed(2) === 1.95.toFixed(2);
      filter by function (!task.isRecurring) && task.originalMarkdown.includes('🔁');
      filter by function task.file.path.toLocaleLowerCase() === 'TASKS RELEASES/4.1.0 RELEASE.MD'.toLocaleLowerCase();
      filter by function const taskDate = task.due.moment; const now = moment(); return taskDate?.isSame(now, 'day') || ( !taskDate && task.heading?.includes(now.format('YYYY-MM-DD')) ) || false;
      filter by function const wanted = '#context/home'; return task.heading?.includes(wanted) || task.tags.find( (tag) => tag === wanted ) && true || false;

Types of changes

Changes visible to users:

• Bug fix (prefix: fix - non-breaking change which fixes an issue)
• Documentation (prefix: docs - improvements to any documentation content for users)

Internal changes:

• Tests (prefix: test - additions and improvements to unit tests and the smoke tests)

Checklist

• My code follows the code style of this project and passes yarn run lint.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
  • Will be done later
• My change has adequate Unit Test coverage.

Terms

• My contribution follow this project's contributing guide
• I agree to follow this project's Code of Conduct