Bug: Case for date for term.key missing in onSearch function

[titanism]

Right now the only two keys related to date supported for search are internaldate and headerdate. If you are using imap-flow to search a mailbox, then the date key is not supported:

  // dates
  for (const key of [
    'before',
    'on',
    'since',
    'sentBefore',
    'sentOn',
    'sentSince'
  ]) {
    await imapFlow.search({ [key]: new Date(Date.now() + 10000) });
  }

Ref:

wildduck/lib/handlers/on-search.js

Lines 204 to 288 in ea24b93

	case 'internaldate':
	{
	let op = false;
	let value = new Date(term.value + ' GMT');
	switch (term.operator) {
	case '<':
	op = '$lt';
	break;
	case '<=':
	op = '$lte';
	break;
	case '>':
	op = '$gt';
	break;
	case '>=':
	op = '$gte';
	break;
	}
	let entry = !op
	? [
	{
	$gte: value
	},
	{
	$lt: new Date(value.getTime() + 24 * 3600 * 1000)
	}
	]
	: {
	[op]: value
	};
	entry = {
	idate: !ne
	? entry
	: {
	$not: entry
	}
	};
	parent.push(entry);
	}
	break;
	case 'headerdate':
	{
	let op = false;
	let value = new Date(term.value + ' GMT');
	switch (term.operator) {
	case '<':
	op = '$lt';
	break;
	case '<=':
	op = '$lte';
	break;
	case '>':
	op = '$gt';
	break;
	case '>=':
	op = '$gte';
	break;
	}
	let entry = !op
	? [
	{
	$gte: value
	},
	{
	$lt: new Date(value.getTime() + 24 * 3600 * 1000)
	}
	]
	: {
	[op]: value
	};
	entry = {
	hdate: !ne
	? entry
	: {
	$not: entry
	}
	};
	parent.push(entry);
	}
	break;

[titanism]

One thing we did to simplify things:

case 'headerdate':
case 'internaldate':
case 'date': {
  // .. logic (see below too)
}

...

// and then in the case statement just use the corresponding property to `term.key`:

// headerdate => hdate
// internaldate => idate
// date => date
let entryKey;
switch (term.key) {
  case 'headerdate': {
    entryKey = 'hdate';
                                                    
    break;
  }
                                                    
  case 'internaldate': {
    entryKey = 'idate';
                                                    
    break;
  }
                                                    
  case 'date': {
    entryKey = 'date';
                                                    
    break;
  }
                                                    
  default: {
    throw new TypeError(`${term.key} unsupported`);
  }
}

And then implement as:

         entry = { 
+             [entryKey]: !ne 
-             idate: !ne 
                 ? entry 
                 : { 
                       $not: entry 
                   } 
         }; 

[titanism]

Ignore previous comment regarding = operator missing. The only bug is the one above with date missing.

[titanism]

Actually, instead of #560 (comment) we would need to do an $or query against idate and hdate for a date query.

[titanism]

Our final (working) solution:

              // headerdate => hdate
              // internaldate => idate
              // date => $or
              // <https://github.com/nodemailer/wildduck/issues/560>
              switch (term.key) {
                case 'headerdate': {
                  entry = {
                    hdate: ne
                      ? {
                          $not: entry
                        }
                      : entry
                  };
                  break;
                }

                case 'internaldate': {
                  entry = {
                    idate: ne
                      ? {
                          $not: entry
                        }
                      : entry
                  };

                  break;
                }

                case 'date': {
                  entry = {
                    $or: [
                      {
                        hdate: ne
                          ? {
                              $not: entry
                            }
                          : entry
                      },
                      {
                        idate: ne
                          ? {
                              $not: entry
                            }
                          : entry
                      }
                    ]
                  };

                  break;
                }

                default: {
                  throw new TypeError(`${term.key} unsupported`);
                }
              }

              parent.push(entry);

[andris9]

Did you check your implementation against rfc3501#6.4.4? It is pretty specific on how to compare dates when searching.

[titanism]

WildDuck is already parsing these and setting the key as date, it's just the handler doesn't account for date key, only internaldate and headerdate. It doesn't look like headerdate is actually used right now (is that a separate bug)?

❯ rg "headerdate"
lib/handlers/on-search.js
247:                        case 'headerdate':

You can see the mapping right now maps sentbefore, senton, and sentsince to "date" key name. There is no mapping right now for "headerdate" and there's no case statement in the onSearch handler in WildDuck for "date" yet (as per above).

[andris9]

Oh, I see it now, instead of case 'headerdate': there should be case 'date': and this would fix the senton/sentsince/sentbefore queries. No-one really uses these, and IMAP search in general, is extremely unreliable, so clients usually use as simple search queries as possible.

[titanism]

PR submitted at #561