WIP Compile and cache Regexes for SUMIFS and COUNTIFS

[igitur]

Follow on from #734. Possible performance improvement.

[igitur]

Doesn't seem to make that much of a difference. I suspected it would - especially for whole-column lookups.

[Pankraty]

This way of caching is not thread-safe, but since the entire calculation engine is not thread-safe too I don't think it's a problem. As for the performance of column-wide computations, I am currently experimenting with the alternative approach that does not require enumeration of empty values. After all, ranges in formulas may cover the whole sheet, and there is not any possibility to enumerate 17 billion values in reasonable time. If I succeed with this optimization the effect of the current PR will become more significant.

[igitur]

@Pankraty Do you think we should first quantify the effect on performance before I merge?

[Pankraty]

If you have time to measure that would be great. This will give us more information for future optimizations.

[Pankraty]

I finally managed to do a little performance testing. Unfortunately, the effect is negligible, if any. Here is the test file I used.

SumIfs.xlsx

Reading I1 value would take ~13 seconds, reading I2 - around 112 seconds, both with this PR and without it. As for I3 and I4, I gave up before the evaluation finished.

Will check out if there are other ways to speed up the computation.

PS. For testing, I rebased this branch on latest develop

[igitur]

Yeah, I was thinking to close this eventually without merging.

[jahav]

I am closing this PR, because the regexps are not a good way to parse criteria for SUMIF/COUNTIF/database functions.

What is a better way forward: Use a custom parser. There already is a allocation free fast parser for wildcards (Wildcard.Search), as well as proper parser of numbers ScalarValue.TextToNumber (kind of awkward place, but it works), as well as XLErrorParser and few others.

A custom parser will parse the criteria into a comparison op and a value/bool/error/wildcard using other parsers that already exist. Benefits:

• More in-line with excel criteria (e.g. ">1 1/2" is a valid criteria in Excel that finds only values greater than 1.5)
• No startup cost for compiling
• IIRC wildcard was faster than regex
• Less dangerous from DoS point of view

Current approach has several drawbacks:

• Potential denial of service, there is a reason why regex match function has optional timeout argument. Regex is a basically a program and attacker could supply a very slow program.
• Requires allocations
• Is mismatched with Excel