bpo-41972: Tweak fastsearch.h string search algorithms

[sweeneyde]

https://bugs.python.org/issue41972

[ambv]

I admit going through _two_way with the double loops and gotos caused a little head-spinning. I'll run some Hypothesis tests on the new version.

Note that this can only go to 3.11 now, as it doesn't qualify as a bug fix and we're past the beta stage for 3.10.

[ambv]

How did you come about those numbers?

[sweeneyde]

I'll consolidate the benchmarks I used and post them here. But this is a chart comparing the default implementation to the existing two-way implementation. The equivalent chart for this implementation is similar, and I'm trying to essentially to cut out the green bits of that image.

For now though, the differences this PR makes have some benchmarks here

[sweeneyde]

There's an overview of the two-way algorithm at https://github.com/python/cpython/blob/main/Objects/stringlib/stringlib_find_two_way_notes.txt

If you want to be able to test more rigorously, you could make the table and its contents smaller with something like this:

#define MAX_SHIFT 3

#define TABLE_SIZE_BITS 2u

[sweeneyde]

Here's the chart of the ratios of the runtimes of the two algorithms, with cyan/black text to distinguish where these thresholds lie. This was run using zipf_benchmarks.py in the gist I posted before.

[ambv]

I was running Hypothesis smoke tests on this PR for most of today and all's well.

Do the benchmarks you provided include the changes in your last two commits?

[sweeneyde]

I re-ran some benchmarks comparing the main branch to commit 4d7d102 and they're here: https://gist.github.com/sweeneyde/c370dcee453a2bcb34157261fb79650e

These use the same benchmarks as I posted before: one randomly generated character-by-character with a zipf distribution, and then a set of benchmarks comparing binary data, C code, Python code, and RestructuredText.

Summary of random string results:
* Geometric mean: 1.05x faster
* 542 cases faster (at best 3.45x faster)
* 166 cases slower (at worst 1.73x slower)

Summary of "real life" benchmarks:
* Geometric mean: 1.22x faster
* 372 cases faster (at best 14.12x faster)
* 129 cases slower (at worst 3.95x slower)

(My most recent commits only tweaked the cutoffs, and the table in my last comment ran with no run-time cutoffs.)

[ambv]

The recent results still look pretty good but are visibly worse from the ones you presented on July 12 on the issue. What do you think the reason for that is?

[sweeneyde]

The recent results still look pretty good but are visibly worse from the ones you presented on July 12 on the issue. What do you think the reason for that is?

There was a bug in my original implementation where it could access past the end of the haystack buffer. The new and old assembly is below (Microsoft Visual C++ 2019, Release x64 Build).

Assembly for commit 4d7d102 (it looks like it unrolled x2):

   440:             for (;;) {
   441:                 LOG_LINEUP();
   442:                 Py_ssize_t shift = table[(*window_last) & TABLE_MASK];
00007FF937A5BFA0  movsx       rax,byte ptr [r9]  
00007FF937A5BFA4  and         eax,3Fh  
00007FF937A5BFA7  movzx       ecx,byte ptr [rax+r15]  
   443:                 window_last += shift;
00007FF937A5BFAC  add         r9,rcx  
   444:                 if (shift == 0) {
00007FF937A5BFAF  test        rcx,rcx  
00007FF937A5BFB2  je          $windowloop+2Dh (07FF937A5BFCDh)  
   445:                     break;
   446:                 }
   447:                 if (window_last >= haystack_end) {
00007FF937A5BFB4  cmp         r9,rbp  
00007FF937A5BFB7  jae         stringlib__two_way+13Ch (07FF937A5BF4Ch)  
   440:             for (;;) {
   441:                 LOG_LINEUP();
   442:                 Py_ssize_t shift = table[(*window_last) & TABLE_MASK];
00007FF937A5BFB9  movsx       rax,byte ptr [r9]  
00007FF937A5BFBD  and         eax,3Fh  
00007FF937A5BFC0  movzx       ecx,byte ptr [rax+r15]  
   443:                 window_last += shift;
00007FF937A5BFC5  add         r9,rcx  
   444:                 if (shift == 0) {
00007FF937A5BFC8  test        rcx,rcx  
00007FF937A5BFCB  jne         $windowloop+14h (07FF937A5BFB4h)  
   448:                     return -1;
   449:                 }
   450:                 LOG("Horspool skip");
   451:             }

Old Assembly from commit
d3f8438 (incorrect, accessed past the end of the haystack buffer):

   432:             while (shift > 0 && window_last < haystack_end) {
00007FF93489BF6C  test        rdx,rdx  
00007FF93489BF6F  je          $windowloop+33h (07FF93489BF93h)  
00007FF93489BF71  cmp         rax,rsi  
00007FF93489BF74  jae         stringlib__two_way+13Dh (07FF93489BF0Dh)  
   433:                 LOG("Horspool skip.\n");
   434:                 window_last += shift;
00007FF93489BF76  add         rax,rdx  
   435:                 shift = table[(*window_last) & TABLE_MASK];
00007FF93489BF79  movsx       rcx,byte ptr [rax]  
00007FF93489BF7D  and         ecx,3Fh  
00007FF93489BF80  movzx       edx,byte ptr [rcx+r14]  
00007FF93489BF85  test        rdx,rdx  
00007FF93489BF88  jne         $windowloop+11h (07FF93489BF71h)  
   436:                 LOG_LINEUP();
   437:             }
   438:             if (window_last >= haystack_end) {
00007FF93489BF8A  cmp         rax,rsi  
00007FF93489BF8D  jae         stringlib__two_way+13Dh (07FF93489BF0Dh)  
   439:                 break; // return -1
   440:             }

It appears that the 10 Zipf results that slowed down the most between these versions all had needles that ended in 'D', which I had originally randomly set to be the most-frequently occurring character (about 25% of all characters). For needles that end in common characters, less time is spent in the Horspool skip table loop, so unrolling made things worse rather than better. Similarly, the benchmarks on binary data that slowed down the most all ended in null bytes, which were very common in the file I looked at.

I can look at reverting commit b8df3e1 and see how that changes things.

[sweeneyde]

Reverting commit b8df3e1 was a wash: "1.02x slower" on the zipf benchmarks and "1.00x faster" on the real-life benchmarks.

[sweeneyde]

Thanks @ambv , and congrats on the new role!

[ambv]

Thanks for the kind words, and thanks for your change!