Fix issue 20493 - incorrect result of BigInt * BigInt

[dkorpel]

The problem comes down to a buffer overflow.
How could a buffer overflow happen in @safe code?
Well, it's basically a logical buffer overflow, while technically no out of bounds memory is accessed.

Two buffers are created in one allocation of size a + b (presumably as an optimization).
buffer[a..$] is given to partial, while scratchbuf remains the whole buffer[0..$] instead of buffer[0..a].
That means that if more than a elements are written to scratchbuf, it messes up partial, which is what happens in the bug report.

The buffer size for Karatsuba multiplication of x * y is calculated as 2 * x.length (x and y are a BigDigit[] which is a uint[]).
Most calls to mulKaratsuba claim 2 * half(x.length) of scratchbuf and pass the rest to a recursive call of half the instance size. I think the underlying assumption is that the geometric series 1 + 0.5 + 0.25 ... <= 2 so the bound is sufficient.

It fails to account for two things however:

• The problem size is integer, and when it is halved, it is rounded up. So when x and y have length 641, the next problem size is 321, then 161, 81 etc. Every time a little more than half of the available space is claimed.
• When the multiplication of the recursive case is unbalanced (i.e. x.length is more than y.length * sqrt(2)), another half of the buffer is claimed.

So why wasn't this caught earlier? Well, there is a KARATSUBALIMIT currently set to 10, and a BigDigit[] with length < KARATSUBALIMIT gets a small size optimization so no Karatsuba multiplication is done. This limit gave some constant leeway to the buffer. When numbers get sufficiently large, this asymptotic growth of the required buffer size catches up with the constant leeway of the actual buffer size with lower asymptotic growth however.

The smallest case of this bug with the current limit is when (x.length, y.length) = (116, 99), so this bug only manifests once numbers reach a size of 2^(32*116) ~= 10^1117 which is probably why it got undetected for so long.

Finding the correct minimum buffer size is hard, since it depends on KARATSUBALIMIT (for base cases), log2(x.length) (for rounding errors), and how often the recursive case is unbalanced.
With some brute force testing I found that a little over factor 2 seems sufficient for at least 10 000 big digits even with a smaller KARATSUBALIMIT of 2, so I settled with 2.25 for now.

I also truncated the scratchbuff so that at least instead of silently giving an incorrect answer, it raises a RangeError on failure.

[dlang-bot]

Thanks for your pull request and interest in making D better, @dkorpel! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Auto-close	Bugzilla	Severity	Description
✓	20493	normal	Incorrect result of BigInt * BigInt

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + phobos#7349"

[thewilsonator]

Ping me when you're happy with this TODO

[dkorpel]

I was hoping to leave that for a future PR. The comment is there to explain that the formula is a bit arbitrary and not the result of rigorous mathematical analysis or benchmarking, but it should be good enough.