Optimize BigInteger.Divide

[kzrnm]

The current BigInteger.Divide, which is implemented in the "grammar-school" algorithm, runs in $O(n^2)$ where n is the number of digits. The implementation of the pull request is implemented in Burnikel and Ziegler's Fast Recursive Division algorithm. It runs in $O(n^{\log 3} + n \log n)$.

Burnikel-Ziegler Algorithm is used in other languages.

• Golang
• OpenJDK
• JavaScript-V8

---

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22631.3155/23H2/2023Update/SunValley3)
13th Gen Intel Core i5-13500, 1 CPU, 20 logical and 14 physical cores
.NET SDK 9.0.100-alpha.1.23615.4
  [Host]     : .NET 9.0.0 (9.0.23.61410), X64 RyuJIT AVX2
  Job-DGWAFF : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
  Job-ITISLD : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2

Method	Job	Toolchain	arguments	Mean	Error	StdDev	Ratio	RatioSD	Code Size	Gen0	Allocated	Alloc Ratio
Divide	Job-DGWAFF	\main\corerun.exe	1024,512 bits	298.2 ns	1.47 ns	1.38 ns	1.00	0.00	1,781 B	0.0143	184 B	1.00
Divide	Job-ITISLD	\pr\corerun.exe	1024,512 bits	312.6 ns	6.05 ns	8.68 ns	1.04	0.03	2,735 B	0.0143	184 B	1.00
Divide	Job-DGWAFF	\main\corerun.exe	262144,131072 bits	66,281,748.3 ns	1,185,267.18 ns	1,108,699.66 ns	1.00	0.00	5,584 B	-	32824 B	1.00
Divide	Job-ITISLD	\pr\corerun.exe	262144,131072 bits	3,749,637.7 ns	74,025.70 ns	69,243.68 ns	0.06	0.00	5,625 B	-	32816 B	1.00
Divide	Job-DGWAFF	\main\corerun.exe	65536,32768 bits	4,452,706.5 ns	81,142.27 ns	71,930.51 ns	1.00	0.00	5,385 B	-	8248 B	1.00
Divide	Job-ITISLD	\pr\corerun.exe	65536,32768 bits	452,050.8 ns	5,390.30 ns	5,042.09 ns	0.10	0.00	5,659 B	0.4883	8248 B	1.00

[ghost]

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The current BigInteger.Divide, which is implemented in the "grammar-school" algorithm, runs in $O(n^2)$ where n is the number of digits. The implementation of the pull request is implemented in Burnikel and Ziegler's Fast Recursive Division algorithm. It runs in $O(n^{\log 3} + n \log n)$.

Burnikel-Ziegler Algorithm is used in other languages.

• Golang
• OpenJDK
• JavaScript-V8

Author:	kzrnm
Assignees:	-
Labels:	area-System.Numerics, community-contribution
Milestone:	-

[tannergooding]

There's a few different BigInteger PRs open right now, this is on my list to get to but I want to try and get some of the other ones completed/merged first

[lskyum]

Will this optimized Divide also result in a faster GreatestCommonDivisor method?

I'm on the verge of rewriting a lot of C# code to Rust, only because of the BigInteger performance.
So if this PR helps (a lot) it would save me months of work :-)

[danmoseley]

@lskyum Just curious, what do you use big integer for?

[lskyum]

@danmoseley I'm using two BigInteger's to represent a "BigFraction", when performing various geometric calculations.
By using fractions it possible to guarentee 100% precision in contrast to, for example double's. The algorithms becomes much easier to reason about and it guarantees correctness. It is the same approach CGAL (https://www.cgal.org/) uses and it uses GMP for the integer/fraction calculations, which sadly is a lot faster than the C# equivalents.

[kzrnm]

@lskyum The division also affects a GCD method.

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22631.3007/23H2/2023Update/SunValley3)
13th Gen Intel Core i5-13500, 1 CPU, 20 logical and 14 physical cores
.NET SDK 8.0.101
  [Host]   : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
  ShortRun : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2

Job=ShortRun  Toolchain=.NET 8.0  IterationCount=3  
LaunchCount=1  WarmupCount=3  

Method	N	Mean	Error	StdDev	Ratio
Orig	10000	4.171 ms	0.7849 ms	0.0430 ms	1.00
New	10000	1.232 ms	0.1088 ms	0.0060 ms	0.30
Orig	100000	422.306 ms	225.2295 ms	12.3456 ms	1.00
New	100000	85.521 ms	9.6445 ms	0.5286 ms	0.20

    [GlobalSetup]
    public void Setup()
    {
        o1 = OrigBigInteger.Pow(3, N) * OrigBigInteger.Pow(5, N);
        o2 = OrigBigInteger.Pow(3, N) * OrigBigInteger.Pow(23, N);
        m1 = MyBigInteger.Pow(3, N) * MyBigInteger.Pow(5, N);
        m2 = MyBigInteger.Pow(3, N) * MyBigInteger.Pow(23, N);
    }

Environment: https://github.com/kzrnm/BigInteger/tree/main/Benchmark

Benchmark code

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Reports;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Toolchains.CsProj;

using MyBigInteger = Kzrnm.Numerics.BigInteger;
using OrigBigInteger = System.Numerics.BigInteger;

public class BenchmarkConfig : ManualConfig
{
    static void Main(string[] args)
    {
#if DEBUG
        BenchmarkSwitcher.FromAssembly(typeof(BenchmarkConfig).Assembly).Run(args, new DebugInProcessConfig());
#else
        _ = BenchmarkRunner.Run(typeof(BenchmarkConfig).Assembly);
#endif
    }
    public BenchmarkConfig()
    {
        AddExporter(BenchmarkDotNet.Exporters.MarkdownExporter.GitHub);
        AddJob(Job.ShortRun.WithToolchain(CsProjCoreToolchain.NetCoreApp80));

        SummaryStyle = SummaryStyle.Default
        .WithRatioStyle(BenchmarkDotNet.Columns.RatioStyle.Value)
        ;
    }
}


[Config(typeof(BenchmarkConfig))]
[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByParams)]
public class BigIntegerGcd
{
    [Params(10000, 100000)]
    public int N;
    public OrigBigInteger o1, o2;
    public MyBigInteger m1, m2;

    [GlobalSetup]
    public void Setup()
    {
        o1 = OrigBigInteger.Pow(3, N) * OrigBigInteger.Pow(5, N);
        o2 = OrigBigInteger.Pow(3, N) * OrigBigInteger.Pow(23, N);
        m1 = MyBigInteger.Pow(3, N) * MyBigInteger.Pow(5, N);
        m2 = MyBigInteger.Pow(3, N) * MyBigInteger.Pow(23, N);
    }

    [Benchmark(Baseline = true)]
    public OrigBigInteger Orig() => OrigBigInteger.GreatestCommonDivisor(o1, o2);

    [Benchmark]
    public MyBigInteger New() => MyBigInteger.GreatestCommonDivisor(m1, m2);
}

[lskyum]

@kzrnm That looks really awesome! Thanks for making this possible.
I hope that will be available in .NET 9 ;-)

[danmoseley]

Do we have enough coverage in dotnet/performance to protect this change?

[kzrnm]

https://github.com/dotnet/performance/blob/8fe05289b450e96c3f2c646af979d20110c46db2/src/benchmarks/micro/libraries/System.Runtime.Numerics/Perf.BigInteger.cs#L69-L74

https://github.com/dotnet/performance/blob/8fe05289b450e96c3f2c646af979d20110c46db2/src/benchmarks/micro/libraries/System.Runtime.Numerics/Perf.BigInteger.cs#L87-L90

The 65536 bits correspond to a uint array of length 4096, which would cover this change.

[kzrnm]

I introduced DisableParallelization to modpowTest and divremTest because some tests sometimes failed.

[jkotas]

I introduced DisableParallelization to modpowTest and divremTest because some tests sometimes failed.

What did the failure look like? Is DisableParallelization working around a bug somewhere else? If yes, we would want that bug fixed.

[kzrnm]

@jkotas

Assertion failed on this line.

runtime/src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.DivRem.cs

Line 510 in a2893b8

Debug.Assert(right.Length < DivideBurnikelZieglerThreshold);

Some assertions expect BigInteger.*Threshold to be immutable. However, the assertion may fail if there are other tests running in parallel that rewrite those thresholds with BigIntTools.Utils.RunWithFakeThreshold.
I suspect that the reason why #94610 did not reproduce was due to parallel execution.
BigIntTools.Utils.RunWithFakeThreshold is also used in parseTest and multiplyTest, but since it overlaps with #97589, I didn't include it in this pull request. Would it have been better to include it?

[jkotas]

Some assertions expect BigInteger.*Threshold to be immutable. However, the assertion may fail if there are other tests running in parallel that rewrite those thresholds with BigIntTools.Utils.RunWithFakeThreshold.

Can the problem be also hit in other unrelated tests that happen to use BigInteger.Divide? It seems that debug-only threshold overrides should be thread-local variable so that they do not affect other unrelated tests.

[kzrnm]

BigIntegerCalculator.Remainder, which has the same implementation as BigIntegerCalculator.Divide, is used in BigInteger.ModPow and BigInteger. GreatestCommonDivisor.

Errors may occur in the following cases

1. A RunWithFakeThreshold test starts.
   1. The test overrides BigInteger.*Threshold.
2. Another class like gcdTest starts.
3. The RunWithFakeThreshold test finished
   1. Override of BigInteger.*Threshold is canceled.
4. Some assertions may fail.