Use Montgomery arithmetic in n_is_prime

[fredrik-johansson]

This speeds up the strong probable prime tests for n between 33 and 64 bits. Shoup reduction is still faster than Montgomery for <= 32 bits (to get a speedup there, a "fast" Shoup with non-canonical residues looks promising).

                  random                          primes               composites without small factors
bits |    old       new    speedup |     old       new    speedup   |     old       new    speedup   
28   | 1.71e-08  1.72e-08   0.994  |  8.13e-08  8.14e-08   0.999  |  7.88e-08  7.88e-08   1.000 
29   | 1.74e-08  1.75e-08   0.994  |  8.44e-08  8.44e-08   1.000  |  8.19e-08  8.18e-08   1.001 
30   | 1.78e-08  1.79e-08   0.994  |  8.77e-08  8.78e-08   0.999  |  8.52e-08  8.51e-08   1.001 
31   | 1.84e-08  1.85e-08   0.995  |  9.09e-08  9.11e-08   0.998  |  8.84e-08  8.92e-08   0.991 
32   | 1.88e-08  1.88e-08   1.000  |  9.44e-08  9.44e-08   1.000  |  9.18e-08  9.16e-08   1.002 
33   | 3.25e-08  2.49e-08   1.305  |  3.25e-07  2.23e-07   1.457  |  1.61e-07  1.21e-07   1.331 
34   | 3.30e-08  2.52e-08   1.310  |  3.34e-07  2.29e-07   1.459  |  1.67e-07  1.25e-07   1.336 
35   | 3.38e-08  2.58e-08   1.310  |  3.40e-07  2.34e-07   1.453  |  1.71e-07  1.27e-07   1.346 
36   | 3.33e-08  2.55e-08   1.306  |  3.49e-07  2.40e-07   1.454  |  1.75e-07  1.31e-07   1.336 
37   | 3.41e-08  2.61e-08   1.307  |  3.60e-07  2.45e-07   1.469  |  1.81e-07  1.33e-07   1.361 
38   | 3.46e-08  2.62e-08   1.321  |  3.70e-07  2.50e-07   1.480  |  1.85e-07  1.35e-07   1.370 
39   | 3.51e-08  2.65e-08   1.325  |  3.79e-07  2.56e-07   1.480  |  1.90e-07  1.38e-07   1.377 
40   | 3.57e-08  2.68e-08   1.332  |  3.87e-07  2.62e-07   1.477  |  1.95e-07  1.41e-07   1.383 
41   | 3.64e-08  2.72e-08   1.338  |  3.98e-07  2.67e-07   1.491  |  2.00e-07  1.43e-07   1.399 
42   | 3.67e-08  2.75e-08   1.335  |  4.08e-07  2.72e-07   1.500  |  2.04e-07  1.46e-07   1.397 
43   | 3.78e-08  2.80e-08   1.350  |  4.17e-07  2.78e-07   1.500  |  2.10e-07  1.49e-07   1.409 
44   | 3.82e-08  2.83e-08   1.350  |  4.25e-07  2.84e-07   1.496  |  2.14e-07  1.51e-07   1.417 
45   | 3.81e-08  2.82e-08   1.351  |  4.36e-07  2.89e-07   1.509  |  2.19e-07  1.54e-07   1.422 
46   | 3.92e-08  2.88e-08   1.361  |  4.44e-07  2.94e-07   1.510  |  2.24e-07  1.57e-07   1.427 
47   | 3.94e-08  2.88e-08   1.368  |  4.54e-07  3.00e-07   1.513  |  2.28e-07  1.59e-07   1.434 
48   | 3.96e-08  2.90e-08   1.366  |  4.64e-07  3.05e-07   1.521  |  2.33e-07  1.62e-07   1.438 
49   | 4.05e-08  2.95e-08   1.373  |  4.73e-07  3.11e-07   1.521  |  2.38e-07  1.64e-07   1.451 
50   | 4.07e-08  2.97e-08   1.370  |  4.83e-07  3.17e-07   1.524  |  2.43e-07  1.67e-07   1.455 
51   | 4.16e-08  3.01e-08   1.382  |  4.92e-07  3.22e-07   1.528  |  2.47e-07  1.70e-07   1.453 
52   | 4.21e-08  3.05e-08   1.380  |  5.04e-07  3.29e-07   1.532  |  2.54e-07  1.75e-07   1.451 
53   | 4.30e-08  3.12e-08   1.378  |  5.16e-07  3.38e-07   1.527  |  2.60e-07  1.80e-07   1.444 
54   | 4.33e-08  3.14e-08   1.379  |  5.25e-07  3.44e-07   1.526  |  2.66e-07  1.82e-07   1.462 
55   | 4.34e-08  3.12e-08   1.391  |  5.35e-07  3.49e-07   1.533  |  2.71e-07  1.85e-07   1.465 
56   | 4.42e-08  3.17e-08   1.394  |  5.43e-07  3.54e-07   1.534  |  2.75e-07  1.87e-07   1.471 
57   | 4.41e-08  3.17e-08   1.391  |  5.53e-07  3.60e-07   1.536  |  2.87e-07  1.90e-07   1.511 
58   | 4.64e-08  3.31e-08   1.402  |  5.63e-07  3.65e-07   1.542  |  2.84e-07  1.93e-07   1.472 
59   | 4.69e-08  3.33e-08   1.408  |  5.72e-07  3.70e-07   1.546  |  2.89e-07  1.95e-07   1.482 
60   | 4.60e-08  3.27e-08   1.407  |  5.81e-07  3.76e-07   1.545  |  2.94e-07  1.98e-07   1.485 
61   | 4.73e-08  3.36e-08   1.408  |  5.93e-07  3.84e-07   1.544  |  3.02e-07  2.03e-07   1.488 
62   | 4.85e-08  3.44e-08   1.410  |  6.04e-07  3.92e-07   1.541  |  3.07e-07  2.08e-07   1.476 
63   | 4.81e-08  3.74e-08   1.286  |  6.14e-07  4.40e-07   1.395  |  3.12e-07  2.37e-07   1.316 
64   | 4.67e-08  3.85e-08   1.213  |  5.96e-07  4.46e-07   1.336  |  2.92e-07  2.40e-07   1.217 

Edit: initial version had a small bug causing 5% performance degradation; updated with timings for the newly pushed version.

[fredrik-johansson]

Should I be concerned by the sudden failure of msolve on CI? It's reproducible (3 times) on CI, but I can't reproduce it locally on my machine.

[vneiger]

Should I be concerned by the sudden failure of msolve on CI? It's reproducible (3 times) on CI, but I can't reproduce it locally on my machine.

After some investigation with the specific compile commands from the CI, I've been able to reproduce, and the minimal context where I can reproduce is when using ./configure --enable-assert for FLINT (for the rest: simply make and sudo make install for FLINT, and for msolve make and make check).

I see no test failure when using just ./configure (without assert), or when using ./configure --enable-assert but from the main branch of FLINT.

My guess is that this could be related to some functions that pick primes / nextprimes in msolve (I'll look into that), but I'm puzzled by why this does not occur without assert. Any idea about what --enable-assert could change in FLINT's behaviour (possibly related to n_is_prime) that could lead to this difference?

[vneiger]

Well, the explanation for the impact of assert is easy: for all three failed tests, launching them yields:

.../flint/src/nmod/pow.c:210: _nmod_redc_fast_2_pow_ui: Assertion `exp >= 64' failed.

We do not call this recent function in msolve directly. May there be a place in FLINT where it is called without satisfying the input assumptions? If not, I can look more closely into the msolve code to see if we call some FLINT functions while not respecting their input constraints.

[fredrik-johansson]

Ah, that would explain it.

Now I need to investigate why our own tests don't pick that up.

[fredrik-johansson]

Oh, our CI doesn't actually run make check with asserts enabled on 64-bit anymore. That's something we'll probably have to fix.