Various improvements to the FileFormats.MPS subpackage

[odow]

• Function barriers for type-unstable hot loops
• Deterministic ordering of rows/columns (Closes MPS code is not deterministic #1106)
• Remove TempRow and TempColumn in favor of vectors
• Use Int instead of String in coef matrix for lower memory usage

Will post before/after benchmarks reading MIPLIB soon.

Still todo:

• Some instances of non-determinisim

[odow]

tl;dr: 5-10x reduction in allocations. 50% faster.

I picked 10 files from MIPLIB. Then

using MathOptInterface
const MOI = MathOptInterface

function benchmark(filename)
    println("Benchmarking $(filename)")
    src = MOI.FileFormats.Model(format = MOI.FileFormats.FORMAT_MPS)
    dest = MOI.FileFormats.Model(format = MOI.FileFormats.FORMAT_MPS)
    @time MOI.read_from_file(src, filename)
    @time MOI.copy_to(dest, src)
    @time MOI.write_to_file(dest, "out_" * filename)
    return
end

for filename in readdir(".")
    if !endswith(filename, ".mps.gz") || startswith(filename, "out_")
        continue
    end
    benchmark(filename)
end

MOI v0.9.14:

Benchmarking bab2.mps.gz
 23.438271 seconds (468.16 M allocations: 15.583 GiB, 36.89% gc time)
  0.258183 seconds (669.10 k allocations: 123.261 MiB)
  6.046640 seconds (17.87 M allocations: 668.275 MiB, 17.82% gc time)
Benchmarking irish-electricity.mps.gz
  3.438548 seconds (39.11 M allocations: 1.673 GiB, 32.11% gc time)
  0.320944 seconds (1.56 M allocations: 101.770 MiB, 17.23% gc time)
  2.253398 seconds (6.44 M allocations: 274.635 MiB, 10.60% gc time)
Benchmarking n2seq36q.mps.gz
  3.309943 seconds (89.87 M allocations: 2.881 GiB, 26.81% gc time)
  0.035270 seconds (101.24 k allocations: 17.046 MiB, 25.98% gc time)
  0.464845 seconds (1.26 M allocations: 58.136 MiB, 1.80% gc time)
Benchmarking rocI-4-11.mps.gz
  0.629294 seconds (9.29 M allocations: 324.349 MiB, 49.87% gc time)
  0.025924 seconds (170.76 k allocations: 11.336 MiB)
  0.141616 seconds (614.75 k allocations: 26.819 MiB)
Benchmarking rocII-5-11.mps.gz
  3.597759 seconds (75.90 M allocations: 2.557 GiB, 29.85% gc time)
  0.075016 seconds (384.91 k allocations: 30.319 MiB, 20.13% gc time)
  0.719202 seconds (1.67 M allocations: 72.313 MiB, 6.14% gc time)
Benchmarking roi5alpha10n8.mps.gz
 25.330902 seconds (451.60 M allocations: 15.394 GiB, 37.51% gc time)
  0.195371 seconds (380.39 k allocations: 115.502 MiB)
  5.367165 seconds (10.38 M allocations: 479.186 MiB)
Benchmarking supportcase42.mps.gz
  3.637927 seconds (73.48 M allocations: 2.573 GiB, 29.64% gc time)
  0.075642 seconds (298.55 k allocations: 33.200 MiB, 21.90% gc time)
  1.063418 seconds (2.75 M allocations: 110.861 MiB, 6.31% gc time)
Benchmarking timtab1.mps.gz
  0.008559 seconds (244.89 k allocations: 8.387 MiB)
  0.000913 seconds (4.72 k allocations: 462.641 KiB)
  0.005361 seconds (28.49 k allocations: 1.395 MiB)
Benchmarking triptim1.mps.gz
  4.903633 seconds (116.63 M allocations: 3.906 GiB, 26.94% gc time)
  0.098737 seconds (342.96 k allocations: 45.298 MiB, 15.67% gc time)
  1.281748 seconds (2.53 M allocations: 111.423 MiB, 10.25% gc time)
Benchmarking uccase12.mps.gz
  6.417404 seconds (152.37 M allocations: 5.062 GiB, 29.72% gc time)
  0.511936 seconds (1.81 M allocations: 123.843 MiB, 29.90% gc time)
  2.292074 seconds (5.96 M allocations: 264.596 MiB, 6.86% gc time)

This PR:

Benchmarking bab2.mps.gz
  8.120397 seconds (60.00 M allocations: 3.294 GiB, 10.38% gc time)
  0.607790 seconds (1.41 M allocations: 167.320 MiB, 26.83% gc time)
  6.349063 seconds (23.42 M allocations: 886.163 MiB, 24.22% gc time)
Benchmarking irish-electricity.mps.gz
  1.998047 seconds (17.71 M allocations: 1006.500 MiB, 18.10% gc time)
  0.256303 seconds (1.56 M allocations: 101.770 MiB)
  1.843973 seconds (6.52 M allocations: 281.954 MiB, 12.73% gc time)
Benchmarking n2seq36q.mps.gz
  1.392445 seconds (5.63 M allocations: 366.960 MiB, 15.70% gc time)
  0.058739 seconds (213.72 k allocations: 29.793 MiB, 15.98% gc time)
  0.444313 seconds (2.04 M allocations: 91.360 MiB, 7.01% gc time)
Benchmarking rocI-4-11.mps.gz
  0.197237 seconds (1.31 M allocations: 76.716 MiB, 12.89% gc time)
  0.025007 seconds (193.60 k allocations: 12.111 MiB)
  0.128624 seconds (774.23 k allocations: 32.935 MiB, 7.74% gc time)
Benchmarking rocII-5-11.mps.gz
  1.514372 seconds (9.43 M allocations: 567.252 MiB, 13.16% gc time)
  0.158044 seconds (441.66 k allocations: 40.952 MiB, 47.94% gc time)
  0.627313 seconds (2.07 M allocations: 92.228 MiB, 6.08% gc time)
Benchmarking roi5alpha10n8.mps.gz
  8.671225 seconds (64.50 M allocations: 3.688 GiB, 7.79% gc time)
  0.448843 seconds (910.19 k allocations: 151.429 MiB, 25.84% gc time)
  6.378291 seconds (14.09 M allocations: 629.152 MiB, 13.56% gc time)
Benchmarking supportcase42.mps.gz
  1.638949 seconds (12.47 M allocations: 748.677 MiB, 12.65% gc time)
  0.060551 seconds (298.55 k allocations: 33.200 MiB, 17.48% gc time)
  0.801507 seconds (2.75 M allocations: 112.805 MiB, 8.41% gc time)
Benchmarking timtab1.mps.gz
  0.004380 seconds (31.82 k allocations: 1.841 MiB)
  0.000771 seconds (5.01 k allocations: 473.047 KiB)
  0.004989 seconds (30.23 k allocations: 1.477 MiB)
Benchmarking triptim1.mps.gz
  2.271965 seconds (14.70 M allocations: 859.014 MiB, 12.57% gc time)
  0.115689 seconds (445.18 k allocations: 48.042 MiB, 13.08% gc time)
  1.189662 seconds (3.24 M allocations: 137.639 MiB, 15.31% gc time)
Benchmarking uccase12.mps.gz
  3.006464 seconds (16.57 M allocations: 987.349 MiB, 16.62% gc time)
  0.478288 seconds (1.85 M allocations: 125.201 MiB, 23.87% gc time)
  1.772791 seconds (6.29 M allocations: 281.404 MiB, 7.97% gc time)

[odow]

Now we're down to:

Benchmarking bab2.mps.gz
  7.710706 seconds (36.92 M allocations: 2.007 GiB, 8.09% gc time)
  4.849239 seconds (14.89 M allocations: 847.306 MiB, 13.33% gc time)
  7.858524 seconds (27.77 M allocations: 1.072 GiB, 18.17% gc time)
Benchmarking bab2.mps.gz
  6.455890 seconds (33.57 M allocations: 1.856 GiB, 9.77% gc time)
  0.539828 seconds (1.41 M allocations: 167.320 MiB, 12.68% gc time)
  6.635904 seconds (23.42 M allocations: 886.163 MiB, 25.67% gc time)
Benchmarking irish-electricity.mps.gz
  1.274335 seconds (10.70 M allocations: 609.689 MiB, 15.58% gc time)
  1.927393 seconds (6.83 M allocations: 363.793 MiB, 15.18% gc time)
  1.906404 seconds (6.53 M allocations: 282.465 MiB, 13.10% gc time)
Benchmarking irish-electricity.mps.gz
  1.210296 seconds (10.53 M allocations: 601.763 MiB, 15.31% gc time)
  0.249854 seconds (1.56 M allocations: 101.770 MiB)
  1.907251 seconds (6.52 M allocations: 281.954 MiB, 16.12% gc time)
Benchmarking n2seq36q.mps.gz
  0.912895 seconds (3.22 M allocations: 220.755 MiB, 3.00% gc time)
  0.065794 seconds (213.72 k allocations: 29.793 MiB, 15.36% gc time)
  0.455553 seconds (2.04 M allocations: 91.360 MiB, 5.00% gc time)
Benchmarking n2seq36q.mps.gz
  0.944816 seconds (3.22 M allocations: 220.755 MiB, 3.90% gc time)
  0.056917 seconds (213.72 k allocations: 29.793 MiB)
  0.446214 seconds (2.04 M allocations: 91.360 MiB, 5.35% gc time)
Benchmarking rocI-4-11.mps.gz
  0.147952 seconds (881.79 k allocations: 51.376 MiB, 4.91% gc time)
  0.551742 seconds (2.04 M allocations: 103.886 MiB, 4.37% gc time)
  0.165044 seconds (776.91 k allocations: 33.090 MiB, 8.65% gc time)
Benchmarking rocI-4-11.mps.gz
  0.132504 seconds (853.32 k allocations: 49.896 MiB, 5.68% gc time)
  0.025387 seconds (193.60 k allocations: 12.111 MiB)
  0.126668 seconds (774.23 k allocations: 32.935 MiB, 7.55% gc time)
Benchmarking rocII-5-11.mps.gz
  0.948792 seconds (5.47 M allocations: 332.206 MiB, 5.60% gc time)
  0.095951 seconds (447.81 k allocations: 41.296 MiB, 14.32% gc time)
  0.626843 seconds (2.07 M allocations: 92.228 MiB, 9.94% gc time)
Benchmarking rocII-5-11.mps.gz
  0.963915 seconds (5.47 M allocations: 332.206 MiB, 5.96% gc time)
  0.090795 seconds (441.66 k allocations: 40.952 MiB, 16.54% gc time)
  0.665347 seconds (2.07 M allocations: 92.228 MiB, 10.22% gc time)
Benchmarking roi5alpha10n8.mps.gz
  6.501416 seconds (35.59 M allocations: 2.069 GiB, 7.53% gc time)
  0.438999 seconds (910.19 k allocations: 151.429 MiB, 9.69% gc time)
  6.390075 seconds (14.09 M allocations: 629.152 MiB, 13.54% gc time)
Benchmarking roi5alpha10n8.mps.gz
  6.500939 seconds (35.59 M allocations: 2.069 GiB, 8.15% gc time)
  0.359993 seconds (910.19 k allocations: 151.429 MiB)
  6.470193 seconds (14.09 M allocations: 629.152 MiB, 12.99% gc time)
Benchmarking supportcase42.mps.gz
  1.074311 seconds (6.90 M allocations: 431.293 MiB, 6.23% gc time)
  0.069341 seconds (304.69 k allocations: 33.544 MiB, 15.08% gc time)
  0.825769 seconds (2.75 M allocations: 112.805 MiB, 6.44% gc time)
Benchmarking supportcase42.mps.gz
  1.071960 seconds (6.90 M allocations: 431.293 MiB, 6.26% gc time)
  0.092562 seconds (298.55 k allocations: 33.200 MiB, 27.43% gc time)
  0.842975 seconds (2.75 M allocations: 112.805 MiB, 8.72% gc time)
Benchmarking timtab1.mps.gz
  0.004202 seconds (24.59 k allocations: 1.423 MiB)
  0.000853 seconds (5.01 k allocations: 473.047 KiB)
  0.005032 seconds (30.23 k allocations: 1.477 MiB)
Benchmarking timtab1.mps.gz
  0.004467 seconds (24.59 k allocations: 1.423 MiB)
  0.001023 seconds (5.01 k allocations: 473.047 KiB)
  0.005218 seconds (30.23 k allocations: 1.477 MiB)
Benchmarking triptim1.mps.gz
  1.535064 seconds (8.28 M allocations: 491.923 MiB, 6.41% gc time)
  0.142356 seconds (445.18 k allocations: 48.042 MiB, 21.32% gc time)
  1.145413 seconds (3.24 M allocations: 137.639 MiB, 3.90% gc time)
Benchmarking triptim1.mps.gz
  1.534380 seconds (8.28 M allocations: 491.923 MiB, 10.80% gc time)
  0.107887 seconds (445.18 k allocations: 48.042 MiB, 14.36% gc time)
  1.194528 seconds (3.24 M allocations: 137.639 MiB, 11.03% gc time)
Benchmarking uccase12.mps.gz
  2.166886 seconds (10.07 M allocations: 610.308 MiB, 10.70% gc time)
  0.548862 seconds (1.85 M allocations: 125.201 MiB, 30.96% gc time)
  1.932424 seconds (6.29 M allocations: 281.404 MiB, 12.16% gc time)
Benchmarking uccase12.mps.gz
  2.127841 seconds (10.07 M allocations: 610.308 MiB, 11.17% gc time)
  0.357176 seconds (1.85 M allocations: 125.201 MiB)
  1.927121 seconds (6.29 M allocations: 281.404 MiB, 14.37% gc time)

[odow]

This is in a reasonable state now, and much better than before. I'm not planning on future changes. I'll leave open for a few days and then, barring objections, merge.