Write up the different formulations of IJKLM

[odow]

Text still to write.

I get these timings.

  # intuitive original
  4.030615 seconds (12.99 M allocations: 485.209 MiB, 3.14% gc time, 9.25% compilation time)
  3.632798 seconds (12.64 M allocations: 462.332 MiB, 2.83% gc time)
  # intuitive tuple
  2.012132 seconds (33.92 M allocations: 1.545 GiB, 16.08% gc time, 15.50% compilation time)
  1.741992 seconds (33.68 M allocations: 1.529 GiB, 17.52% gc time)
  # fast tuple
  1.196328 seconds (33.37 M allocations: 1.512 GiB, 25.87% gc time, 26.52% compilation time)
  0.859798 seconds (33.12 M allocations: 1.495 GiB, 31.13% gc time)
  # dataframe
  0.143748 seconds (461.88 k allocations: 51.189 MiB, 14.14% gc time, 21.02% compilation time)
  0.118874 seconds (435.75 k allocations: 49.386 MiB, 19.03% gc time)
  # fast dataframe
  0.142556 seconds (245.42 k allocations: 32.769 MiB, 13.89% gc time, 21.10% compilation time)
  0.092028 seconds (219.34 k allocations: 30.968 MiB)

[mlubin]

Thanks for putting this together. I realized at the end that I should've commented on the .jl file, not the .md file.

[odow]

[odow]

It might also be worth pointing out that "fast Pyomo" is faster than "fast JuMP" because they use a different data structure.

In this case, Pyomo gets given the JKL and KLM sets as dictionaries already indexed by the first two, and not as a single list:

https://github.com/justine18/performance_experiment/blob/0aa5512e34c9041d719fa8c0763fdc892e021415/IJKLM/run_pyomo.py#L122

[blegat]

Wouldn't it be even faster with Int instead of String ? The fact that the indices are prefixed with "i", "j" etc.. is useless since they are already in different columns so there is no risk of clash. Probably this isn't relevant here since otherwise the blog post won't be applicable for cases where the indices are not integers.

[blegat]

It might also be worth pointing out that "fast Pyomo" is faster than "fast JuMP" because they use a different data structure.
In this case, Pyomo gets given the JKL and KLM sets as dictionaries already indexed by the first two, and not as a single list:
https://github.com/justine18/performance_experiment/blob/0aa5512e34c9041d719fa8c0763fdc892e021415/IJKLM/run_pyomo.py#L122

Yes, I would add it in the blog post at the end

[jd-foster]

2c. I wish datastructures was a compound word.

[odow]

lots of changes. Plot is now

[odow]

Okay, I think we're pretty much there.

[mlubin]

LGTM besides the intro paragraph.

[odow]

This is good to go by me, but I'll wait for some more approvals before merging.

[joehuchette]

GAMS, not Pyomo?

[odow]

Pyomo. Related to this image:

@RichardOberdieck was wondering https://oberdieck.dk/p/not-so-fast-gams/

[RichardOberdieck]

Uh, nice, I guessed that you guys would follow up on this :) These timings look nice, go JuMP!

[odow]

I have tried to “beat” GAMS two or three times in model generation speed using gurobipy, and it is pretty tough. The best I’ve gotten was around 10% slower than GAMS.

@RichardOberdieck what was the fastest? Did you try a pandas approach, or just using the gp.tuplelist? It's a bit of a weird model for a benchmark because the choice of input data structure has such a large impact.

[RichardOberdieck]

So I didn't try it for this one because I don't access to a GAMS license anymore. When I was still working at Gurobi, I had a client using GAMS and I wanted to show the gurobipy could be as fast. So spent about 2-3 days working on formulating that MIP in the most vectorized, and optimized form I could think of. I ended up being 10% slower roughly. The same happened with a different client a few months later, but that was a smaller model.

With gurobipy you can be really fast when you have matrix-like operations. There you can use the matrix API, which is very efficient.

Regarding this benchmark I agree: it is pretty useless, and I don't think anyone sense-checked the numbers. Otherwise pyomo would never have been faster than JuMP...

By the way: what is the plan now with this PR, are you going to make a blog post about it? Would be a fun read :)

[blegat]

By the way: what is the plan now with this PR, are you going to make a blog post about it? Would be a fun read :)

Yes, once this PR is merged, this should appear in the blog post: https://jump.dev/pages/news/

[odow]

Here's the post: https://jump.dev/2023/07/20/gams-blog/