Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update JuMP and linopy benchmark scripts #163

Merged
merged 1 commit into from
Sep 19, 2023

Conversation

odow
Copy link
Contributor

@odow odow commented Sep 19, 2023

Hey! I've been meaning to check this project out. People keep mentioning it to me.

Your docs are very nice, and I quite like this style of modeling. I couldn't get the benchmarks to run (told me it couldn't find ortools-python and it took ages to try and find a feasible install?), but here are a couple of minor changes. (Feel free to close this PR if you would prefer to keep them as-is.)

I thought it might be helpful to explain why JuMP's memory usage is so high: we actually store three copies of the problem data.

  • One is in JuMP, that's all of the x and y variables
  • One is in a MathOptInterface cache, so that we can swap out solvers etc
  • One is the solver's in-memory representation.

Here's a demonstration:

using JuMP, Gurobi

function basic_model(n, solver)
    m = Model(solver)
    set_silent(m)
    N = 1:n
    M = 1:n
    @variable(m, x[N, M])
    @variable(m, y[N, M])
    @constraint(m, [i=N, j=M], x[i, j] - y[i, j] >= i-1)
    @constraint(m, [i=N, j=M], x[i, j] + y[i, j] >= 0)
    @objective(m, Min, sum(2 * x[i, j] + y[i, j] for i in N, j in M))
    optimize!(m)
    return objective_value(m)
end

function basic_model_1(n, solver)
    m = Model(solver) 
    set_silent(m)
    @variable(m, x[1:n, 1:n])
    @variable(m, y[1:n, 1:n])
    @constraint(m, x - y .>= 0:(n-1))
    @constraint(m, x + y .>= 0)
    @objective(m, Min, 2 * sum(x) + sum(y))
    optimize!(m)
    return objective_value(m)
end

function basic_model_2(n, solver)
    m = direct_model(solver())
    set_silent(m)
    @variable(m, x[1:n, 1:n])
    @variable(m, y[1:n, 1:n])
    @constraint(m, x - y .>= 0:(n-1))
    @constraint(m, x + y .>= 0)
    @objective(m, Min, 2 * sum(x) + sum(y))
    optimize!(m)
    return objective_value(m)
end

julia> GC.gc(); @time basic_model(700, Gurobi.Optimizer)
  7.008127 seconds (32.83 M allocations: 2.411 GiB, 20.01% gc time)
8.56275e7

julia> GC.gc(); @time basic_model_1(700, Gurobi.Optimizer)
  7.231301 seconds (32.83 M allocations: 2.482 GiB, 22.91% gc time)
8.56275e7

julia> GC.gc(); @time basic_model_2(700, Gurobi.Optimizer)
  5.856433 seconds (28.91 M allocations: 1.932 GiB, 19.70% gc time)
8.56275e7

Biggest change with direct_model is that memory usage drops by 20% (not the expected 1/3 because there's some other overhead, etc).

If we turn off passing variable names to Gurobi, we get more improvement:

julia> function basic_model_3(n, solver)
           m = direct_model(solver())
           set_string_names_on_creation(m, false)
           set_silent(m)
           @variable(m, x[1:n, 1:n])
           @variable(m, y[1:n, 1:n])
           @constraint(m, x - y .>= 0:(n-1))
           @constraint(m, x + y .>= 0)
           @objective(m, Min, 2 * sum(x) + sum(y))
           optimize!(m)
           return objective_value(m)
       end
basic_model_3 (generic function with 1 method)

julia> GC.gc(); @time basic_model_3(700, Gurobi.Optimizer)
  5.596101 seconds (23.03 M allocations: 1.728 GiB, 22.79% gc time)
8.56275e7

But I wouldn't expect most users to care about this.

This commit slightly tweaks the JuMP and linopy benchmark scripts.

The JuMP models are rewritten to be more similar to the style of linopy, even
though it is slightly slower.

The linopy objective is tweaked to result in a single multiplication instead of
n^2.

In future, the memory usage of the JuMP model could be reduced by using
m=direct_model(solver()). This is  not the default because it does not work for
all solvers (like Cbc).
@codecov
Copy link

codecov bot commented Sep 19, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (620724b) 87.30% compared to head (95f7a45) 87.30%.
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #163   +/-   ##
=======================================
  Coverage   87.30%   87.30%           
=======================================
  Files          14       14           
  Lines        3119     3119           
  Branches      707      707           
=======================================
  Hits         2723     2723           
  Misses        289      289           
  Partials      107      107           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@FabianHofmann
Copy link
Collaborator

Hey @odow, thank you for your PR. This improves readability and performance at the same time, nice :) JuMP is a wonderful package. Some years ago, we even tried switching to Julia for our energy modelling system in order to benefit from JuMP, but it proved too cumbersom for different reasons...

Yes, JuMP's internal memory consumption has already been mentioned by a colleague of mine, but it is still relatively low. For comparability, I would keep the variable names (the other packages also keep them). Turning on the direct model would be totally fine by me.

I'll merge this one. If you have other propositions, feel free to create a PR :)

(I think I should remove or-tools from the list, I also had huge problems to install it...)

@FabianHofmann FabianHofmann merged commit fc39d25 into PyPSA:master Sep 19, 2023
14 checks passed
@odow odow deleted the od/benchmark branch November 27, 2023 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants