Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create benchmark script #896

Merged
merged 1 commit into from
Mar 15, 2024

Conversation

abelsiqueira
Copy link
Contributor

@abelsiqueira abelsiqueira commented Feb 21, 2024

Create a benchmark script.
@manuelma @Tasqu, @suvayu and I created this simple benchmark using the first test in run_spineopt.jl.
However, this is not substantial enough.
Can you

  • suggest improvements to make it more substantial; and/or
  • share some data in sqlite format?

Thanks in advance.

Checklist before merging

  • Documentation is up-to-date
  • Unit tests have been added/updated accordingly
  • Code has been formatted according to SpineOpt's style
  • Unit tests pass

@abelsiqueira
Copy link
Contributor Author

@tarskul, maybe you can help us with data?

@manuelma
Copy link
Collaborator

manuelma commented Mar 6, 2024

run_spineopt_benchmarks.zip
@abelsiqueira I was working on something before, here it is for inspiration?

Edit: it uses the function _load_test_data from SpineOpt/test/run_tests.jl

@tarskul
Copy link
Collaborator

tarskul commented Mar 6, 2024

I'm currently working on a workflow with data from PyPSA, converting it to the ines-spec and then using that to run a model in PyPSA and SpineOpt as a comparison benchmark. Once I have a small working example (it may take a few weeks as I cannot always allocate time for this), I'll share that and I'll be able to help with data or the benchmarking itself. If you cannot wait, you can find data for/from PyPSA on Zenodo.

@abelsiqueira
Copy link
Contributor Author

Thanks @manuelma, we've used your example and changed our script.
It should be a good benchmark to start, so we think it's ready to merge.

We haven't added any automation to this for simplicity. We can create a new issue for that, if you want.

@abelsiqueira
Copy link
Contributor Author

By the way, this warning came up:

┌ Warning: no objective terms defined
└ @ SpineOpt ~/projects/work/MOPO/SpineOpt.jl/src/run_spineopt_standard.jl:229

@abelsiqueira abelsiqueira marked this pull request as ready for review March 6, 2024 10:40
@abelsiqueira
Copy link
Contributor Author

@tarskul, thanks, it might be useful later, maybe in addition to the example given by @manuelma. For now, we have enough for this issue, but we can revisit in a few weeks.

Copy link
Collaborator

@clizbe clizbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@clizbe clizbe merged commit 77bba79 into spine-tools:master Mar 15, 2024
@abelsiqueira abelsiqueira deleted the create-benchmark-script branch May 2, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants