Skip to content

set ens members to one, this is about 3x faster for a single forecast#12

Merged
KristianHMoller merged 8 commits intodmidk:mainfrom
JoachimKoenigslieb:feature/ens-member-1
May 8, 2026
Merged

set ens members to one, this is about 3x faster for a single forecast#12
KristianHMoller merged 8 commits intodmidk:mainfrom
JoachimKoenigslieb:feature/ens-member-1

Conversation

@JoachimKoenigslieb
Copy link
Copy Markdown
Contributor

Running with this change I get:

((venv) ) (base) jjk@nwp-archive2:~/run-sunflow> time ./venv/bin/sunflow --run_mode files --dataset DWD --bbox CUSTOM --custom-bbox="-10.75,47.25,20,63.5" --time "2026-02-01T09:00"
Pysteps configuration file found at: /home/jjk/run-sunflow/venv/lib/python3.12/site-packages/pysteps/pystepsrc
...
2026-05-06 13:42:11.480 | INFO     | sunflow.main:cli:478 - Script end time = 2026-05-06 13:42:11 UTC

real    0m4.343s
user    0m3.673s
sys     0m0.762s

And without I get:

real    0m12.015s
user    0m57.413s
sys     0m10.522s

So wall-clock is about 3x faster and the user a lot lower because it does not do multi-processing stuff to get the ens members I presume.

@KristianHMoller
Copy link
Copy Markdown
Contributor

KristianHMoller commented May 6, 2026

Thanks @JoachimKoenigslieb !

This is great!

But perhaps, we should do it a bit more future proof and use this variable from config.py: ens_members=int(os.getenv("ENS_MEMBERS", "1")),
It defaults to one, so there should be no issue

@JoachimKoenigslieb
Copy link
Copy Markdown
Contributor Author

Yes, that sounds like a great idea.

I've used the value from the config instead!

@KristianHMoller
Copy link
Copy Markdown
Contributor

Thanks a lot @JoachimKoenigslieb !

I can confirm the speed up:

With update:

real	0m8.143s
user	0m10.613s
sys	0m1.547s

Without update :

real	0m15.351s
user	1m32.068s
sys	0m11.578s

If you fix the linting (with pre-commit install; pre-commit run --all-files it should be done automatically) and update the CHANGELOG.md, I will approve and then we will see who has authorization to merge.

Copy link
Copy Markdown
Contributor

@KristianHMoller KristianHMoller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With an entry in CHANGELOG.md, this is ready for approval and merge!

Comment thread CHANGELOG.md Outdated
Comment thread CHANGELOG.md
Copy link
Copy Markdown
Contributor

@KristianHMoller KristianHMoller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

Comment thread CHANGELOG.md Outdated
@KristianHMoller
Copy link
Copy Markdown
Contributor

@JoachimKoenigslieb I took the liberty of doing a slight modification of your CHANGELOG entry to save you the hassle. I hope that is okay?

Are you allowed to merge this yourself? Or are you okay with me merging it?

@JoachimKoenigslieb
Copy link
Copy Markdown
Contributor Author

Hi,

You're always welcome to make changes directly on my PRs! I will remember to add links + attribution in changelogs going forward.

I'm not able to press merge anywhere, so I think someone on your side needs to press it.

@KristianHMoller KristianHMoller merged commit 530d44c into dmidk:main May 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants