Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE REQUEST] Reduction of memory for HISTORY diagnostics with species dimensions #270

Closed
yantosca opened this issue Apr 3, 2020 · 2 comments
Assignees
Labels
category: Feature Request New feature or request
Milestone

Comments

@yantosca
Copy link
Contributor

yantosca commented Apr 3, 2020

Currently, the State_Diag arrays for HISTORY diagnostics with species dimensions (e.g. SpeciesConc, DryDep, WetLossConv, WetLossLS) are always allocated with the maximum possible number of slots (e.g. State_Chm%nSpecies, State_Chm%nDrydep, etc.). This can cause a large amount of memory to be allocated, especially if only a subset of species need to be archived.

@yantosca is currently working on an update that will only allocate as much memory as there are species requested in HISTORY.rc. For example, if you only request SpeciesConc_O3 and SpeciesConc_NO, then the State_Diag%SpeciesConc array will only be allocated with (NX, NY, NZ, 2) slots instead of (NX, NY, NZ, State_Chm%nSpecies) slots. This will result in a substantial savings of memory and also CPU cycles.

This work, which leverages the DiagList and TaggedDiagList data structures implemented by @christophkeller and @lizziel, should be ready for 12.9.0. More information will be forthcoming soon.

@yantosca yantosca added the category: Feature Request New feature or request label Apr 3, 2020
@yantosca yantosca added this to the 12.9.0 milestone Apr 3, 2020
@yantosca yantosca self-assigned this Apr 3, 2020
@yantosca
Copy link
Contributor Author

yantosca commented Apr 8, 2020

Also need to handle the case if a diagnostic has more than one wildcard.

@msulprizio msulprizio modified the milestones: 12.9.0, 13.0.0 Jun 2, 2020
@yantosca
Copy link
Contributor Author

This has now been completed. Initial testing of the budget diagnostics shows a ~60% decrease in the time needed to compute budget diagnostics for only 2 species vs. budget diagnostics for all species. Further time testing is needed.

This update spans several commits, but the most recent are f847746 and geoschem/GCClassic@ed800c8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: Feature Request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants