Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping of statistics doesn't work when "num_valid" is 0 #1080

Closed
lewisblake opened this issue Mar 26, 2024 · 0 comments · Fixed by #1111
Closed

Dropping of statistics doesn't work when "num_valid" is 0 #1080

lewisblake opened this issue Mar 26, 2024 · 0 comments · Fixed by #1111
Assignees
Labels
bug 🐛 Something isn't working
Milestone

Comments

@lewisblake
Copy link
Member

lewisblake commented Mar 26, 2024

Describe the bug
Please provide a clear and concise description of what the bug is.

  • Pyaerocom version: 0.16.0 (with unreleased updates from main-dev)
  • Computing platform: PPI
  • Configuration file (if applicable): /lustre/storeB/users/lewisb/Python/Evaluations/aeroval/config/config_files/obsonly/cfg_obs-meteorology.py
  • Error message (if applicable): NA, the run doesn't crash.

To Reproduce
Steps to reproduce the behavior:

  1. cd /lustre/storeB/users/lewisb/Python/Evaluations/aeroval/config/config_files/obsonly
  2. qsub run_obs-meteorology.sh
  3. Check files in map

Expected behavior
All statistics provided in drop_stats should be dropped from all fields, not just those that have valid data points.

Screenshots
Not required.

Additional context
Dropping all the stats in all fields is important to reduce file size. The obs-only runs can be very large (previously 60G). They're now about 49G but we want to reduce all the unnecessary clutter from these files.

@lewisblake lewisblake added the bug 🐛 Something isn't working label Mar 26, 2024
@lewisblake lewisblake added this to the m2024-05 milestone Mar 26, 2024
@lewisblake lewisblake self-assigned this Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants