-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement useful checks from kvalobs #37
Conversation
My initial thoughts on these questions: range check direction I do not understand this either. Y wind speed? What is special with -3.0?? Maybe PiM knows who to ask? logger_t This should just be our special_values_check, right? Maybe it says so since it is using the same flag, fr/controlinfo(1). I find explanations like "Gjoer tester paa 9999-verdier og returner en ny verdi for grenseverdi-flagg" and "fr=7 Kontrollert. Funnet å svare til spesialverdi som betyr manglende") min_gt I do not understand this after the first quick look. geonor_drift Geonor should be the precip buckets that fills with water as it rains. They are continuously weighed to measure the precipitation. -> values should be monotonically increasing. During the summer I guess some water can evaporate away. I do not know way they use an extra range check here. I can only see 1 flag in the log, not 4. On the consistency checks (badweather_highvisibility, mediumclouds_low, cloudbase_hi, lt_eq, gt_eq, maxtemp_nosign, range_max_min): consistency checks clouds uses 170, 171, 172, 173, and 175: and this one related to seatemp uses 179: and something related to temp uses 177 and 178 in the naming: also visibility/weather uses 176: Here I find the consistency checks for pressure variables: |
I think Y is wind speed, since it makes sense that wind direction would become meaningless when wind speed is 0, but it's not mentioned in the description. As for the -3.0 it seems like special handling for some specific sensor?
I think this is maybe easier to explain in person and with a diagram, we can bring it up at the next meeting
I meant that in your document it is listed as 4 entries:
But it's unclear to me what (if any) difference there could be between these checks, as the linked code has only 1 function which does not take in any params
Ok so those tests leaked into the list because the regex you used on the logs wasn't quite right? Or is it a mistake that they're named like that? In that case it seems like I shouldn't implement them, right? I feel kind of sad that I spent so long puzzling about this when it never made sense. You linked the new set of consistency checks that actually are used for pressure, do you want me to implement these? are they actually flagging anything? |
Sounds reasonable. I will put it on my list to try to ask some people to verify.
Perfect. It is now in the agenda for the next meeting.
It should be the same test (https://gitlab.met.no/met/obsklim/bakkeobservasjoner/data-og-kvalitet/kvalobs/kvoss_intern/-/blob/master/kvmeta/algorithms/geonor_drift.pl?ref_type=heads), but run for the different typeids (501,502,330,...,342).
Correct, leaked in w/ the regex. Maybe that is showing us that naming most of the tests with the paramid, but also naming lists of tests with the same numbers can be confusing... |
4278258
to
cbc6522
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still haven't fully looked into buddy_check
and src
🥲
(and fix lints we missed)
in step_check, spike_check, and flatline_check
038ae83
to
aea3e8f
Compare
Closes #34 and #1
- [ ] freeze_check_uu- [x] badweather_highvisibility (implemented as range_check_pair)- [ ] mediumclouds_low- prototype implemented as cloud_consistency_check, but need help to verify types, and provide a better description
- this check only seems to look at clouds, but in the doc it's marked as flagging pressure?
- [ ] cloudbase_hi- prototype implemented as cloud_consistency_check2, but same comments apply as mediumclouds_low.
- Additionally, I need help picking better names here
- as above, I am concerned that this is used to QC pressure but doesn't actually look at pressure
- [x] lt_eq- implemented as lower_limit_special_value_pair
- as above, I am concerned that this is used to QC pressure but doesn't actually look at pressure
- [x] gt_eq- implemented as upper_limit_special_value_pair
- as above, I am concerned that this is used to QC pressure but doesn't actually look at pressure