Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate results after SingleTestRun/EnsembleExperiment #7966

Open
oyvindeide opened this issue May 24, 2024 · 1 comment
Open

Validate results after SingleTestRun/EnsembleExperiment #7966

oyvindeide opened this issue May 24, 2024 · 1 comment

Comments

@oyvindeide
Copy link
Collaborator

Sometimes it is a bit hard for users to tell if all the required observations and parameters are loaded in the callback. In AHM we allow some observations to be missing, and only crash if all observations are deactivated.

The REFCASE has some functionality to check that the correct dates are present for summary data, however there has historically been a problem that it is not necessarily in sync with the schedule file, and as such, might give false positives or negatives.

One way we can improve this, is by validating that SingleTestRun and EnsembleExperiment validate the results for missing parameters and responses. This can be as simple as raising an exception when it has completed if something is missing, and everything should show up correctly in the gui.

@frode-aarstad frode-aarstad self-assigned this Jun 5, 2024
@oyvindeide
Copy link
Collaborator Author

We can do something similar to here:

def _load_observations_and_responses(
, i.e. load all the observations and check if any are missing, similar to what we do in the update report:
if self.status == ObservationStatus.MISSING_RESPONSE:

We already validate that the forward model is running as expected, with this we also validate all the observations.

@frode-aarstad frode-aarstad mentioned this issue Jun 5, 2024
9 tasks
@frode-aarstad frode-aarstad removed their assignment Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

2 participants