Ensemble ingestation with read operator #157
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
jfrost-mo
force-pushed
the
ensemble-ingestation
branch
from
f9d1524
to
3ebb96a
Compare
Merged
jfrost-mo
force-pushed
the
ensemble-ingestation
branch
2 times, most recently
from
fad32a0
to
69df812
Compare
jfrost-mo
commented
Jul 14, 2023
jfrost-mo
force-pushed
the
ensemble-ingestation
branch
2 times, most recently
from
36c631c
to
9438a45
Compare
jwarner8
reviewed
Jul 14, 2023
jwarner8
reviewed
Jul 14, 2023
jwarner8
reviewed
Jul 14, 2023
jfrost-mo
force-pushed
the
ensemble-ingestation
branch
from
9438a45
to
7fab2bc
Compare
jfrost-mo
force-pushed
the
ensemble-ingestation
branch
from
c9594fd
to
7fab2bc
Compare
|
I think this PR is ready to merge, but I've sort of forgotten what is in it over the past two weeks, and want to refamiliarise myself with it first. |
jfrost-mo
commented
Aug 1, 2023
Allow read_cubes operator to read from multiple files This is done by giving the path to a directory. A shell wildcard can also be given to match only specific files there. Add reading code for ensemble data Reads ensemble data, and will add the member number/realization from the file name if it is missing. read_cubes now reads ensemble data & adds realization Actually add realization to deterministic cubes Make realization number detection more robust Expand documentation of read_cubes operator Simplify ensemble realization retrieval from file name The naming conventions aren't really knowable until partners actually use it.
Add test stubs for ensemble reading Added tests for ensemble loading with read_cube Also added some ensemble test data which is a 2 member ensemble cropped down to just Exeter. Test read_cubes with ensemble and constraint Test fieldsfile ensemble filename parser
jfrost-mo
force-pushed
the
ensemble-ingestation
branch
from
b2a7599
to
03a328a
Compare
|
I've had a look back over and rebased this PR, so with your approval @jwarner8 it is ready to merge. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improves the read operator so it can read from multiple files, and correctly tag ensemble data with a realization coordinate.
Still to do:
Allow more generic templating for where the ensemble number is in the name? Depends if this is different for partners.We'll leave this for now as it is basically impossible to know all the naming conventions until partners actually use it, and its an easy thing to change.Fixes #133