Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Including aggregate operator. #241

Merged
merged 25 commits into from
Nov 17, 2023
Merged

Including aggregate operator. #241

merged 25 commits into from
Nov 17, 2023

Conversation

Sylviabohnenstengel
Copy link
Contributor

@Sylviabohnenstengel Sylviabohnenstengel commented Nov 9, 2023 •
edited by jfrost-mo

It requires a modification to the plot operator to allow spatial plots from 3D cubes that contain timeseries data.
#240

Fixes #169

jfrost-mo
jfrost-mo requested changes Nov 10, 2023
View reviewed changes
docs/source/contributing/testing.rst Outdated Show resolved Hide resolved
src/CSET/operators/aggregate.py Outdated Show resolved Hide resolved
src/CSET/recipes/aggregate_precipitation.yaml Outdated Show resolved Hide resolved
src/CSET/recipes/aggregate_precipitation.yaml Show resolved Hide resolved
src/CSET/operators/aggregate.py Outdated Show resolved Hide resolved
src/CSET/operators/aggregate.py Show resolved Hide resolved
jfrost-mo
jfrost-mo requested changes Nov 10, 2023
View reviewed changes
Copy link
Member

@jfrost-mo jfrost-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach is good, but we need to decide if we are just aggregating over time, or an arbitrary coordinate, and once that is decided changes need to be made to fit that decision.

src/CSET/operators/aggregate.py Outdated Show resolved Hide resolved
@jfrost-mo
Copy link
Member

I've pushed an update to the aggregate function that makes it more general, though I'm not entirely sure this is the way we want to go, as it may instead be better to have specific functions for aggregating over time, etc, as we would then be able to make them nicer to use.

@Sylviabohnenstengel
Copy link
Contributor Author

I've pushed an update to the aggregate function that makes it more general, though I'm not entirely sure this is the way we want to go, as it may instead be better to have specific functions for aggregating over time, etc, as we would then be able to make them nicer to use.

Based on discussion with are going with a single aggregation file that contains separate functions for time and horizontal and vertical aggregation as we then need to consider weighting functions as well.

@Sylviabohnenstengel
Copy link
Contributor Author

included timedelta functionality. This requires pip install isodate and I included that in pyproject and environments.yaml.
Recipes can now pass a timedelta to aggregate over following ISO 8601 format. This requires python package isodate to be included to convert timedelta format from PnYnMnDTnHnMnS into integer.

@jfrost-mo
Copy link
Member

Because the dependencies changed the conda lock files need updating. Will do that now.

jfrost-mo
jfrost-mo requested changes Nov 17, 2023
View reviewed changes
Copy link
Member

@jfrost-mo jfrost-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking nearly there. Lots of small style comments, but the main change is just renaming the operator from aggregate to time_aggregate now it is more specific.

docs/source/contributing/testing.rst Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
src/CSET/operators/aggregate.py Outdated Show resolved Hide resolved
src/CSET/operators/aggregate.py Outdated Show resolved Hide resolved
src/CSET/operators/aggregate.py Outdated Show resolved Hide resolved
tests/operators/test_aggregate.py Outdated Show resolved Hide resolved
tests/operators/test_aggregate.py Outdated Show resolved Hide resolved
tests/operators/test_aggregate.py Outdated Show resolved Hide resolved
tests/operators/test_aggregate.py Outdated Show resolved Hide resolved
tests/operators/test_aggregate.py Outdated Show resolved Hide resolved
@Sylviabohnenstengel @jfrost-mo
Update tests/operators/test_aggregate.py
6366d9b 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update docs/source/contributing/testing.rst
b415f82 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update pyproject.toml
680ffd0 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update src/CSET/operators/aggregate.py
701d729 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update src/CSET/operators/aggregate.py
9bca1b5 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update src/CSET/operators/aggregate.py
4ff1ac1 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update tests/operators/test_aggregate.py
0ad4318 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update tests/operators/test_aggregate.py
0eb95a5 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update tests/operators/test_aggregate.py
35a0355 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel @jfrost-mo
Update tests/operators/test_aggregate.py
730221d 
Co-authored-by: James Frost <james.frost@metoffice.gov.uk>
@Sylviabohnenstengel
Copy link
Contributor Author

@jfrost-mo thanks for pointing out the rest :-) all changes made as requested.

Sylviabohnenstengel
Sylviabohnenstengel commented Nov 17, 2023
View reviewed changes
Copy link
Contributor Author

@Sylviabohnenstengel Sylviabohnenstengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy with changes.

Sylviabohnenstengel
Sylviabohnenstengel commented Nov 17, 2023
View reviewed changes
Copy link
Contributor Author

@Sylviabohnenstengel Sylviabohnenstengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy with suggestions

Sylviabohnenstengel
Sylviabohnenstengel commented Nov 17, 2023
View reviewed changes
Copy link
Contributor Author

@Sylviabohnenstengel Sylviabohnenstengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy with suggestions

@jfrost-mo
Copy link
Member

Just want to get the tests passing now, ahead of merging. I think I just caught the last aggregate rather than time_aggregate.

Sylviabohnenstengel
Sylviabohnenstengel commented Nov 17, 2023
View reviewed changes
Copy link
Contributor Author

@Sylviabohnenstengel Sylviabohnenstengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy with changes

@Sylviabohnenstengel
Copy link
Contributor Author

still to be implemented:
For PERCENTILE YAML file requires i.e. method:
'PERCENTILE' additional_percent: 90.

@jfrost-mo jfrost-mo self-requested a review November 17, 2023 09:51
Sylviabohnenstengel
Sylviabohnenstengel commented Nov 17, 2023
View reviewed changes
Copy link
Contributor Author

@Sylviabohnenstengel Sylviabohnenstengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hopefully all changes addressed

@jfrost-mo
Copy link
Member

Just need to get the tests passing now. But it seems there is a difference in the number of coordinates...

@github-actions GitHub Actions
Copy link
Contributor

Coverage

jfrost-mo
jfrost-mo approved these changes Nov 17, 2023
View reviewed changes
Copy link
Member

@jfrost-mo jfrost-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are passing, and code looks good. Go ahead and merge! 🚀

@jfrost-mo jfrost-mo merged commit afab373 into main Nov 17, 2023
5 checks passed
@jfrost-mo jfrost-mo deleted the Aggregate_sylvia branch November 17, 2023 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jfrost-mo jfrost-mo jfrost-mo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

New CSET operator aggregate
2 participants
@Sylviabohnenstengel @jfrost-mo