-
Notifications
You must be signed in to change notification settings - Fork 279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cube list concatenation oddity #3696
Comments
btw I just gave you a star so now you have to answer 😀 🍺 |
Hi @valeriupredoi, sorry for the delay getting back to you on this. In terms of what Iris should do here, what's the ideal behaviour for you? My go-to list of options (below) mostly have big enough downsides that I don't think we'd want to implement them. The behaviour is currently expected to "warn" (I realise it's a cryptic warning) by returning you (from your example) My options and their downsides:
Let me know what you think! |
We already have concatenate_cube. |
cheers muchly @wjbenfold for your reply and being willing to improve the functionality - I have just opened an issue where we may discuss this from our end's point, it may as well be that people are perfectly fine with the current functionality so we'll close this, but let's give my fellow devs a couple days see if there are any suggestions/complaints 😁 Many thanks to you guys for being well responsive to our input BTW 🍺 |
An instance of this just came up on MO Yammer. In that particular case, the overlapping period had equal data, metadata, etc. So in that case Iris might reasonably be expected to throw away the duplicate part and concatenate anyway. How easy that would be to implement I've no idea! |
My immediate thought is that we'd at least have to realise the data to do that check, which might be enough to rule it out without at least a special mode that users have actively chosen? |
I notice that over at ESMValGroup/ESMValCore#1423, @zklaus has argued that such functionality should be added in downstream packages to address specific use-cases. We could maybe improve the error message from import iris
fname = iris.sample_data_path("SOI_Darwin.nc")
complete_cube = iris.load_cube(fname)
overlapping_parts = iris.cube.CubeList([complete_cube[:100], complete_cube[50:]])
overlapping_parts.concatenate_cube()
|
What do you reckon would be more helpful @rcomer? Given that a failure of the cubes to all look the same will, from my understanding, fail inside concatenate because of
If that seems sensible then I can put up the PR to make it happen (or you can and I'll review it) |
Looking back at this issue, if there's future appetite for functionality that handles overlap, I can imagine an argument to |
I think |
We could add descriptive errors all the way through that function ( It looks like |
Put this in Iris 3.7 , but can't commit to resolving it : we will look into it now |
* first draft of fix with comments to be removed * Removed comments, added warning * responded to pull request * Updated tests and test warnings * fixed dtypes on bounds tests * Updated "what's new" docs section * added link to github page --------- Co-authored-by: alex.chamberlain-clay <achamber@vld760.cmpd1.metoffice.gov.uk>
Closed by #5382 |
Hey guys, hope yous doing well and healthy and working from home 😁
I got a question described in this comment and it seems to me there has to be some sort of check in
iris
for such cases. here goes:iris.cube.CubeList(cubes).concatenate()
[A, B, C, D]
where A spans over a time t, B continues from where A stopped and spans for another time t, C starts a bit before B finishes and spans for another time t, D starts where C stopped and spans for another time t (t is arbitrary, what matters is where cubes start and end on the time axis)iris.cube.CubeList([A, B, C, D]).concatenate()
is[M=(A+B+D), C]
This is pretty bad since there is a serious time gap between end of B and start of D (where C should be, partially) -> is this something that you would consider writing a warning or a fix for? It's particularly difficult to notice such gapey concatenations when dealing with, say, multiple CMIP experiments to be concatenated into one.
If you guys say meh, then we'll plug a fix in our concatenation wrapper, but I just wanted to alert you on this - 🍺
The text was updated successfully, but these errors were encountered: