-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Netcdf multiple samples a day #268
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2 comments in geotiff, would it be possible to already apply changes of netcdfrddwriter to develop branch?
Duration.between(fixedTimeOffset, key.time).getSeconds % secondsPerDay != 0 | ||
case _ => | ||
false | ||
}).isEmpty()) TemporalResolution.days else TemporalResolution.seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm afraid that doing this on the rdd triggers an extra job in Spark.
Suggestion would be to just choose the resolution when you create the filename. Consequence is that in theory, filename format could change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Merged the NetCDF part, the Tiff part is split to a different MR: #269
Duration.between(fixedTimeOffset, key.time).getSeconds % secondsPerDay != 0 | ||
case _ => | ||
false | ||
}).isEmpty()) TemporalResolution.days else TemporalResolution.seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm afraid that doing this on the rdd triggers an extra job in Spark.
Suggestion would be to just choose the resolution when you create the filename. Consequence is that in theory, filename format could change.
For tiff and netcdf.
When saving the tiles, we usually round the date to daily resolution.
Now, when there is a date that does not exactly match midnight, we assume the temporal resolution is higher, and we save the results with seconds in resolution.
For tiff export, the ':' in the date part is replaced with '-', to avoid problems on Windows file systems: Example
openEO_2017-01-01T01-00-00Z.tif
I used
ISO_ZONED_DATE_TIME
in the filename. Following an example I found ingeotiff/package.scale