Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

706 writing large volume of data #708

Merged
merged 5 commits into from
Sep 22, 2023
Merged

Conversation

mauzey1
Copy link
Collaborator

@mauzey1 mauzey1 commented Sep 22, 2023

Resolves #706

This PR will replace the use of int with size_t in the function cmor_write_var_to_file when calculating the number of elements in an array. This will allow cmor_write to write data arrays with more than 2^31-1 elements.

Copy link
Contributor

@durack1 durack1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mauzey1 this is great, thanks!

There maybe some very subtle changes required to deal with the new MIP-cmor-tables, a string -> list type change for coord descriptions, so then we can wrap that up as 3.7.3 or 3.8 if the osx-arm64 change happens in the same release

Ping @matthew-mizielinski @taylor13

@durack1
Copy link
Contributor

durack1 commented Sep 22, 2023

We'll need to check on those Linux build failures too

@mauzey1
Copy link
Collaborator Author

mauzey1 commented Sep 22, 2023

There maybe some very subtle changes required to deal with the new MIP-cmor-tables, a string -> list type change for coord descriptions

Could you elaborate on this feature? Please post an issue detailing it.

We'll need to check on those Linux build failures too

The conda rerender step is taking too long when installing packages for some reason. I encountered that issue when working on another branch but I was able to resolve it by just rerunning the failed workflows. I will try rerunning them after the OSX builds are done.

@mauzey1 mauzey1 merged commit 292efd9 into master Sep 22, 2023
@mauzey1 mauzey1 deleted the 706_writing_large_volume_of_data branch September 22, 2023 23:42
@durack1
Copy link
Contributor

durack1 commented Sep 23, 2023

@mauzey1 will do (post a new issue). I am travelling next week and will meet with Matt in person, so we can figure out what is required and document that in an issue.

And yep, you're right, second pass on circleci passes fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CMOR failure when attempting to write large volume of data
2 participants