Skip to content

Nexus: Handle pickle protocols between python versions - #4385

Merged
ye-luo merged 4 commits into
QMCPACK:developfrom
kayahans:develop
Jan 8, 2023
Merged

ye-luo merged 4 commits into
QMCPACK:developfrom
kayahans:develop

Conversation

@kayahans

@kayahans kayahans commented Jan 6, 2023

Copy link
Copy Markdown
Contributor

Proposed changes

Handles the pickle (https://docs.python.org/3/library/pickle.html) protocol differences that can arise due to using different python versions, (such as using multiple computers having different python versions, but sharing the same filesystem). In the current code, if python 3.8 is used to generate pickle (.p) files, nexus uses protocol 5. If python 3.7 is used to rerun the same script, protocol 5 is not accessible in that python version (https://docs.python.org/3/library/pickle.html#data-stream-format)

What type(s) of changes does this code introduce?

  • Refactoring (no functional changes, no api changes)

Does this introduce a breaking change?

  • No

What systems has this change been tested on?

Andes, Summit

Checklist

Update the following with a yes where the items apply. If you're unsure about any of them, don't hesitate to ask. This is
simply a reminder of what we are going to look for before merging your code.

  • Yes. This PR is up to date with current the current state of 'develop'
  • No. Code added or changed in the PR has been clang-formatted
  • No. This PR adds tests to cover any new code, or to catch a bug that is being fixed
  • No. Documentation has been added (if appropriate)

@prckent prckent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add exception handling on the pickle5 import and print a message about why it is needed if the import fails.

The mixed python situation is fairly common. e.g. Old system python, user installed recent python.

@prckent

prckent commented Jan 6, 2023

Copy link
Copy Markdown
Contributor

While tweaking this code, add some error handling on the open call? Or is the file always guaranteed to be there and readable at this point?

@kayahans

kayahans commented Jan 6, 2023

Copy link
Copy Markdown
Contributor Author

Import error exception is added, please check. File is always guaranteed to be there and readable per nexus workflows.

@prckent prckent changed the title Handle pickle protocols between python versions Nexus: Handle pickle protocols between python versions Jan 7, 2023

@prckent prckent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I tweaked the message since there are many ways to install python modules and pip might not be the relevant one.

@prckent

prckent commented Jan 7, 2023

Copy link
Copy Markdown
Contributor

Test this please

@ye-luo
ye-luo enabled auto-merge January 8, 2023 15:31
@ye-luo

ye-luo commented Jan 8, 2023

Copy link
Copy Markdown
Contributor

Test this please

@ye-luo
ye-luo merged commit 89f64e0 into QMCPACK:develop Jan 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants