Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[minor] Update pack and unpack methods #1440

Merged
merged 85 commits into from
Aug 16, 2024
Merged

[minor] Update pack and unpack methods #1440

merged 85 commits into from
Aug 16, 2024

Conversation

samwaseda
Copy link
Member

@samwaseda samwaseda commented May 23, 2024

Update of the pack and unpack methods. #775

  • pack includes the csv file inside the archive
  • pack filename is optional, uses ".tar.gz"
  • pack with same name as project should not delete the project
  • pack selected jobs by id
  • pack all files in a job
  • pack from a different directory than where project is located
  • unpack method can be called as pr = Project(filename.tar.gz, unpack=True) or pr = Project(filename, unpack=True)
  • unpack should not nest project automatically
  • unpack jobs into existing Project
  • Update tests
  • Update docstrings
  • Update workflow template

Potentially backward-incompatible changes:

  • I removed the argument compress from unpack. You have to put the full file name so that pyiron knows whether it has to decompress or not.
  • Now export.csv is inside the compressed folder. The algorithm could fail if there is another export.csv in the project.

@samwaseda
Copy link
Member Author

I am going to take over @srmnitc's work, so I copied the PR from his forked branch

@samwaseda
Copy link
Member Author

ok I just realized that I had a different PR open and I opened this one on top. Let me correct this first XD

@samwaseda samwaseda changed the base branch from main to force_bool_restart May 23, 2024 08:57
Base automatically changed from force_bool_restart to main May 23, 2024 08:57
@jan-janssen
Copy link
Member

Thanks @samwaseda for picking this up. There are a couple of fixes also included in #1401 so maybe it makes sense to merge those as well.

@samwaseda samwaseda changed the base branch from main to combine_example_notebooks May 23, 2024 13:52
Base automatically changed from combine_example_notebooks to main May 23, 2024 14:03
@jan-janssen jan-janssen marked this pull request as draft June 11, 2024 19:54
@samwaseda samwaseda requested review from pmrv and srmnitc August 15, 2024 16:38
@samwaseda
Copy link
Member Author

@pmrv @jan-janssen @srmnitc I guess it's now feature complete and this is probably going to be the last PR of the series of pack/unpack. After this one a minor release can be made and it's hopefully settled.

@jan-janssen
Copy link
Member

As far as I understand the csv file is now included in the tar archive, correct? How do you handle the backwards compatibility for old archives which do not include the csv file? Maybe it makes sense to have a short example either in the Docstring or the jupyter notebook to handle this backwards compatibility. Finally, I liked the option to take a look at the csv file to see which jobs are included in the archive before importing the corresponding jobs. Previously, I did this by loading the csv file with pandas. I can still load the csv file manually from the tar archive, but for other users it would be great to take a look at the archive and get the job table of the contained project from the python side.

samwaseda and others added 2 commits August 15, 2024 20:59
Co-authored-by: Jan Janssen <jan-janssen@users.noreply.github.com>
Co-authored-by: Jan Janssen <jan-janssen@users.noreply.github.com>
@samwaseda
Copy link
Member Author

As far as I understand the csv file is now included in the tar archive, correct? How do you handle the backwards compatibility for old archives which do not include the csv file? Maybe it makes sense to have a short example either in the Docstring or the jupyter notebook to handle this backwards compatibility.

That's a good point that I should mention in the code. I guess it helps the future generation to understand the origin of some of the code.

Finally, I liked the option to take a look at the csv file to see which jobs are included in the archive before importing the corresponding jobs. Previously, I did this by loading the csv file with pandas. I can still load the csv file manually from the tar archive, but for other users it would be great to take a look at the archive and get the job table of the contained project from the python side.

That sounds good but I guess it's a different PR.

@samwaseda
Copy link
Member Author

Can I merge this one or should I still wait for a review?

@samwaseda samwaseda merged commit ffff683 into main Aug 16, 2024
26 checks passed
@samwaseda samwaseda deleted the pack_unpack branch August 16, 2024 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor add functionality in a backward compatible manner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants