Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

verdi migrate --in-place between file systems #4393

Merged
merged 2 commits into from
Oct 26, 2020

Conversation

ltalirz
Copy link
Member

@ltalirz ltalirz commented Sep 24, 2020

fix #4392

verdi import --in-place assumed that the temporary directory for the
migrated archive and the working directory are on the same file system.
If that was not the case, it would crash with

OSError: [Errno 18] Invalid cross-device link

Switching from os.rename to shutil.move should resolve this issue.

verdi import --in-place assumed that the temporary directory for the
migrated archive and the working directory are on the same file system.
If that was not the case, it would crash with

  OSError: [Errno 18] Invalid cross-device link

Switching from os.rename to shutil.move should resolve this issue.
@ltalirz ltalirz force-pushed the issue_4329_verdi_import_in_place branch from 9c979ff to 7a691fa Compare September 24, 2020 12:21
@codecov
Copy link

codecov bot commented Sep 24, 2020

Codecov Report

Merging #4393 into develop will decrease coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #4393      +/-   ##
===========================================
- Coverage    79.36%   79.35%   -0.00%     
===========================================
  Files          476      476              
  Lines        34950    34951       +1     
===========================================
- Hits         27734    27733       -1     
- Misses        7216     7218       +2     
Flag Coverage Δ
#django 73.22% <100.00%> (+0.01%) ⬆️
#sqlalchemy 72.45% <100.00%> (-<0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aiida/cmdline/commands/cmd_export.py 90.98% <100.00%> (+0.07%) ⬆️
aiida/engine/daemon/client.py 72.42% <0.00%> (-1.14%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 861a39f...ee50227. Read the comment docs.

@greschd
Copy link
Member

greschd commented Sep 24, 2020

Just to mention: While os.rename is guaranteed to be atomic (on Linux), AFAIK think shutil.move makes no such promises. I guess that's why it's able to move between file systems in the first place.

Not sure if that's a guarantee that we need here - I guess it would be nice to be protected against corruption of the in-place-migrated archive. But then again, one can also not do it in place in cases where that's really important.

@ltalirz ltalirz added the pr/ready-for-review PR is ready to be reviewed label Oct 5, 2020
@ltalirz ltalirz requested a review from sphuber October 20, 2020 08:35
@sphuber sphuber changed the title verdi import --in-place between file systems verdi migrate --in-place between file systems Oct 26, 2020
@sphuber sphuber merged commit 7c49471 into aiidateam:develop Oct 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/ready-for-review PR is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

verdi import --in-place crashes when export file is on different file system
3 participants