Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: Cannot create more than one SIP from a transfer #270

Closed
peterVG opened this issue Oct 12, 2018 · 3 comments

Comments

Projects
None yet
5 participants
@peterVG
Copy link

commented Oct 12, 2018

Expected behaviour
When arranging SIPs from backlog on the Ingest tab, it should be possible to make multiple SIPs from one or more transfer sources, as long as the same files are not included in more than one SIP.

screen shot 2018-10-12 at 10 21 01

Current behaviour

After successfully creating a first SIP, the "Create SIP" option for a second SIP hangs indefinitely, i.e. the spinning wheel to the right runs indefinitely without creating the second SIP.

screen shot 2018-10-12 at 10 25 20

Steps to reproduce

  1. Create a transfer and send it to backlog (e.g. using "/SampleTransfers/DemoTransfer").
    screen shot 2018-10-12 at 10 01 38
  2. Create two SIP directories under the "arrange" directory on the Ingest tab and add one or two files from the transfer to each one.
  3. Select the first SIP and press the "Create SIP" option to create the first SIP.
  4. Select the second SIP in the "arrange" pane and press the "Create SIP" option.

Your environment (version of Archivematica, OS version, etc)

AM 18x.qa (Bionic), AM 18x.qa (Xenial), AM 17x.qa (Xenial)


For Artefactual use:
Please make sure these steps are taken before moving this issue from Review to Verified in Waffle:

  • All PRs related to this issue are properly linked 👍
  • All PRs related to this issue have been merged 👍
  • Test plan for this issue has been implemented and passed 👍
  • Documentation regarding this issue has been written and it has been added to the release notes, if needed 👍

@peterVG peterVG added this to the 1.8.0 milestone Oct 12, 2018

@sevein sevein changed the title PROBLEM: Cannot create more than one SIP from a transfer Problem: Cannot create more than one SIP from a transfer Oct 12, 2018

@sallain

This comment has been minimized.

Copy link
Contributor

commented Oct 12, 2018

Confirmed this on 1.7.2 (augustus) but using the Appraisal tab - I can create my first SIP, with two of the objects from my transfer, but when I try to create the second SIP it fails to create. I also tried this on 1.8 (bionic) and experienced the same.

To recreate:

  1. Run a transfer through to backlog. In these tests I used the DemoTransfer folder.
  2. In the Appraisal tab, search for items in backlog.
  3. Using the Arrangement pane, click "Add directory" and create a directory, Dir1.
  4. Drag a couple of items from the backlog to Dir1, and then select Dir1 and click Create SIP

You should get a green banner with a success message, and Dir1 should disappear from the Arrangement tab.

  1. Without refreshing the page, create another new directory, Dir2. Dir2 will not appear in the Arrangement pane.
  2. Refresh the page - you will see that Dir1 still exists and that Dir2 is nested inside it. Delete Dir1 (which also deletes Dir2).
  3. Create a third new directory, Dir3. There are still items remaining in the backlog from our original transfer. Drag a couple of items from the backlog to Dir3, and then select Dir3 and click Create SIP.

A pink banner will appear with the message SIP could not be started! Check dashboard logs.

@sevein

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2018

This was caused by artefactual/archivematica-storage-service#320 which introduced the ability to use rename (effectively moving) instead of rsync (effectively copying) between locations as a performance improvement. The downside is that the originals are not present in the backlog anymore, but they're expected to be. There are mechanisms in place (SIPArrange) to hide already transferred items from the backlog, but things like logs and metadata are not expected to be moved like artefactual/archivematica-storage-service#320 is doing.

The endpoint that we're using in SS is Move files to this location. It claims that it moves but I think we've been relying on the fact that it used to copy instead - until artefactual/archivematica-storage-service#320.

If this is correct, a potential solution could be to have the endpoint learn a new parameter to disallow moving even when it's possible.

CC @jhsimpson.

qubot pushed a commit to artefactual/archivematica-storage-service that referenced this issue Oct 25, 2018

Location: disallow move based on purpose
The `posix_move` op should not be always allowed for certain locations.
This commit restricts it for backlog and AIP storage.

This is connected to archivematica/Issues#270.

qubot pushed a commit to artefactual/archivematica-storage-service that referenced this issue Oct 25, 2018

Location: disallow move based on purpose
The `posix_move` should not be always allowed for certain locations. This
commit ensures that it's not used when the original location is backlog or
AIP storage.

This is connected to archivematica/Issues#270.

qubot pushed a commit to artefactual/archivematica-storage-service that referenced this issue Oct 25, 2018

Location: disallow move based on purpose
The `posix_move` should not be always allowed for certain locations. This
commit ensures that it's not used when the original location is backlog or
AIP storage.

This is connected to archivematica/Issues#270.

qubot pushed a commit to artefactual/archivematica-storage-service that referenced this issue Oct 26, 2018

Location: disallow move based on purpose
The `posix_move` should not be always allowed for certain locations. This
commit ensures that it's not used when the original location is backlog or
AIP storage.

This is connected to archivematica/Issues#270.

@sallain sallain assigned sallain and unassigned sevein Oct 26, 2018

@sallain

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2018

This seems to be working now! I was able to split a transfer into two SIPs without any issue. I didn't even have to refresh the page.

There is a bit of a lag between starting an arranged SIP from the Appraisal tab and the folder that you start the SIP from disappearing from the Arrangement pane, about a minute, but I don't think it's a big deal.

@sevein sevein added Type: bug and removed Type: bug bug labels Oct 31, 2018

@sromkey sromkey closed this Nov 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.