Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial duplicate of preservation event #1416

Closed
chalkos opened this issue Nov 23, 2018 · 3 comments
Closed

Partial duplicate of preservation event #1416

chalkos opened this issue Nov 23, 2018 · 3 comments
Assignees
Labels
Milestone

Comments

@chalkos
Copy link
Contributor

chalkos commented Nov 23, 2018

Steps to reproduce the bug:

  1. upload some file to transferred resources
  2. ingest it without auto-accept
  3. go to the aip and follow the "preservation events" link
  4. this happens:

image

  • reindexing solves it
  • the extra object only exists in solr, and only has an (unique, valid) UUID and an ID equal to other preservation event (in my testing it was always the ID of a preservation event with the same aipID)
  • in some cases it happens even without auto-accept

Solr document dumps:

Good preservation event:

SolrDocument{
  uuid=urn:roda:premis:event:5911c08b-118d-45b4-a07a-a4a4ca43cb4c,
  id=urn:roda:premis:event:5911c08b-118d-45b4-a07a-a4a4ca43cb4c,
  aipID=a453e702-a766-44e3-9728-45e4a3146a99, 
  objectClass=AIP, 
  eventDateTime=Fri Nov 23 16:18:01 WET 2018, 
  eventDetail=The ingest process has ended., 
  eventType=ingest end, 
  eventOutcome=SUCCESS, 
  linkingAgentIdentifier=[
    {
      "type":"URN",
      "value":"urn:roda:premis:agent:org.roda.core.plugins.plugins.ingest.ConfigurableIngestPlugin@1.0",
      "roles":[]
    }, 
    {
      "type":"URN",
      "value":"urn:roda:premis:agent:admin",
      "roles":[]
    }
  ],
  linkingSourceObjectIdentifier=[
    {
      "type":"URN",
      "value":"urn:roda:transferred_resource:asdfghjkl/jquery-3.3.1.min.js",
      "roles":["source"]
    },
    {
      "type":"URN",
      "value":"urn:roda:aip:a453e702-a766-44e3-9728-45e4a3146a99",
      "roles":["outcome"]
    }
  ],
  linkingOutcomeObjectIdentifier=[
    {
      "type":"URN",
      "value":"urn:roda:transferred_resource:asdfghjkl/jquery-3.3.1.min.js",
      "roles":["source"]
    },
    {
      "type":"URN",
      "value":"urn:roda:aip:a453e702-a766-44e3-9728-45e4a3146a99",
      "roles":["outcome"]
    }
  ],
  permission_users_GRANT=[admin], 
  permission_users_UPDATE=[admin], 
  permission_groups_GRANT=[administrators], 
  permission_users_CREATE=[admin], 
  _version_=1617942753665089536, 
  permission_groups_DELETE=[administrators], 
  permission_users_READ=[admin], 
  permission_groups_CREATE=[administrators], 
  permission_groups_READ=[administrators], 
  permission_users_DELETE=[admin], 
  state=UNDER_APPRAISAL, 
  permission_groups_UPDATE=[administrators]
}

Bad preservation event:

SolrDocument{
  uuid=urn:roda:premis:event:50949323-428f-3e98-b27c-bf49c87659f8,
  aipID=a453e702-a766-44e3-9728-45e4a3146a99,
  id=urn:roda:premis:event:5911c08b-118d-45b4-a07a-a4a4ca43cb4c,
  _version_=1617945685594734593,
  state=ACTIVE
}
@chalkos chalkos self-assigned this Nov 23, 2018
@chalkos chalkos added the bug label Nov 23, 2018
@chalkos chalkos added this to the 3.0 milestone Nov 23, 2018
@chalkos
Copy link
Contributor Author

chalkos commented Nov 27, 2018

More debug info, since I'm putting this on hold.

during ingest

debug is doc at the end of PreservationEventCollection.toSolrDocument and originating preservation metadata

~~~~>PreservationMetadata [id=urn:roda:premis:event:b5cb8f7d-6120-4fde-98b3-8e0bcbdb6f2f, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:b5cb8f7d-6120-4fde-98b3-8e0bcbdb6f2f (uuid and id match). detail: Extracted objects from package in file/folder format.

~~~~>PreservationMetadata [id=urn:roda:premis:event:b5cb8f7d-6120-4fde-98b3-8e0bcbdb6f2f, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:b5cb8f7d-6120-4fde-98b3-8e0bcbdb6f2f (uuid and id match). detail: Extracted objects from package in file/folder format.

~~~~>PreservationMetadata [id=urn:roda:premis:event:b7c669bc-5219-4f69-9d5f-522880f3964a, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:b7c669bc-5219-4f69-9d5f-522880f3964a (uuid and id match). detail: Checked that the received SIP is well formed, complete and that no unexpected files were included.

~~~~>PreservationMetadata [id=urn:roda:premis:event:a61655ad-fb93-4f0c-9103-7e2110c7478a, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:a61655ad-fb93-4f0c-9103-7e2110c7478a (uuid and id match). detail: The ingest process has started.

~~~~>PreservationMetadata [id=urn:roda:premis:event:8cd529ab-87e7-4a96-8612-fe0ca62717ff, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:8cd529ab-87e7-4a96-8612-fe0ca62717ff (uuid and id match). detail: Checked whether the descriptive metadata is included in the SIP and if this metadata is valid according to the established policy.

~~~~>PreservationMetadata [id=urn:roda:premis:event:ef464c41-b9ea-4f4d-a9b9-eac7ed96f836, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:ef464c41-b9ea-4f4d-a9b9-eac7ed96f836 (uuid and id match). detail: Created base PREMIS objects with file original name and file fixity information (SHA-256).

~~~~>PreservationMetadata [id=urn:roda:premis:event:c900e5f1-8cb7-42a6-91fc-a24facc80cb8, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:c900e5f1-8cb7-42a6-91fc-a24facc80cb8 (uuid and id match). detail: Identified the object's file formats and versions using Siegfried.

~~~~>PreservationMetadata [id=urn:roda:premis:event:b24018c4-a7b1-4b98-9dbc-8f635dc6a7af, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:b24018c4-a7b1-4b98-9dbc-8f635dc6a7af (uuid and id match). detail: User permissions have been checked to ensure that he has sufficient authorization to store the AIP under the desired node of the classification scheme.

~~~~>PreservationMetadata [id=urn:roda:premis:event:0f530afb-9802-4f22-9c5d-c3c37830dae8, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:0f530afb-9802-4f22-9c5d-c3c37830dae8 (uuid and id match). detail: Added package to the inventory. After this point, the responsibility for the digital content’s preservation is passed on to the repository.

~~~~>PreservationMetadata [id=urn:roda:premis:event:4d12f836-f6ee-4c7f-a9f4-651e7ac1b9c0, aipId=8dccc7b8-b367-4ae7-a6d9-ca4a1afb6807, representationId=null, fileDirectoryPath=null, fileId=null, type=EVENT]
----> doc with uuid: urn:roda:premis:event:4d12f836-f6ee-4c7f-a9f4-651e7ac1b9c0 (uuid and id match). detail: The ingest process has ended.

after ingest, listing preservation events for the AIP

doc at the start of PreservationEventCollection.fromSolrDocument

<---- doc with uuid: urn:roda:premis:event:4d12f836-f6ee-4c7f-a9f4-651e7ac1b9c0 (uuid and id match). detail: The ingest process has ended.
<---- doc with uuid: urn:roda:premis:event:0f530afb-9802-4f22-9c5d-c3c37830dae8 (uuid and id match). detail: Added package to the inventory. After this point, the responsibility for the digital content’s preservation is passed on to the repository.
<---- doc with uuid: urn:roda:premis:event:b24018c4-a7b1-4b98-9dbc-8f635dc6a7af (uuid and id match). detail: User permissions have been checked to ensure that he has sufficient authorization to store the AIP under the desired node of the classification scheme.
<---- doc with uuid: urn:roda:premis:event:c900e5f1-8cb7-42a6-91fc-a24facc80cb8 (uuid and id match). detail: Identified the object's file formats and versions using Siegfried.
<---- doc with uuid: urn:roda:premis:event:ef464c41-b9ea-4f4d-a9b9-eac7ed96f836 (uuid and id match). detail: Created base PREMIS objects with file original name and file fixity information (SHA-256).
<---- doc with uuid: urn:roda:premis:event:8cd529ab-87e7-4a96-8612-fe0ca62717ff (uuid and id match). detail: Checked whether the descriptive metadata is included in the SIP and if this metadata is valid according to the established policy.
<---- doc with uuid: urn:roda:premis:event:b7c669bc-5219-4f69-9d5f-522880f3964a (uuid and id match). detail: Checked that the received SIP is well formed, complete and that no unexpected files were included.
<---- doc with uuid: urn:roda:premis:event:b5cb8f7d-6120-4fde-98b3-8e0bcbdb6f2f (uuid and id match). detail: Extracted objects from package in file/folder format.
<---- doc with uuid: urn:roda:premis:event:a61655ad-fb93-4f0c-9103-7e2110c7478a (uuid and id match). detail: The ingest process has started.
<---- doc with uuid: urn:roda:premis:event:0880b18e-7403-313f-80a6-c08aa286f65c; id: urn:roda:premis:event:b7c669bc-5219-4f69-9d5f-522880f3964a. detail: null

@chalkos
Copy link
Contributor Author

chalkos commented Nov 27, 2018

@hsilva-keep @luis100 I think this might be related to the Solr collection changes (and there is also another problem with indexing the log which I still haven't found a way to reproduce).

Stacktraces: https://gist.github.com/chalkos/5ec06c85f1738616ce35bc54be4138d7 (these are from a different ingestion, so different IDs but the problem is there nonetheless)

@hsilva-keep hsilva-keep modified the milestones: 3.0, 3.1 Nov 29, 2018
@hsilva-keep hsilva-keep assigned hsilva-keep and unassigned chalkos Dec 6, 2018
@hsilva-keep hsilva-keep modified the milestones: 3.1.0, 3.0.1 Dec 6, 2018
hsilva-keep added a commit that referenced this issue Dec 6, 2018
…sed the events to not have permissions neither the AIP actual state (fixes #1416)
@hsilva-keep
Copy link
Member

Solved.
This problem was due to misuse of UUIDs when updating preservation events, which was causing indexed preservation events without correct state & permissions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants