Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DURACLOUD-1310: use number in chunkId when index number is possibly missing in a chunk manifest #140

Merged
merged 2 commits into from Jun 10, 2021

Conversation

nwoodward
Copy link
Contributor

This PR changes the file stitching process to handle cases where the index number is missing from entries in a chunk manifest.


JIRA Ticket: https://duracloud.atlassian.net/browse/DURACLOUD-1310

What does this Pull Request do?

Adds a comparison between the index number and the number at the end of each chunkId.

How should this be tested?

  1. Use the SyncTool to upload a large file that gets chunked to a DuraCloud space.
  2. Modify only the dura-manifest file in the space by removing the index=X tag from each tag.
  3. Use the production version of the RetrievalTool to retrieve the file. It will fail with a checksum mismatch error.
  4. Use this snapshot version of the RetrievalTool to retrieve the file.

Interested parties

Tag @duracloud/committers

Copy link
Member

@bbranan bbranan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested this and it works as expected, solving the problem with missing index entries in the manifest xml. The code looks good as well except that there is currently more logging than is really needed long term.

In the case where the manifest index and parsed index match, I see no need to log as everything is working as expected. I can see including an info-level log when they don't match, but perhaps updating the text to be more understandable by the end-user that is likely to be seeing the message. Maybe something like:
log.info("The entry in the chunk manifest for chunk {} is missing an index field; using the index from the filename instead.", entry.getChunkId());

@bbranan bbranan merged commit e6509c6 into duracloud:develop Jun 10, 2021
fozboz pushed a commit to scholarsportal/duracloud that referenced this pull request Jul 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants