Should version information contain a digest for the previous version?

[zimeon]

From 2018-03-28 meeting notes there was discussion about the possibility of:

JSON-LD file for each version could contain the checksum of the JSON-LD file of the previous version

• This eliminates the additional Namaste file in each object version
• Each version is validated entirely by the next
• The latest version is validated by the top level SHA (since it’s JSON LD is the same as the top level one)
• This does not need to be computed when a new version is created, it is just a copy of the top level one

[ahankinson]

The context being that we needed redundancy in the version-tracking, so that losing one write doesn't knock out the entire history.

[ahankinson]

Need to clarify with example in specification

[zimeon]

Having decided in #8 that we will use sha-512 the use of Namaste files would be rather ugly/long

[ahankinson]

Need to change the spec to use forward-version: put previous version file in new version folder

[rosy1280]

@julianmorley: we can't put files into the previous version directory once we mint a new version.

all agree to this.

[rosy1280]

i edited the original ticket to clarify that the JSON-LD is the JSON-LD file, not information in the JSON-LD.

[ahankinson]

@ahankinson to work up an example directory structure.

.
├── inventory.jsonld
├── v1
    └── inventory.jsonld  (v1)  <--- does not contain any inventory checksum
└── v2
    └── inventory.jsonld  (v2)  <--- contains checksum of v1 inventory.jsonld

[rosy1280]

where do we put the checksum for the root inventory.jsonld

[ahankinson]

@neilsjefferies to come up with an example

@ahankinson to integrate that into a gist

[neilsjefferies]

Checksum of root stored in root as a inventory.jsonld.checksum file. Format of contents of the file is the same as that used for the checksum of v1/inventory.jsonld stored in v2/inventory.jsonld

[ahankinson]

@neilsjefferies could you give a more concrete example of the format of the contents?

Perhaps we could follow the convention used by package distributors with checksum files and store them as inventory.sha512 or inventory.jsonld.sha512, with the contents of the file being something like:

[some long sha512 has] inventory.jsonld

It sounds like you're proposing something different, Neil, so it would be good to make that more concrete.

[neilsjefferies]

Yes, Since the checksum for the /v1/Inventory.jsonld in /v2/invnterory.jsonld should be in JSON-LD then my suggestion would be that the entry in inverntory.jsonld.sha512 should also be in JSON-LD. That is all. Neil

…

On 2018-05-23 18:47, Andrew Hankinson wrote: @neilsjefferies [1] could you give a more concrete example of the format of the contents? Perhaps we could follow the convention used by package distributors with checksum files and store them as inventory.sha512 or inventory.jsonld.sha512, with the contents of the file being something like: [some long sha512 has] inventory.jsonld It sounds like you're proposing something different, Neil, so it would be good to make that more concrete. -- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub [2], or mute the thread [3]. { ***@***.***": "MessageCard", ***@***.***": "http://schema.org/extensions", "hideOriginalBody": "false", "originator": "37567f93-e2a7-4e2a-ad37-a9160fc62647", "title": "Re: [OCFL/spec] Should version information contain a digest for the previous version? (#7)", "sections": [ { "text": "", "activityTitle": "**Andrew Hankinson**", "activityImage": "https://avatars0.githubusercontent.com/u/163183?s=160\u0026v=4", "activitySubtitle": ***@***.***", "facts": [ ] } ], "potentialAction": [ { "name": "Add a comment", ***@***.***": "ActionCard", "inputs": [ { "isMultiLine": true, ***@***.***": "TextInput", "id": "IssueComment", "isRequired": false } ], "actions": [ { "name": "Comment", ***@***.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"IssueComment\",\n\"repositoryFullName\": \"OCFL/spec\",\n\"issueId\": 7,\n\"IssueComment\": \"{{IssueComment.value}}\"\n}" } ] }, { "name": "Close issue", ***@***.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"IssueClose\",\n\"repositoryFullName\": \"OCFL/spec\",\n\"issueId\": 7\n}" }, { "targets": [ { "os": "default", "uri": "#7 (comment)" } ], ***@***.***": "OpenUri", "name": "View on GitHub" }, { "name": "Unsubscribe", ***@***.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"MuteNotification\",\n\"threadId\": 320306098\n}" } ], "themeColor": "26292E" } Links: ------ [1] https://github.com/neilsjefferies [2] #7 (comment) [3] https://github.com/notifications/unsubscribe-auth/AkJz40RQppNMattMKRPwXMIZRYTtx1DMks5t1aCqgaJpZM4TF3uy

[julianmorley]

I worry that putting the prior version inventory files in version directories will make it hard to have immutable archive versions of those directories.

If we alter the structure of the inventory file to be composed of a manifest + deltas, then the most current inventory file can be used to reconstitute the object at any prior revision. If we wished to save the prior version of the inventory file, could we rename it slightly and place it in the /logs directory?

e.g.

├── inventory.jsonld  (v3)          <-- contains checksum from v2
├── inventory.jsonld.sha512         <-- contains the checksum of current version (v3)
├── logs
    ├── inventory.jsonld.v1.sha512  <--- contains the original (v1) inventory checksum
    ├── inventory.jsonld.v1  (v1)   <--- does not contain any inventory checksum
    └── inventory.jsonld.v2  (v2)   <--- contains checksum from v1

We already know that the contents of the /logs directory can be mutable, and with a forward-delta history in the inventory, we no longer need prior inventory files to reconstruct prior versions. The prior versions are now just basically backups.

[ahankinson]

Propose making inventory.jsonld and jsonld.sha512 as SHOULD store in the versions directory in the spec, rather than MUST. This will help with institutions migrating to OCFL as they do not need to change previously immutable objects to add version information.

[zimeon]

As I read through this discussion I'm left wondering how much we want a version to be able to live on its own. I agree with @ahankinson that having options for a strong notion of version-to-version linking and consistency is good. But perhaps this should be optional to allow clean distribution and sync of immutable versions that stand by themselves until reassembled.

[zimeon]

Discussed in https://github.com/OCFL/spec/wiki/2018.07.11-Editors-Meeting and agreed that there is no need to include previous digests in new inventory files. Closing in favor of #29