Editor Show and Tell #1

[brittnylapierre]

How to use GitHub Discussions

If you're new to GitHub an GitHub discussions, please check out the following guide to help you get started.

GitHub Discussions Guide - Video Version
GitHub Discussions Guide - Text Version

Feature Description

The editor is the tool that you will you to change properties of a manifest or a collection. You're also able to create and edit new collections and manifests from the admin app.

Feature Guide

Please read this guide.
and/or
TODO - Please watch the following video. Please adjust the playback quality as needed.

Try it out yourself

Go to the admin app.

Then, follow the guide to create a manifest and a collection.

Use the manifest you've created to add canvases and re-order them. Try removing some canvases, too.

Use the collection you've created to add members and re-order them. Try removing some members, too.

Try to find your manifest or collection by searching for it from the home page.

Edit the slug, label, etc. of your collection and manifest.

Publish your manifest and collection.

When you are done the above, unpublish and delete your manifest and collection.

Now go back to the home page, and search for a manifest and collection. Go to the editor for that manifest and collection, and make changes, then revert them.

When you have collected some thoughts about the editor, please fill out the form below.

Feedback Form

To help gather feedback, we're using a format inspired by a feedback grid activity.

(Copy paste the following into a comment, then fill out the prompts to give us your feedback)

What worked well?
I liked...

Do you have any questions?
What does... ?

Did you experience any blockers?
The tool had a bug...

Are there any changes you'd like us to make?
I think this would be easier if...

Have any shiny new ideas?
It would be cool if...

[BeS4]

Hi Brittny,

Thanks so much for creating the guide and to everyone involved in creating the Editor tool. I admit that I have not actually tried to do anything that would be saved. At this point, I have just looked at the guide and followed along with the exercises on the Editor. But I did not make any changes that would be saved. I will do that when I have something to change (and a bit more confidence) and then will provide additional feedback on those experiences.

So far, here are my comments ...

What worked well?
I really liked the quide and the simplicity and user friendliness of the Editor. Well done!

Do you have any questions?

• What does "Behaviour" and its two options (multi-part or unordered) mean in the Editing Collections function? It would be really helpful to me if these were explained a bit in the guide (see pg. 26)
• I'm unsure as to when we would begin using these tools? And how to go about the actual switchover and the implications of that with the use of our current ingest tools. For e.g. one thing I'm thinking about it the ingest of newspaper issues and how to ensure they go into the right collection. Currently, I think the configurations set up in the ingest tools as well as the issueinfo metadata helps guide putting the issue into the right newspaper collection. So as long as we are doing that, I don't really need to use the collection tools to do that sort of thing. I'm trying to envision when I would start to use the Editor to do these things. (Sorry if this sounds confusing. I really need to think about this more.)

Did you experience any blockers?
Sometimes the screen shots were difficult to read. But perhaps the actual detailed text is not that important to read. As well, likely there is nothing that can be done to improve readability, but thought I'd mention it just in case there is.

Are there any changes you'd like us to make?
For moving and removing the canvasses, I find it a little scary that the changes seem to be saved automatically. I fear that I might start moving pages around and get all mixed up, and decide that it's better to start from scratch again. It might take a long time to get the pages back to their original order. Would it be possible to put "save" or "cancel" buttons here? This approach might also be helpful in the "Manage Members" section.

Have any shiny new ideas?
It would be cool if you could create a Table of Contents for the guide.

[RussellMcOrmond]

I can answer some of the non-UI related questions.

• The concept of a "series record" has become an ordered collection (AKA: "multi-part"). Nothing relating to a "series record" will be part of the preservation platform in the future. Whether something is a series or a monograph are access concepts, and shouldn't really be part of the preservation system (Monographs vs series vs issues don't exist as concepts in Archivematica).

• As far as preservation data is concerned, everything becomes a stand-alone group of preserved images that can be imported into access to become a manifest (a group of canvases, which have images in them). A manifest becomes an issue of a series when that manifest is added to an ordered collection in a specific position.

• While the new Access platform will still support Issuinfo records until fully transitioned, the "series" and "sequence" fields no longer have any meaning. A manifest becomes a member of a "series" at a specific "sequence" when it is added to an ordered collection (the series) at a specific position (the replacement of the concept of "sequence").

• The metadata team will be able to move forward with consolidating descriptive metadata formats if they wish. MARC could be used for everything (a monograph, an issue of a series, ordered "multi-part" collections, or unordered collections), with no need to have multiple descriptive formats that are specific to any of these. How collections and manifests are connected to each other is no longer tied to descriptive metadata in any way.

• Re-creating a manifest is as easy as running the "Import from Preservation" (AKA: Smelter) again. If you get the canvases all wrong, then just delete the manifest and re-create it. Please do get it wrong, and then delete your mistakes, as this is something everyone who is involved in managing data in the access platform will need to know how to manage. What I have been doing with my testing is prepending "rwm" in front of things (IE: instead of smelting using the slug of "oocihm.51224", I would use "rwmoocihm.51224" ). Remember that these are slugs, which are just a string of characters for a transparent identifier: that part in front of the period is not a "depositor", as the access platform doesn't have the concept of a depositor. It is just a transparent identifier which we have called a "slug" that happens to have a period in it. You could just as easily call it "Be51224". Slugs must be unique, but there is nothing saying that collections must not have a period and manifests must have a period.

[brittnylapierre]

@BeS4 Hey Beth! Thank you for your feedback!

I added in the table of contents and the description of the 'Behaviour' field to the collection properties section. (Page 6)

I will try to do these docs in portrait mode so I can make the screenshots larger. I could also zoom into specific parts of the screenshots that require attention. For the most part the text isn't supposed to be too important other than when I say you'll see a message that looks like... but it's good to make sure it's all readable in general. Thanks!

I'll speak with the other developers for the suggestion of the save button for the members/canvas lists, too!

[RussellMcOrmond]

I'll speak with the other developers for the suggestion of the save button for the members/canvas lists, too!

That concept was removed from this version of the tool as part of the paging work. In order to facilitate potentially massively long lists for collections and manifests, having everything in memory at once (and requested and saved as single requests) is not possible.

Since correcting an problem is easy enough to do, such a feature would be outside the scope of the current project. Remember: We are only replacing existing functionality (activities that required SIP updates, or things related to collections as that was also transitioned), and not trying to be perfect.

[RussellMcOrmond]

@brittnylapierre ,

A suggestion for the manual is to move the "Create New Object" to the top. Then staff learning to use the tool will be encouraged to create new objects which they then play around with and later delete, rather than thinking they need to start with objects which were already published.

Unpublished objects is the sandbox which staff can use for any experimentation if they are worried about the public seeing their edits.

[brittnylapierre]

Thank you @RussellMcOrmond that's a great idea! I moved the create manifest/create collection to before editing manifests/collections.

[brittnylapierre]

@DivyaKrishnan26 FYI in case the logical ordering affects the video!

[RussellMcOrmond]

Series Records and Issues of series

As an enhancement to the guide, there should probably be a section that makes very clear what replaces a series record, and how issue metadata will be different.

Currently, a series record is a AIP that is ingested which has no images, only metadata. The METS record has TYPE="series". That metadata must be DC or MARC.

With the new version of the access platform, no series records should be ingested. Instead, a collection is created with "Behaviour: multi-part". The terms used for behaviour come from IIIF, and we only support 2 currently to support the two types of collections the earlier version of the access platform supported.

Once created, the collection should have descriptive metadata loaded. While the earlier access platform was specific about what types of metadata could be used, the new version allows any supported descriptive metadata to be used for any type of access document.

Currently, an issue of a series is an AIP with images that MUST use the Issueinfo descriptive metadata type, and has METS type="issue". The Issueinfo format was an interesting combination of descriptive metadata as well as two additional fields: "series" and "sequence". These fields were not descriptive, but detailed relationships between different issues in a series (the order) as well as what parent series record to reference.

These non-descriptive fields are now ignored by the new version of the platform. An manifest is an issue of a series because that manifest has been added to a multi-part collection. Its "sequence" is determined by the order that the collections appear within the collection.

In the new platform, a manifest that is to be treated as an issue of a series can use any of the supported metadata types (Currently DC, MARC, Issueinfo), and is no longer tied to Issueinfo. In fact, as a future project of the metadata team, @nataliemacdonald will hopefully decide to retire Issueinfo entirely.

Smelter processes METS TYPE="document" and TYPE="issue" the same (doesn't differentiate between those types), and rejects all other types (IE: TYPE="series" can't be smelted). Archivematica METS records don't make any of these types of distinctions.

[RussellMcOrmond]

Large lists

The logic of the editor needed to change to allow the handling of collections/manifests with a large number of members/canvases. Part of the feedback needs to be from working with some of these larger lists.

Heritage is a good place to look, as many of the manifests have thousands of images, and those issues are members of series ("multi-part" collections) which also have many members where the order is important.

Large Manifests

The following CouchView will show some of the largest manifests, those with more than 8000 images. (Note: Log into https://admin.canadiana.ca/ first, as CouchView doesn't send you to the right view when redirected back from authentication).

https://admin.canadiana.ca/couchview/access/tree/canvases?endkey=9999999&group=1&keytype=number&range=true&reduce=true&startkey=8000

Count	Slug
8021	"oocihm.lac_reel_c7071"
8060	"oocihm.lac_reel_c7081"
8086	"oocihm.lac_reel_c9812"

Large multi-part collections

Multi-part collections (such as what we previously called series records) are collections where staff will want to manage the order, and not simply add and remove members. This is where the interface for large number of members will be important.

The following is a view of multi-part collections that have more than 1000 members (See note above -- log in first, otherwise this link may lock up your browser).

https://admin.canadiana.ca/couchview/access/tree/members?endkey=%5B%22multi-part%22%2C%7B%7D%5D&group=2&keytype=json&range=true&reduce=true&startkey=%5B%22multi-part%22%2C1000%5D

Count	Slug
1013	"oocihm.8_04155"
2012	"oocihm.8_06569"
12317	"oocihm.N_00367"

Large unordered collections

To be complete, here is the view for large unordered collections. These won't be a problem to manage as there is an alternative. Staff can use the collections interface to add members to the collection (where doesn't matter - they are unordered). To remove a member from a collection, instead go to the member and look at the "...is a member of the following collections:" section, which allows you to remove there. You don't ever have to search the large number of members of an unordered collection looking for a specific one to remove.

https://admin.canadiana.ca/couchview/access/tree/members?endkey=%5B%22unordered%22%2C%7B%7D%5D&group=2&keytype=json&range=true&reduce=true&startkey=%5B%22unordered%22%2C1000%5D

Fixing labels.

When staff looks at the collections an access object is a member of, and the label is wrong (or "undefined"), it is probably good to get into the habit of clicking on that collection and fixing its label.

There are only 30 unordered collections at the moment, and these are the ones most likely to not have a label set.

https://admin.canadiana.ca/couchview/access/tree/members?endkey=%5B%22unordered%22%5D&inclusive=true&keytype=json&range=true&reduce=false&startkey=%5B%22unordered%22%5D

[nataliemacdonald]

I have been using the editor tool for the past couple of days and will need to use it quite a bit over the next few weeks as the metadata team tries to fix errors in identifiers and delete duplicate items in the access platform.

What worked well?
The first couple of changes I made worked perfectly and it was so great to have the documentation on the page to use as instruction.

Do you have any questions?
No questions right now.

Did you experience any blockers?
Yesterday when I tried to perform two changes they were still processing hours later. I just checked this morning and both say that they were successful, but I'm not seeing the changes on the access side. I'll outline both of the situations below.

Are there any changes you'd like us to make?
I really like the tool and the UI

Have any shiny new ideas?
Nothing right now.

---

1. I tried to remove the "b" from the end of this slug - https://www-demo.canadiana.ca/view/oocihm.24662b/1. The processing page says it was successful - https://access.canadiana.ca/object/edit/69429/m0pv6b27q092. But, it was not. The URL with the "b" should no longer work, but it does and the one without the "b" does not work, even though it should. When I searched for the slugs in the editor tool it only brings up 24662.
2. I tried to unpublish this item - https://www-demo.canadiana.ca/view/oocihm.00387b/1. It says it was successful, but I can still access the item.

Thanks for all your help!

[RussellMcOrmond]

Did you do a large batch of metadata updates earlier yesterday as well?

Changes made to the Access platform documents are processed to generate the cache documents used by the portals in the order that they are put in. At the moment there are still 54888 ID's being processed by "press" (the second stage after something called "Hammer", with the next stage being the streaming to the Solr search engine).

The "Update Finished: 2022-02-16T09:50:45Z" you see in the web interface is the finishing of the first state (Hammer), but doesn't indicate the other two stages.

We have some ideas about upgrades to this process in the future (separating Search and Presentation, merging the "press" stage into the "Hammer" stage), but it isn't yet a current project.

[nataliemacdonald]

I did not do a large batch, but yesterday Brittny mentioned there was a backlog of processes. I'm not sure what the backlog is, but I don't think it's from me. I tried to do a large batch last week, but it failed eventually I canceled the rest of it.

[RussellMcOrmond]

@brittnylapierre ,

Our CouchDB cosearch and copresentation documents don't have dates, but we (Well, William 10+ years ago) configured our Solr search core to have a date. If knowing the date of the last change to the primary slug document is needed to be added to the web interface, that is possible. That interface will need to change as we change search cores in the future.

https://github.com/crkn-rcdr/solr is the current project.

https://github.com/crkn-rcdr/solr/blob/8b05e2348061d81146bb15a28bc9c0a5404711f6/cosearch/conf/schema.xml#L50-L51 is the relevant field, updated whenever the document is updated.

[nataliemacdonald]

This morning I noticed my updates from two days ago have gone through, which is great! There were:

• unpublish and delete 00876b
• Edit slug of 24662b to 24662
• Edit slug of 00297b to 00297

Then this morning I started performing more changes.

• Unpublish and delete 35271 - Worked perfectly and within a couple of minutes
• Edit slug 35271x to 35271 - https://www-demo.canadiana.ca/view/oocihm.35271**Processing**
• Edit metadata - https://www-demo.canadiana.ca/view/oocihm.88004/1 Processing

But, now when I continue to try to make more changes every single item I search in the "Edit in Access" tool says "Currently Processing", even though I have not done any other changes this morning than what I have written above. I checked 10 different CIHM numbers and they all say a currently processing time of "2022-02-17T13:04:55Z".

[RussellMcOrmond]

@brittnylapierre and I will look more closely into this. I have a guess of what might be happening that we'll investigate.