[OJS] Versioning for published articles

[lilients]

Hi @asmecher, @NateWr, @bozana and everybody interested :)

I am currently finishing the work my colleague @felixgruender did to enable versioning of published articles and files in OJS. Because this is a big topic with many aspects to concider and a lot of code I separated the task into tree steps and wrote descriptions at my github wiki.

1. file versioning (display file versions at article page) description
2. article metadata versioning (allow user to create versions of the metadata) description
3. connection of article metadata versions with file versions. (work in progress)

There could be more steps in future. For example to make the versioning work with galley DOIs I am thinking about adding versioning for galleys (the file versioning would then be replaced). But this should have no effect on my ideas for the last step. And I would like to discuss these with you.

The original solution adds the versioning at the metadata tab and connects files with the article metadata version during the file upload. But the concept in OJS changed: there are now at least three places that needs to be changed to display version information (metadata and identifiers, schedule for publication, galley files) and there is no file metadata page anymore.

To simplify the GUI I would like to add a central overview of the existing versions and put all versioning actions at one place. This could be a grid that lists all versions of the article. Following information could be displayed per version:

• (optional) information about the version (change log, creator, etc.)
• article metadata (done), identifiers (todo) and publication (publication date, etc.) (done)
• galley files (todo: grid would need update to display only files of this version)

The original places of article metadata and identifiers, schedule for publication and galley files could display only the info of the current version. A link could lead to the versions grid to get information about old versions.

One big question is where to put the grid at the backend. I see the following options (neather one perfect):

1. at the top level beside metadata (does not really fit with metadata)
2. as a new workflow step after production (is versioning logically a new step?)
3. at the workflow step production above galleys (maybe to much info at the page)

Any other ideas?

Conclusion:

• Would a central overview of all article version be useful? - all versions of an article should be displayed together
• Where should it be placed? - - displayed at the production tab, see [OJS] Versioning for published articles #2072 (comment)

[NateWr]

Hi @lilients, thanks for posting about this.

I think you're on the right track, raising concerns about the way the GUI for this data is split up into three places. I think this causes some unique problems and I'd like to explore that a bit more to see if we can find a more robust way of guiding the user through this process.

My main concern is that we're asking the user to make a decision about versioning in a series of separate save operations. Go to metadata, make changes, decide whether they should be versioned and then click save. Go to galleys, make changes, decide whether they should be versioned and then click save. And so on.

I think this is an approach that's going to be prone to human error. And it will also inject additional cognitive overload to the initial publication process, which may be used for a minority of articles.

I'd like to see a UX approach in which the editor is automatically versioning by default, and has to go out of their way not to version changes. Here's my first take on what that could look like.

1. The journal opts into versioning under Settings > Distribution > Versioning. Once they do so, the following workflow steps occur by default.

2. An editor schedules a publication normally. When that issue is published, OJS automatically creates a version of that article.

3. From the date of publication, a new alert message appears at the top of the submission workflow advising the editor that they are editing a new version of the article. If they wish to make changes to a previous version of the article, they must go to a separate UI.

4. Any changes the editor makes after publication are saved to a new version automatically. These changes are not reflected on the frontend until the editor publishes an updated version.

5. To publish an updated version, the editor clicks a button under the Publication stage, "Publish New Version". This launches a UI which shows them what has changed and advises them they will be publishing a new public version. They click Publish New Version to confirm their decision.

If an editor wishes to modify a published version without actually creating a new version, they must click on a new button in the Publication stage: "Edit Published Versions". This will launch the UI you've sketched out @lilients.

The editor can click on a prior version to load that version in the Submissions workflow. When this version is being edited, a large banner notification will appear at the top which informs the editor they are editing a previously published version and that any changes they make will be displayed immediately.

Does this sound like an approach that isn't a nightmare to handle on the backend? Is the workflow clear or would anyone benefit from mocked-up screenshots?

---

To address some of your other specifics:

URLs

I like the URL approach. I think the "current" version should also be available at .../article/version/[articleId]/[version] so that people who want can make permanent links to a version. And canonical link tags should be used to point to the current version at all times.

Article Page

I'd like to see a small note beside the Published date, like:

Published
Jan 21, 2017
This article has been revised. View all versions.

That could be just an anchor link that jumped a section below as you outlined, but showing the date rather than the title:

I think only the currently-viewed versions' galleys should be displayed at any one time.

Finally, when any version other than the current version is being viewed, a banner notification should indicate that an out-of-date version is being viewed. Something like "This article has been updated. View the latest version".

---

Hope this is helpful and not wildly out-of-scope. I know there's been some discussion about revisions within the team, and it's a really tough issue.

[mtub]

I'd like to see a UX approach in which the editor is automatically versioning by default, and has to go out of their way not to version changes.

I support this and all of your specific comments at the end. This is how I would imagine versioning of metadata and galleys to look like, both from a reader's and from an editor's point of view.

[lilients]

Hi @NateWr,

thank you for your reply and your concrete feedback. I also think creating a new version by default is a good idea. But I found it hard to explain to my colleges and I fear it is confusing for users who only want to have a look at the current state and do not want to create a new version.

I understand your sketched workflow and think it is a very good approach - I especially like the idea of being able to edit a version and making it available later. This simplifies the connection of metadata and files a lot.

If an editor wishes to modify a published version without actually creating a new version, they must click on a new button in the Publication stage: "Edit Published Versions". This will launch the UI you've sketched out @lilients.
The editor can click on a prior version to load that version in the Submissions workflow.

I fear that users will not understand this - different information at the same place (production stage with information about different versions). I now also see a similar problem with the grid I sketched. The same information will be displayed at different places (e.g. galleys at the production tab vs. galleys in the grid and metadata at the top level and metadata at the grid). And there is always the metadata tab that is kind of detatched from the version info.

The solution I see now would be similar to the review rounds at the workflow stage "review". Each version would be displayed at its own tab. Files and metadata can be accessed and changed at one place. The metadata link at the top level could be hidden or it could display the metadata of the current version with a link to the old ones. The discussion could be used for the whole stage as in the review stage.

Currently only the recent version can be edited. Do you think that is ok, or would the users want to change all versions?

Thank you for your feedback on the URL structure as well. This might get tricky once there will be files connected with the article version. But we can discuss that later on. :)

I'd like to see a small note beside the Published date

Thank you very much for this idea. I think it fits much better.

I think only the currently-viewed versions' galleys should be displayed at any one time.

Yes, the file versioning might be a lost cause. ;)

Hope this is helpful and not wildly out-of-scope. I know there's been some discussion about revisions within the team, and it's a really tough issue.

Yes, thank you very much for your thoughts and ideas! It helped a lot. We did do a lot of discussion. But I think it is needed and is getting better every time. :)

Conclusion

• Would a tab view similar to review round be a solution to display versions at the production stage?
• Should we allow to edit all old versions or only the recent one?

[mtub]

Hii @lilients, I fear that your approach will lead to the wrong behaviour - if I can change files and metadata (of the most recent version) easily but have to press a button to create a "new version", people might just update their most recent version of that article instead of creating a new version. But in most cases creating a new version is the correct approach - this is why adding versioning to OJS/OMP is so extremely important. The software should make it as easy as possible to do the right thing. Changing already published articles is not an option that should be encouraged by the software.

We should not fear confronting users with truths like these. Publishing is not a game, and journal editors need to know what they are doing. So I'd rather see an approach that redirects efforts to edit files and/or metadata of published versions to the "new version" option. It should be hard, not easy, to change something after publication date. (@asmecher, @stranack : this will also need good documentation).

(Maybe it would be a good idea to also introduce a standardized "corrections" feature for minor corrections that change something in the article without creating a new version, like a correction notice for an image or reference that someone forgot to include during production.)

[asmecher]

Maybe we could support a MediaWiki-style "this is a minor revision" distinction that would lead to the revision being tracked, but receiving lesser consideration? I'm thinking about e.g. whitespace removals, typo fixes, etc. that are perhaps important to track but definitely shouldn't receive full "revision" status.

[mtub]

Hi Alec, but most journals adhering to a more formal publication ethics will consider these changes not as "minor" - you will want to avoid that two people linking to the same article (version), using the same DOI etc., do not talk about the same works. This is why I think versioning is so important, so that journals can update/correct articles without acting against publication standards. But the most important thing is to educate users so that errors are dealt with before publishing the first version. We might not always be happy with the way things are at the moment, but there is a difference between scholarly publishing and less formal ways of publishing on the web. And frankly, I definitively want to be able to link to stable versions of articles that are not edited after publication date.

Adding something like "Corrections" might be a good compromise for minor editing/minor versions. But even then we would need something standardized for displaying the corrections. This would IMHO not include changes to metadata or galleys, the corrections would be something like an add-on to the published galleys.

[asmecher]

How about if we were to track all revisions (including minor ones), thus linking would be stable, and expose all revisions of published content -- but allow for expand/collapse of changes that are flagged "minor"?

[lilients]

Hi everybody,

I would not differentiale between different types of corrections. I concur with @mtub that every change should be a new version. Even a typo could be quoted and would lead to wrong quotes if corrected.

I think we could add the default feature to my last proposal. After the publication of a version a new tab will be created by default, so that users will edit a new version. To view/edit old versions they would need to click on the previous tabs.

[asmecher]

@stranack, your input on this would also be valuable!

[NateWr]

Great discussion. It looks like we've got two things which feel technically similar but may have different conceptual bases or use-cases. I'll try to outline those to see if we can spark some further thinking along these lines.

1. Journals want to make a significant revision to an article, one which should be prominently displayed to end users and may include substantial changes to key article content and metadata. (Thinking about OMP, this seems even more conventional for monograph editions.)

2. Journals want to make minor revisions to an article, which might include typographical errors or updates to metadata (like keywords). These are consequential and should be tracked. But journals may not wish to make a big public notice about them.

The workflow I sketched out was definitely directed at the first use-case. The idea was that the second use-case would be filled by editing versions, and these could or could not be tracked.

I wonder if it makes sense to consider different technical solutions for each of these cases. To me, the first use-case sounds like we want a separate copy of the article in the database. Something so that we can present different article representations on the frontend and in the workflow. But the second use-case sounds like we just want to diff changes.

Imagine how amazing it would be if we had a full textual representation of an article version, and we could just diff it for minor changes? We could then store an audit log of minor changes with minimal impact on the database, and we'd be able to use existing libraries for working with diffs.

Of course, we're not yet at full textual representation. But how close is our XML exporter to being able to produce something resembling an XML representation of an article version that could be diffed?

That wouldn't pick up differences between article galley files. But it could register new file paths, thus recording the fact that a change occurred. If this were combined with preserving the former files and preventing overwrites, it could be an effective audit log.

[lilients]

Thank you @NateWr. Yes, I think the two use cases make sense.

1. The article metadata versioning should be a solution for the first use case. Metadata like the title, authors, abstract, etc. will be stored for each version. Each article metadata version and each attached file should also have a different DOI (I will implement that soon). (I am currently working on the connection of the metadata version with the file. The sketched workflow should solve this problem.)

2. Minor changes could be allowed at the recent version and would mean the editor overwrites data without creating a version - without creating new DOIs. This is very problematic and there should be warnings. But I think it should be allowed for (the recently) published version/s.

I like the idea of displaying diff changes and marking significant revisions of an article but I think those are additional features to the use cases you described. Unfortunatly my timeframe is limited and I need to finish the versioning as good as possible as quickly as possible. But I think we could add both features later on:

• A first step to the diff display might be the display of the file history I already implemented (A file versioning is already implemented in OJS - all uploaded files and versions are stored at the server. The addition simply displays previous versions of files at the article page). Later on a new feature could compare those file versions and display diffs. I think the file history could be displayed in addition to the article versioning - Maybe @NateWr has some ideas how.

• The article metadata versions could be added by tags like "major" or "minor". This way OJS could handle the versions differently. For example major versions could be displayed at the home page of the journal. But I think both should get their own version of article metadata - and especially DOIs. So this would be an addition to the use case 1 and not another implementation.

I will try to describe the planned concept of versioning in OJS and store the decisions of this discussion here:
https://github.com/lilients/ojs/wiki/Versioning-in-OJS

For now I would concentrate on getting the first use case finished. I need to display those article metadata versions at the backend and connect them with the file/galley versions. Any feedback on the tab view of versions at the production stage? :)

[NateWr]

I totally get that you've got a deadline on this. We'll probably have to go through a slower development process to make sure we've got things right before rolling it out to everyone. Are you building this as a plugin for now?

Any feedback on the tab view of versions at the production stage? :)

I'm a little wary of the sub-tab approach, mostly because of the problem whereby title and metadata live outside of these tabs. But I think this is the result of a more general problem we have where we don't really have a clear presentation of a published object.

When published, an article just kind of sits in its workflow. And what constitutes the published object is pulled together from various things scattered around the page (metadata, galleys, participants, issue assignment).

I think that a UI for versioning would probably become a lot clearer to us if we had a view of a published object. We've talked about this as a final publishing step before: presenting a kind of confirmation view of all article core data, metadata, galleys, authors, DOI, and issue assignment with a big button that says, "yes, publish all of this I'm looking at right now".

This is probably out-of-scope for what you need to do within the time you've got. But it might be something for us to consider as a target for versioning down the road.

In the meantime, I don't really have a better idea than the tabs you present. No matter where the versioning actions are placed, they're going to be alienated from some of the customary tools for managing them.

[lilients]

I think it is a very good approach to think things through before implementing. ;) And I appreciate your feedback a lot!

Felix started the versioning at the core - not as a plugin. You can have a look at the code here: https://github.com/lilients/ojs (most relevant for the article metadata versioning is this commit lilients/ojs@dd07570). I am always rebasing to prevent divergence. I hope to get to a point soon where I can create a pull request.

I came to the same conclusion like you: There is no ideal way of presenting versions because of the scattered elements of a published article. But I hope we can find a solution that is usable and can easily be ported to future concepts.

The idea of presenting a published article with all published data in one place sounds interesting. Would that be part of the workflow or outside?

I think the versioning should be part of the workflow. It might be useful to be able to use some workflow elements - like storage of files and discussion - for new versions.

I tried to put all metadata belonging to a version inside the version tab. So this would be a step into the direction of presenting all data of published articles in one place. And maybe it could be ported to a future concept...

Another way to implement versioning could be a new workflow step after production. But the problem with the metadata at the top level would remain and there would also be doublings with the galleys.

So for now I would go with the tab solution inside the production stage.

[lilients]

Hi @NateWr and everybody interested,

I implemented the tab view at the workflow tab production. It looks like this:

Now I have to add the functionality to create a new article version. I see different approaches at the GUI and would like your advice.

Open question 1: create new version by default or with a button?

Option 1: create new version by default when scheduled for publication

This is what you see before scheduling for publication:

Afterwards there will be a new unpublished version.

Might be confusing for users - especially if they do not want to create a second version and there will never be a second version, then there will always be two tabs - one unused.

Option 2: button "new version"

This way only the current version is displayed. To create a new version users would need to click the button "new version".

Open question 2: allow editing old versions or only the current one or none at all?

Option 1: Allow changes at published versions but add an alert note when user wants to do so.

Option 2: Disable changes when a version has been published. This way users would be forced to create a new version instead of editing a published one.

I would prefer the button solution with no option of changing published versions. I think that would be less confusing and still preventing the user from changing already published stuff.

What do you think? Do you see other options?

Looking forward to your feedback and ideas.

Best,
Svantje

[mtub]

Hi Svantje,

if published versions cannot be edited (Question 2 - Option 2), the button solution for creating new versions (Question 1 - Option 2) is fine, I think.

[lilients]

Hi @asmecher, @bozana, @NateWr , @mtub and everybody interested,

as I mentioned before I think we need to add a versioning for galleys as well. Mainly because there are DOIs attached to galleys and statistics uses them. I would like to check my approach with you before implementing.

Currently there are versions of submission metadata stored in submission_settings like this:
submission_id, submission_revision, .... Functions that access this table has been adapted as well as export and import functions. On the todo list are DOIs at article level.

Currently files are connected with galleys (1-1) and do have DOIs assigned to them. The galleys are now attached to submission versions. As of now for each submission version a new galley is being created automatically. But this means that there is no connection between the galleys. This enables a different DOIs for each version but seem to be a problem for the statistics. So I think it would be convenient to add a versioning for galleys simultaneously to the verisoning at submission level. This would look like this:

• The table galleys and galley_settings would be added with "galley_revision" and the primary key changed to "galley_id" + "galley_revision"
• DOIs and publisher ids would be stored for galley versions - maybe the galley version could be added as variable to the DOI syntax
• Statistics would cumulate the access to galley versions based on the galley id
• Export and import needs to be updated to cope with galley versions
• Files would be automatically copied into the new galley version and are getting a new file id (so no connection between "file version" here - this means the file versioning wouldn't be useful at the article page anymore, but maybe we do not need it ... if we do, it could be reimplemented at galley level in future)

Is this ok with you? Do you see problems or aspects I missed?

Thanks for your thoughts!

Best,
Svantje

[lilients]

Hi everybody,

I refined the database structure simultaneous to Felix versioning of submissions (storing info that needs to be versioned in settings):

submission_galleys (all information that will not be versioned):

• galley_id, submission_id, locale, label, seq, is_approved (leftover from OMP?) - moved: file_id, remote_url

submission_galley_settings (information that will be versioned):

• galley_id, version (new), locale (seems to be double - do we need it?), setting_name (doi, publisher_id, new: remote_url), setting_value, setting_type

submission_galley_files (new table to connect galley versions with files, does not fit in submission_galley_settings and cannot stay in submission_galleys because needs to be versioned):

• galley_id, version, file_id

The version number of the galley is the same as submission_revision. This means there is no real galley versioning but the galleys are connected to an article version. This way the URL structure can be simplified and looks like this:

• current article: article/view/[articleId]
• current galley: article/view/[articleId]/[galleyId]
• old article: article/version/view/[articleId]/[version]
• old galley: article/version/view/[articleId]/[version]/[galleyId]

• Would users want to change labels of galleys between versions?

Feedback and questions are welcome.

Best,
Svantje

[NateWr]

HI @lilients,

Sorry for the lack of feedback on my part. Your screenshot looks good, with the tabbed versions. I agree with @mtub that using a tab named "New Version" is better than just naming a new version. A couple thoughts:

• On all version pages except the first, can we change "Schedule for Publication" to "Publish Updated Version" or something like that?
• We have two areas where "Metadata" exists: above the submission title and in the version tab. Is there anything in the metadata panel linked above the submission title that is not in the metadata panel linked in the version tabs? If not, then I would suggest we remove the Metadata link above the submission title and just keep it under Production. That may be confusing for users at first, but I think it will reduce confusion in the long-term by keeping it under the Production sphere of interest.
• How hard would it be to make the versioning UI an opt-in checkbox under settings? I know the team is keen to get versioning merged into core as soon as possible. I'm a little bit nervous about exposing the additional complexity to all users. An opt-in setting would allow us to ship this soon while giving us some more time to tinker with the UI before enabling it for everyone. cc @asmecher on this, in case you think that's a terrible idea.

[lilients]

Hi @NateWr ,

thanks for your feedback. Opt-in is already implemented.
I think I can rename the button "Schedule for Publication" if versioning is enabled.
Currently the upper metadata window displays the metadata of the most recent version. The metadata of old versions can be viewed at the production tab. Moving the metadata completly to the production stage is a big decision I guess. I think we could change that later on.

@ everybody Any feedback on the database stuff? :)

Best,
Svantje

[lilients]

Hi @asmecher , @NateWr , @bozana and everybody else,

I added the galley versioning as descibed above and made some GUI changes (Description). Works fine. :-D

Cleaning up I found another problem. For the article versioning the date_published has been moved from published_submissions to submission_settings. But there are some functions that use the date in published_submissions for ORDER BY, MAX and MIN . I see these options to solve this problem:

1. remove ORDER BY and MAX and MIN from SQL and implement it only if necesarry in php with the latest/first date from submission_settings
2. keep date_published in published_submissions and update it with date of the first version or the date of the latest version

I would try to go with option 1 because I think it would be cleaner. Otherwise the publication date would be double in the database. Is it really necessary to order the published article by date? Would it be sufficient to order only by sequence?

What do you prefer? Do you see other options?

Best,
Svantje

[bozana]

I think that for such kind or ordering we could consider the date_published of the first article version -- if I see it correctly those are functions like "get all published articles of a journal" -- I think that for these cases the date when the article is first published is 'asked'/'important' and not of the later article versions i.e. changes. Thus I would try to consider that in the SQL if possible i.e. to take and order by the setting_value from submission_settings where submission_id = ?, version = 1 and setting_name = 'date_published'. Would that be possible?
The sequence attribute/column is relative to the section and issue, which is then not relevant if we want to get all published articles for a journal, for example.
Also, the published_submission_id could not be chronological (e.g. if a journal imports back/older issues after it has published some newer issues), so we cannot rely on that either.

[NateWr]

I added the galley versioning as descibed above and made some GUI changes (Desciption). Works fine. :-D

The frontend display of versions looks good. What about adding the version name to the date? I'm thinking about cases where someone might release two versions on the same day. It would be good to have:

Version 3 — 2016-02-03
Version 2 — 2016-02-03
Version 1 — 2016-01-02

[lilients]

Thanks for your brainpower!
@bozana: brilliant idea! Seams to work as well. 💃
@NateWr: also a very good idea. :) I will add the version number to the date.

[lilients]

Done that lilients/ojs@42b5fce and lilients@ee16e4e ... next topic to be discussed: URL structure.

We already discussed the principles via mail:

• URLs shouldn't be ambiguous (i.e. we shouldn't have to guess at whether a URL part is a revision number, custom file ID, etc)
• Old URLs should continue to work
• If possible, we should avoid introducing more separators etc. into URL parts

I would propose (and already implemented ;)) this structure:

• current article: article/view/[articleId]
• current galley: article/view/[articleId]/[galleyId]
• old article: article/version/[articleId]/[version]
• old galley: article/version/[articleId]/[version]/[galleyId]

This way the view argument gets replaced by version for old versions (to prevent ambiguous structures) and in the ArticleHandler the function version() redirects to the function view(). Do you think we could live with that? Or is the view argument crucial?

[NateWr]

I like having the version in the URL. Is the current version always available at /article/version/articleId]/[current-version-number] as well? I think that would be good, though we should add a canonical tag to that page that points to the article/view/[articleId] location.

The benefit of having that is that, if I want, I can generate a link to the current version of an article that I can trust will always point to that version, even if it's later updated.

[bozana]

I would agree with Nate... Maybe it would be good to always have the version in the URL: if a user reads an article and then cites it with the URL article/view/[articleId] the URL could be wrong a few month later, when the new article version appears under that URL i.e. the URL of that article changes. Also for DOIs/URNs -- the URL should not change with the version change, I think...

[lilients]

Yes, the current version is available at the "default" URL and also at the URL with the version number.
I also think that the DOIs/URNs should be registered with the URL with the version number, so they do not need to be changed later. The "default" URL is the same as before in OJS (I think this is important for backward compatibility) and will always lead to the most recent version - this is the same concept as wikipedia uses. If you want to cite a specific version you would need to use the URL with the version number. We need to think of a good place to display this URL. Currently the "version history" is only shown, when there are more than one version. Maybe it would be enough to change the URL in "How to cite" to the version URL?

[NateWr]

No, I think it's good the way you've got it @lilients. If someone really wants to surface the version URL before there are multiple versions, they can do that themselves.

👍

[NateWr]

Work on this is now being tracked in the versioning project.