Content Versions, including history and draft

[CrispinF]

In SimpleContent we already have the ability to make a whole page a draft. But we don’t have the ability to make an edited version of that page a draft while the existing version remains live. And we don’t have the ability to review previous versions, and restore them.
And I think these requirements can be combined into one (in contrast with mojoPortal's approach where these were two separate things that could become confusing for editors). So "Content Versioning" would deliver management of past versions and a future draft version.
This would apply to content pages. Blog posts would only need the "history" part of this as draft content can already be handled adequately.
My initial ideas on this would be:

History:

I think another tab: Content | Settings | History
History tab lists of previous versions with date/time and editor, with ability to “Restore to editor” or “Compare to current” (plus delete and delete all).
I’d suggest the “Compare” might perhaps open the page in a full browser window rather than a modal, with difference highlighting. (The modal approach means layout is rarely faithful and so comparison is not easy – anyone doing this kind of work can manage browser tabs and windows). This might be achieved by opening a page with URL param identifying the version to open, with this only working for users in an editing role. But the modal approach would be ok too.

Draft content:

Regarding drafting once a page is already published, I’ve thought through a few ways and one nice simple implementation might be…

On creating a new page…

Hide the “Is Published” check box and “Publication Date” in this context and instead have these controls:
[Save Draft] [Publish Now] [Publish Later:] […date/time picker…] [Delete]
[Developer Tools]
“Save Draft” sets “Is Published” to False on the Page.
“Publish Now” sets “Is Published” to True on the Page and sets “Publication Date” to now.
“Publish Later” sets “Is Published” to True on the Page and sets “Publication Date” to specified date/time. If none specified, do as “Save Draft”.

On editing an existing page…

always edit the draft if one exists. Depending on how this is all stored, this could always be the most recent in the history. Don’t give the ability to edit a previous (published) version (it made things too confusing in mojo - editors can achieve that by restoring the published version into the editor).
Show the “Is Published” check box and “Publication Date” in this context, but call them “Page Is Published” and “Page Publication Date”.
Provide same set of buttons/controls:
[Save Draft] [Publish Now] [Publish Later:] […date/time picker…] [Delete]
(this allows a changeover in content to be scheduled).

There could well be some loopholes here!

I feel Content Versioning would be enabled globally. I cannot imagine why it would not be useful across the board, and it doesn't make the publishing workflow any harder. So perhaps just always there in SimpleContent - no need even to switch it on. The only reason not to would be the extra storage needed for the superseded versions, but that shouldn't be an issue these days, and there could be an Administrative capability to clean out history (All, or older than a specified period (e.g. 1 month, 6 months, 1 year).

Also useful in the Administration area would be a view of all Content History (descending date order by default, filterable by Editor or page), permitting View (but not restore).

Note this does not provide any differentiation in roles for editing and publishing – we think that’s rarely needed and makes things more complex, but a potential future enhancement.

[joeaudette]

I think this would be an excellent enhancement and appreciate the detail and thought towards user experience you've put in.

I would like to implement this however it is not a trivial amount of work and I'm not sure how soon I would get around to doing this on my own time. If we could work out a financial sponsorship to make this a priority then I would be able to make it happen near term.

I think the draft should go on the same table with the page content which ensures there is no possible way to have more than one draft at a time. A separate table would be used for History and content would only be pushed in there if content is changed, ie the page was published a new draft edited, just before publishing the new draft we push the existing version to history.

I think we need to clarify about "Editor". Currently we have an Author field that is editable but pre-populated with the user name on new content. Since SimpleContent is loosely coupled to cloudscribe core, it does not currently capture userid. And in a list of history for example there is not an easy way to join tables to cloudscribe core user table which lives in a different dbcontext. Probably we should introduce new string fields for CreatedBy, LastModifiedBy that could be populated with the user name and not editable but visible in the history list. And also we would need to add new Date fields for CreatedDate and LastModified date.

The Author field is editable and other fields are editable that are not part of the content. If all fields need to be versioned the design becomes more complicated and just having a draft field in the table is not going to work. It would probably require a separate table for drafts that mirrors the page table.

There is also a problem/challenge for scheduling publishing in the future. It is easy the way it is now, a page is published if the IsPublished is true and the PubDate is not in the future. But if we have a draft in a separate table publishing needs to mean it moves to the main table and history is archived. Ideally a task would do that as it would add overhead to check on every page request to see if there is a draft that needs to be promoted due to the draft publish date having passed.
For a new page publish later could mean move it to the main table with a future pubdate, but if it is a scheduled update then we need to keep the currently published one until the draft publish date and then archive the current one and promote the draft.

We've done things with Tasks using Hangfire, but I'm a bit reluctant to have a Hangfire dependency baked into SimpleContent. We could certainly have an integration library but I think it should be optional and some alternative way to do the changes for moving from draft to publish needs to exist and be the default way. Hangfire for example can't be used with NoDb, there exists a SQLite for hangfire but it is not recommended for production and I found it un-reliable in my testing task execution was being duplicated multiple times.

We could probably get away with using the same table for blog post history as for page history. I would prefer to focus on just pages for the first implementation of this but could plan for a source field in the history table to indicate page or blogpost. I agree that only supporting history is sufficient for blog posts, not drafts other than using the existing pubdate and not scheduled updates for existing posts.

Anyway these are my initial thoughts. I think this is fairly complex to implement.

[joeaudette]

I think whatever design is arrived at for content versioning and drafts needs to take into account #292 Content Templating, and probably that should be implemented first or both should be developed at the same time.

For example if we store a serialized custom model and now we want to make a draft for an updated version of the page then we need an additional field for the draft version of the serialized model so that the existing one remains intact until ready to publish the draft.

I'm still leaning towards the draft fields being in the same Pages table as the current published version, and that it would only be the content that is subject to publishing and drafts, other settings would not have draft versions, they would be updated on every save and shared between the draft and the current published version.

In the case where there is existing published content and a draft to be published in the future, then we need an additional field for DarftPubDate.
When the page is requested we would have everything on the page object from the same database hit, so we could check if the page has draft content and the draft pubdate has passed and if so automatically update the state of the page by creating the history record, pushing the draft fields into the main fields then clearing out the draft fields and draft pubdate so it is ready for a future new draft. That would eliminate the need for any background task and still ensure the new content is seen on the first request after draftpubdate.

[CrispinF]

Lots going on here, so just a few observations:

• Very strong need to see who was responsible for each version, even if this doesn't overwrite the published "author" field (but fine if it did too).
• Agree it would be undesirable to build in more dependencies to manage tasks for this (I don't think needed).
• Draft content (which I agree could also take in serialised model and even cached pre-rendered html version of that) in same table as live seems fine to me. Could in same manner have DraftAuthor too if desired.

Happy to discuss sponsorship for this - will be an excellent enhancement.

[joeaudette]

@CrispinF great! Once we have clarified the design a bit, I can produce a ballpark estimate of how much effort will be required, then I can contact you privately by email to discuss the finances. We can agree to an hourly rate with a cap, or a fixed bid. If we come to an agreement that is sufficient funding I can assign the issue to myself and begin work, otherwise we can optionally seek additional sponsors to contribute funding.

Are you also interested in sponsoring #292 ? at the same time or do you prefer to start with this one? I do think working on both of those at once would be more efficient.

I will draft a new post on this thread shortly to describe the implementation design plan and you can review that and provide feedback to help arrive at an agreed design plan to base my estimate on.

[joeaudette]

Implementation Design Plan

User Experience

This is largely based on your initial post.

On creating a new page…

Hide the “Is Published” check box and “Publication Date” in this context and instead have these controls:
[Save Draft] [Publish Now] [Publish Later:] […date/time picker…] [Delete]
[Developer Tools]

On editing an existing page…

Always edit the draft if one exists.
Don’t give the ability to edit a previous (published) version (it makes things too confusing - editors can achieve that by restoring the published version into the editor).
Show the “Is Published” check box and “Publication Date” in this context, but call them “Page Is Published” and “Page Publication Date”.
Provide same set of buttons/controls:
[Save Draft] [Publish Now] [Publish Later:] […date/time picker…] [Delete]
(this allows a changeover in content to be scheduled).

The only parts of an existing page that will be supported as draft are the main content and the Author Name. All other fields will be shared by the current published version and the draft. I think it would be rare to change the Author but it will be possible to do so.

It is not planned to support draft edits of existing Blog posts. We can capture history of edits, and possibly provide the ability to restore previous versions. I think this will be rarely used and would only be a safety net for user mistakes on editing.

Restoring Previous Versions

I think it will be a fairly rare event to need to restore previous versions. Therefore I would rather not load a list of previous version in a tab on the edit page as that list could get long if there are lots of small edits over time and that just adds overhead to loading the page. Instead I think we would just provide a link to "Previous Versions", and maybe it should open in a new tab so there is no risk of losing an in progress edit by clicking the link and navigating away. So there will be a separate controller action to view the list of previous versions. The list will have links to open a previous version in a new tab, and a link to restore the previous version to the editor.

The restore link will just link to the edit page and pass an extra parameter with the id of the history item to restore to the editor. The restoration then becomes a new draft. If there was an existing unpublished draft we should probably show a warning indicating that we just replaced an unpublished draft in the editor that will be lost if the save or publish buttons are clicked, and indicating the user should click cancel if this is not intended.

A Master List of Content History

A new page will be in the administration menu to view a paged list of all Content History (descending date order by default, filterable by Editor or page), permitting View (but not restore).

Miscellaneous

Content Versioning would be enabled globally, not something that can be disabled

Implementation Design Plan

New fields will be added to page model and table:

• DraftContent
• DraftAuthor
• DraftPubDate
• CreatedByUser
• LastModifiedByUser
• CreateUtc
• LastModifiedUtc

A new ContentHistory class and table will be introduced.
It will have all of the fields in the Page table plus:
HxId
ContentType : Page or BlogPost
IsDraftHx
ArchivedUtc
ArchivedBy - the person who made an edit that triggered pushing a copy to history before applying the edit.

There will be new query and command interfaces and implementations for CRUD operations on history data. It will be possible to delete individual history, all history, or history older than x days, much like we have in system log. Possibly a config setting to disable deleting history for those who don't want to allow that.

There will be a new interface IPublishDraftContent and an implementation of that to be injected into page controller. When a page is requested that will inspect the page to determine if it has a draft with a draft pubdate in the past. If so it will create history or the current version, the move the draft content to the main content fields and clear out the draft fields and save the page. This will ensure that the new version is seen on the first request after the draftpub date.

Estimate of Effort

Estimating is hard, I'm not good at it, things often take less or more time than expected, more often they take more time than I originally guess. Sometimes things go smoothly and progress rapidly, other times unanticipated challenges emerge.

I think this work would take at minimum 1 week of effort and maximum 2 weeks. Doing it at the same time as #292 would be optimal since the new model changes and data fields could be updated all at once.

[joeaudette]

@CrispinF question, what about history of draft edits, if a user is working on a draft edit for a published page should we save history on every edit and do we need separate ability to restore a previous draft to the editor?
The only down side is then we get more rows in history where the published content has not changed, which may be confusing. Would probably also require links to preview the previous draft separately form previewing the previous published version.

[CrispinF]

@joeaudette my gut feel is that saves on a draft should not be versioned themselves. But a question in return... how does "autosave" fit here? Is it really saving data into the database? and would this therefore count as a version?

[joeaudette]

@CrispinF the autosave is just a CKeditor plugin that saves to local browser storage to help prevent loss of work if connectivity is lost or the browser is accidentally closed. When the save button is clicked it clears the browser copy so it won't prompt to restore on the next edit session, whereas if you make some edits and wait whatever the autosave seconds is and close the browser and re-open to the edit page it will detect you have changes in the browser storage and prompt whether you want the local copy restored to the editor. So it doesn't save to the server and won't create content history records.

It is less work without implementing draft version history and not having to build functionality to view and restore previous drafts in addition to previous published versions.

In general I think restoring previous versions is going to be a rarely used feature but the times when it is used its going to save the day for someone who maybe worked on a big content and chopped out a section, and later regretted it. And that could happen for large articles where the draft may go through many edits. We could have an additional field in the history table to know if it was created by saving a draft edit or saving the previously published version when publishing the draft.

[CrispinF]

@joeaudette thanks for clarification re autosave.
I'm left unsure whether you are suggesting we should include versions of drafts or not. For me this would be a nice but not an essential , and since, if implemented, it would significantly increase numbers of versions and storage, it might even therefore be a site setting. Or how about when any draft is published, its earlier draft versions are removed, as by definition the admin has declared the content good.

[joeaudette]

@CrispinF I'm in favor of capturing every edit including draft edits. I think your idea to delete the draft edit history upon publishing the draft is the perfect solution. That will keep the history from being too cluttered and will have a side benefit that when looking at the master list of history it will be easy to spot content that has editing in progress.

Also having the indicator in the the history table will mean that the logic to restore and preview will be simple, if it is draft history we preview or restore the draft rather than the main content that is also in the same record.