Editor Review - Initial Scoping

[ssddanbrown]

We need to upgrade and/or replace the existing WYSIWYG editor. We can use this as an opportunity to find something that will better suit BookStack's users requirements now that we have almost 6 years of experience and feedback on the matter. There are currently two editors within BookStack that need to be considered:

• The WYSIWYG Editor (TinyMCE4)
• The Markdown Editor (Codemirror 5 with custom preview pane).

Main Goals

Here are some high-level goals it would be great to achieve with a new editor. These are not hard requirements but things that would improve upon the existing setup:

• Align the functionality available within both editors.
• Allow easy switching between the editors.
• Retain all functionality, in a way that produces a compatible output, from the existing editors.
• Allow the future potential of collaboration on a page.

Requirements

This is primarily a list of current functionality we'll ideally need to retain to ensure forward compatibility.

• Retain a similar ease of use to the existing WYSIWYG editor.
• Sane API abilities for adding custom functionality.
  • Inserting content (At end, at start, at cursor).
  • Selection management.
  • UI button/controls support.
  • Allow external extensibility (To BookStack Users)
• Block formats: div, p, h2...h6, blockquote, codeblock (Currently codemirror embed on WYSIWYG), callout (div with set class), hr.
• Inline formats: code, bold (strong), italic (em), underline, strike (span with inline style), Superscript (sup), subscript (sub).
• Background and text color (Span with inline style).
• Text left, center, right, justify alignment (Class align-{center,right,left} or inline style).
• Indenting (Inline padding-left at block element).
• Lists (ol/ul) with x-level nesting support.
• Table insert & management (With header row support), ??TinyMCE provides a lot of sub-menus here, Need to support still??
• Images
  • Managed via Custom BookStack UI.
  • Custom dimensions with UI-handle resizing.
  • Alignment (Inc Floating for left/right) via alignment buttons.
  • Linked images.
  • Alt text.
• Links
  • Custom BookStack UI for linking to content.
  • [target] control.
  • Title text control.
  • Assistance for linking to in-content headings.
• Text direction for RTL language content.
• Drawings, As per images but custom functionality on double click.
• Media (iframe) embeds.
  • Dimension controls.
• Clear formatting.
• Source code viewing.
• Full screen editor viewing.
• Content preview (Markdown).
• Editor shortcuts.
• Compatible license.

Possible Options

These have not necessarily been validated as meeting all requirements as of writing, these are just found options, at a first glance, or options that have been suggested. These will start to be crossed off while being validated.

• TinyMCE 5
• CKEditor
• ProseMirror
• Quill
• EditorJS
• ToastUI
• Slate

Initial Thoughts

From a quick glance, ProseMirror would be my initial frontrunner. Will need a lot of up-front work but I'd have trust for the core system based on my experience of working with CodeMirror, which has been really pleasant.

[arpadgabor]

Great to see this initiaitive!

As another proposal, tiptap v2 was released in beta recently, it is built on top of ProseMirror and offers a very nice API and comes unstyled out of the box, so the styling is up to the dev completely (very easy to implement).

A few great features:

• Possiblity to add custom Vue/React/etc. components in the editor
• Multiplayer support (currently in closed beta)
• Bubble-style controls
• Possibility to use the native ProseMirror API's alongside what tiptap exposes
• Typescript
• HTML/JSON Output

Downsides:

• Still in beta
• No out-of-the-box markdown output support. It has markdown shortcuts though.

Tiptap checks pretty much all the boxes except for (partly) the markdown one.

[kayvanaarssen]

I Think; https://ui.toast.com/ has some great options for both Markdown and normal WYSIWYG.
It would be nice for normal users to have the normal WYSIWYG editor by default, but for advanced users the option to switch to Markdown, and see "Markdown" and the result side by side like Toast does.

Of course all the current elements needs to be transferrable to markdown and the other way around.

For more advanced users writing in Markdown is faster for sure.

p.s. Only thing that is missing i think; syntax highlighting in the code blocks like PowerShell Bash etc.

[namxam]

I haven't checked it out, but I heard some interesting use cases of monaco editor. Especially when it comes to collaboration. It's a full fledged editor and might be overkill. And I don't know about WYSIWYG. But it might be extremely powerful for advanced users.

https://github.com/microsoft/monaco-editor

[bridgeyuwa]

Please Add support for MathML / MathJax

[JHenneberg]

I definetly would prefer TOAST.UI as a editor.

It has the benefit of having a markdown Editor and a normal WYSIWYG for markdown. The difference to the current one is, that the WYSIWYG is not producing any additional html overhead so switching between the different input method is not a problem.
This is great because from markdown you can convert in nearly any other file format quite easy. But not only this, you can also "feed" your bookstack from a lot of sources which are already written in markdown.

@bridgeyuwa: it looks like math is supported via custom blocks:
https://github.com/nhn/tui.editor/blob/master/docs/en/custom-block.md

[ssddanbrown]

Input from @Xiphoseer vis Discord:

---

Very interesting to see this topic come up. I was interested in the internals of a collaborative markdown editor in the past and tried out ProseMirror to see what was necessary on the server side to support that and I'd like to share what I learned in the process.

The first observation is that a lot of these server side components are running on Node.js (Etherpad, HackMD, CryptPad) to be able to use the same collab-library as the client. I tried out to create something compatible with ProseMirror in another language (Rust) and created https://xiphoseer.de/padington as on such prototype. That works, but it's hard to track down all inconsistencies that come from being a browser app. The same might be true for PHP and laravel.

The second observation (about choosing Prosemirror) is that you need to choose a document model up-front (at editor start) because the actual HTML is generated as you type in a similar fashion to nested views/components in vue or react. That also means that you might need to implement a secondary transformation for the final page HTML that's saved to the database and store sth. like the json
representation of the model in much the same way markdown is currently stored.

Finally, one advantage is that you have a plugin system where advanced features (inline code editor, collaboration server, potentially also drag and drop, ...) can be added or swapped out as needed, even without changing the document model if only the UI changes. I use this in "padington", where the server understands the prosemirror-md document model (and stores as markdown to disk) and the client is just a plain WYSIWYG prosemirror instance with a plugin that uses my app as the collab server.

[tangramor]

How about vditor? It supports both wysiwyg / markdown / instant rendering (similar to Typora).

And it even support following content types:

Large image

[ssddanbrown]

January 2022 Update

Experience so far

I've been heavily getting into the depths of prosemirror this month.

My aim has been in working towards having a document model on the Prosemirror side with conversion to/from markdown for instant switching. As I've progressed I've grown more cynical about this approach due to complications and edge cases. It'll be a long and painful road to end up with a result that provides a stable consistent experience in that regard, while working to the main goals of what we're attempting to achieve.

Forgetting markdown for now, I've otherwise found Prosemirror to be a great editor once I could get my mind around the concepts but it's implementation for BookStack will be hindered by the legacy of what we need to support. The block model is going to be complex for all what needs to be supported, and even then some of the existing content cases just doesn't fit the block model so would need some level of handling. It would be a great option if we were building for fresh content only.

I'm concern that with the current path I'm going to consume all my time, in which I'm not employed, on attempting to get this editor to work for our needs. After that, we'd be left with an additionally very large core part of functionality to maintain.

A different potential path

We could instead remain on TinyMCE, upgrading to 5/6, which alleviates a large range of compatibility concerns. For the markdown switching we take a more pragmatic approach and allow switching via a page refresh cycle likely with a warning in the user journey to warn of potentially formatting loss concerns. Conversion would be done via the back-end systems we already maintain for exports. Editor type tied to the last page content saved. Admin options to control who can change editor type (Content-Global, User- Maybe role based).

This path allows us much more time to tackle new/improved abilities due to time saved in supporting current functionality.

One of the goals was "Allow the future potential of collaboration on a page" which was a core driver behind the ProseMirror path but, even if achieved, it would be a fair amount of time before we got to the server side support. TinyMCE does advertise a premium plugin for live collab so maybe we could attempt to engineer something ourselves if really needed, otherwise we could always again look to use another editor at a layer point.

Moving Forward

Getting a bit burnt out on this right now so gonna leave it a couple of days to mull it over. I'm siding towards the TinyMCE path at this stage since we'll pragmatically be able to make tangible progress much quicker.

Notes about some of the other editor options

• ToastUI - It's impressive what they've achieved but I don't think it's right for BookStack. It's based upon Prosemirror hence we'd face much of the same challenges in terms of existing content support. We'd have to make some quite core changes which will likely be more challenging to do with their abstractions, Would probably have to fork it which I wouldn't want to do. In addition the existing presentation and featureset is quite set, it's really more of a ready-to-use editor than one to extend.
• TipTap - Extends prosemirror. Again, great project but for us it will still be tricky. Would make the easy 80% of the work swift but we'd still remain with the tricky 20% to finish off, which may be harder working around abstractions or an already extended framework. At that stage I'd probably prefer to stay close to the core underlying library (Prosemirror).
• Editor.js - Block based, quite simple by default. I don't think a block based approach is right for BookStack at this time when keeping in mind our intended audience, plus getting to the required compatible featureset would take a lot of work.
• Slate - Similar thoughts to ProseMirror. Very reacty, not a dealbreaker but a different style to get into. Think TinyMCE may be using this in some fasion at some stage.
• CKEditor - Not sure it achieves us anything upon TinyMCE. Questionable licensing compatibility.
• Quill - Questionable project status, more intended for simpler cases.

Commenting

In regards to any comments desiring certain editors for content features, or block types, they support please keep in mind that is not the intention of this issue. This is for the main goals originally outlined. Supporting new types of content is a separate concern.

[ssddanbrown]

About to merge in #3247 so will close this off. Thanks all for your input.