Modularisation

[MikeAliG]

Introduction

Hello! First of all, thank you for the great deal of work you have done for this project and the community as a whole. Ever since this plugin has been created, we have used it in many of our internal and commercial projects and since then, we had some thoughts that might benefit everyone and make this project even greater! And I am 100% open to new ideas and if you are okay with the following ideas, I can send a PR with the initial draft and we can discuss it even further. So feel free to let me know your ideas, I'd love to hear them.

(PS. Some of these ideas are closely related to #326 #178 )

Abstract

This proposal will solve many performance, bundle size issues and make this plugin usable in medium/large scale applications and improves developer happiness.

Initial thoughts:

1. If we take a closer look at at Bundlephobia we can see two things, First, this plugin is Treeshakable, Second, it is packed with 23.1% of uifabric modules. Although it might not sound a lot but in reality, it is somewhat incrementally adding to the bundle size and the biggest issue here is the uifabric's performance! We are usually dealing with huge documents with so many custom elements and editable voids, etc. And UIFabric has so many performance issues that it is not safe to use it in production especially on devices with lower memories. Again, this leaves us two options:

   • Forgetting the plugins and going with @udecode/slate-plugins-core and re-implementing the plugins via custom plugins, which would be bad in terms of maintenance since if there are any breaking changes to the core, we cannot use those patch fixes and hotfixes of the original plugins and consequently we have to fix everything ourselves. Besides, the plugins might also have some improvements and changes which we have to match with our custom implementation.
   • Biting the bullet and using the original plugins despite performance and bundle size issues.

   Although none of those are desirable, we have been choosing the first approach and we had to make them in sync with the original plugins and API which was not easy. Considering how fantastic this plugin is, we could not leave it altogether since it has done a great deal for us and to be frank, plain Slate JS, cannot do much since nothing is documented properly on the Slate's website.

2. The utility functions in the slate-plugins are truly amazing and do the heavy-lifting for most of the stuff for any custom plugin but unfortunately they are under the plugins workspace and we have the same issues mentioned above.

3. It is currently very hard to do analytics on each component and implementing those analytics requires Custom Component structure which I will expand further on this issue below.

4. Full Docs of each plugin and how they could be extended to cover all of the cases in which these functionalities based on the Atomic Design.

Final Thoughts:

We thought about the logistics of how we could essentially fix all these problems without having any sort of breaking changes in the library and making sure everything is in its place with no visible change to the existing users of slate-plugins while giving more freedom to use the funtionality of the plugins to extend it even further. As we can see the project is a yarn workspace and these issues could be fixed by incorporating the workspace approach. I have detailed the possible solutions in the proposal below.

Proposal

Introducing Renderless Slate-Plugins & Utils:

Most of the issues mentioned above could be solved by making two new workspaces/packages as seen below:

├── packages
│   ├── core
│   ├── slate-plugins
│   ├── slate-plugins-utils
│   ├── slate-plugins-renderless

Slate-Plugins

The slate-plugins will not have any noticable change by the end user but the only difference is that it will be using the slate-plugin-renderless to add styling functionality for the users who want to have basic structure of the original module. (More details below)

Slate-plugins-utils

The slate-plugin-utils can have all of the Basic Utilities to make great experiences for the custom user made plugis.

Slate-plugins-renderless

The slate-plugins-renderless can have the renderless packages with no styling support and just the bare functionality which could be extended upon. Essentially doing the heavy-lifting with no styling or extra libraries as detailed below:

• Each plugin can have an option to receive a custom element and rendering those custom elements without having the extra styling libraries and extra modules added to them. For example the ParagraphRenderElementPropsOptions
  can receive a Custom component and it can do all the other operations on those custom components. For example:

// renderElement options given as props
export interface ParagraphRenderElementPropsOptions
  extends Omit<StyledComponentPropsOptions, 'children'> {}

could be changed to:

export interface ParagraphRenderElementPropsOptions{
   component: <Props>(props:Props) => React.ReactElement
}

and could be used as:

ParagraphPlugin({
    component: (props) => <MyCustomComponent {...props}>
})

And the same can go for the withPlugins.

The breaking change from this issue could be solved by unleashing the full potential of the yarn workspaces and having these Renderless elements as their own packages/workspace and having the original Slate-Plugin using those Renderless components to add the styling funtionality as before but with more flexibity. This means every user from those who are more comfortable with using the Styled-components and those who just want to use the Renderless components can use the package suits their needs while not compromising one for the other AND even more flexibility for the previous users.

• Most of the elements, decorators and widgets, etc. are able to be their own separate packages within the yarn workspace while being re-exported from the main slate-plugins module to tackle the breaking change issue. This gives better performance and smaller bundle size out-of-the-box but if somebody wants to install all the packages, they can use the original slate-plugins package from which all these other elements, widgets, etc are re-exported and they can treeshake if they wish to do so.

Rationale

While studying the whole codebase we realised these issues are solvable and can be solved by a PR, if you would like to go with these proposals.

Compatibility

This approach does not seem to have any sort of breaking change to the existing codebases but if you find any breaking change, please let me know, I'll try my best to find a solution.

Conclusion and Ending Words

So, again, thanks for everything you've done for this project. If you have any questions please let me know.

Have a great day.

[zbeyens]

Hello, thanks for the detailed proposal.

Until now, this was not high priority as we're still in a "beta" phase and the demand was not high. But I think it's the good time to make these changes for 1.0.0 release.

Packages

As you may have seen, all the independent plugins are already organized by folder, most having the common folder as dependency. The intent is to make a package @udecode/slate-plugins- for each folder:

• slate-plugins: having all the below dependencies (requirement: tree-shaking) as a quick start.
• core: core functionality providing EditablePlugins
• common: I would rather call it like this as it contains many types of function: transforms, queries, utils,...
• preview-md
• deserialize: only has these dependencies
  • deserialize-html
  • deserialize-md
• serialize: only one dependency (a bit useless then)
  • serialize-html
• dnd
• elements
  • a package for each element
• marks
  • a package for each mark
• utils ( propose a better name?): if the remaining plugins do not have any dependencies (fully tree-shakeable), just make one package gathering these
  • a package for each handler, normalizer, widget

Decoupling Rendering/Styling

I started a discussion to resolve this issue, can you give your feedback? #295

Slate Components

Package: @udecode/slate-plugins-components

In the current state, you can pass a custom component using the component option of each plugin.

We also want to let people create slate "design system components" inside a single package: material-ui, antd,.. But it would be cumbersome to manually add the provided components to the options of each plugin.

Solution: <SlateComponents components={slateComponents} /> provider.

interface SlateComponent<Props> {
	component: (props: Props) => React.ReactElement,
	props?: Props // there would be the fluent styles props
}

interface SlateComponents {
	[type: ComponentType]: SlateComponent<ComponentProps>
}

// Hook to get a component:
useSlateComponent: (type: string) => (props: RenderElementProps) => React.ReactElement

• each rendering plugin will render a component:
  • options.component if defined
  • useSlateComponent(pluginKey) if defined.
  • default to a div for an element, span for a mark.
• it would be nice to have intellisense (typescript) on each component props

Fluent Components

Package @udecode/slate-plugins-components-fluent
Dependency: @udecode/slate-plugins-components

These components support css-in-js using the styles props.

• BREAKING CHANGE: remove the styles option from rendering plugins.
• move all plugins components to this package.
• exports slateComponents: minimal styling.
• exports slateStyledComponents: full styling.
• exports createSlateStyledComponents: (slateComponents: SlateComponents) => SlateComponents to override the styles of slateComponents
• Storybook examples will use slateStyledComponents.
• if we don't want to use all the components (bundle size), we can create a custom SlateComponents object.

Components having dependency can have their own package for bundle size.

See #283 #230

Styling with css

As a good practice, all the slate nodes should have (nested) class names, so anyone can style these without slate-components or css-in-js solution.

Requirement

I have one main requirement to keep our development productivity with storybook: we need a command to compile all the packages on each change (rollup watch).

What do you think about this?

[MikeAliG]

Hi, thanks a lot for your great response.

I like your approach a lot better and of course, more optimal especially in terms of compartmentalisation and ease of use. Your approach is also better since the core functionality of each plugin is separated to their core functions/packages which could provide even more flexibility and bundle size control.

Recognition and momentum

As previously said, this package has more than enough features to make it the default way to create rich text editing experiences in React with the massive heavy-lifting it is doing in terms of plugin structure and many more. It is currently standing at ~ 3,251 downloads which is fantastic but I believe we can make it more predominant. Of course, if you are okay with it. I was having plans to make a whole dedicated docs to have detail explanation for each plugin and hopefully writing some blog post about the use cases from the people who have been using it in production since early days. My initial thoughts are condensed below.

Docs

The new dedicated docs can have detailed documentation about each plugin/util/helper/element/etc. with their dedicated sections detailing everything from roadmap of features to detailed props and live playground to interact with the plugins in the browser and introducing an slate-plugins live playground where the users can play around with the code and plugins both in the dedicated sections of the docs and all-together playground. Of course, the Storybook will still be in place after the docs are released as this is in no way a replacement for storybook but as for the beginners, it is much easier for them to have everything explained which is understandable. This will encourage more users to test out the plugins/packages and consequently building more momentum.

Blog posts

We can work on a series of tutorials on building a full-featured rich text editor in React from covering the most common use cases to providing advanced use cases such as custom plugins to showcase the potential of the Atomic plugins and utility functions. Throughout the series we can post those blog posts across dev blogging platforms and also the slate-plugins' website/docs.

Package facilitations

Common & Utils Packages

As for the common and utils packages, I also fully agree on your decision on those since as you rightfully said common package may have more unique functions which should not be mixed together with utils. One possible name for the utils package may be helpers but both are fine in my opinion.

Serialise Package

The serialise package may be very useful especially in terms of transports. For example, if you have plans to support Docx format or Markdown serialisation, they could be facilitated via this package.

Decoupling Rendering/Styling

@udecode/slate-plugins-components

This is a fantastic approach not only in terms of structure but also in terms of intellisense. We could potentially expand it further to receive custom props and generics in the component and merge them to provide better intellisense (even maybe in the browser!). Also the toolbar components can also become renderless in this package and the components could be supplied via the same Provider Pattern which I will explain further below.

Provider Pattern

There are two great ways of incorporating the Provider Pattern.

1. Using <SlatePluginProvider/>

Benefits

• Render time evaluation.
• Wrapper component is easier on the eyes

2. Using a useSlatePlugins hook

Benefits

• Faster Composition
• Better encapsulation support

As for the packages which provide the above pattern, we have tried several Providers for our plugins and we have learnt some stuff from those in testing with files sizing around 60-70 MB of rich content and I can compare and contrast them in the tables below.

General Facts

React.createContext()	Zustand	Redux	Recoil
No dependency	Small size (775 B)	Big bundle size (considering all the utility functions)	Very big (14 kB)
Very Fast	Very Fast	Slow	Fast
Great Perfomance	Great Performance	Not so great performance	Good Performance

Benefits and Contrasts

React.createContext()	Zustand	Redux	Recoil
No dependency	Small size (775 B)	Big bundle size (considering all the utility functions)	Very big (14 kB)
Context Loss (Especially in large files)	No Context Loss	No available data	No available data
No concurrency Support	Concurrency Support	(?)	Concurrent Mode (?)
No Stale Data	No Stale Data	Stale Data and Zombie Child Processes	No Stale Data
Higher VDOM manipulation	Moderate VDOM manipulation	Moderate VDOM manipulation	Moderate VDOM manipulation

But these are just some of the most famous ones but if you have more of them, or if you know better solutions, please let me know. We internally use Zustand to manage the data transfer between the nodes and the plugins and of course, in our use case we are dealing with constant changing data but if you'd like to add no extra dependency we can stick to the React.createContext()

Provider API

<SlatePluginProvider/>

As you proposed in #295 This one could receive an object and a one-to-one prop for each function and the rest could be handled by the @udecode/slate-plugins-components. The possible API proposal is mentioned below:

<SlatePluginProvider components={{
        paragraph: (props) => <MyCustomParagraph {...props}/>
}}>
        <EditablePlugins ... />
</SlatePluginProvider/>

useSlatePlugins

This could also be another option from which could potentially encapsulate most of the common functionality. For example, it can receive the createEditor function and withPlugins and we can pipe and memoize them internally and have an stable API out-of-the-box. For example:

const {editor, plugins} = useSlatePlugins({
      // This editor plugin will be piped with the `withPlugins`  below.
      editor: createEditor,
      withPlugins: [withReact, withHistory],
     // The reason for this re-exportation is that, in this case we can internally check for incompatibility 
     // issues and throw error if their custom components are not setup properly instead of having a soft fail
      plugins: [ParagraphPlugin, ListPlugin],
     // This is the part where they can supply their custom components:
     components: {
         paragraph: (props) => <MyCustomParagraph {...props}/>,
     }
})

As always thanks for reading this! Let me know your ideas. I'd love to hear them.

[zbeyens]

Thanks again for your interesting response.

Docs & blog posts

That would be a big pleasure, please go forward. I'm totally open to use something else than Storybook, either as complementary or as a replacement.

Packages

We can go for slate-plugins-helpers.

Store

I was using https://github.com/diegohaz/constate resolving the "Higher VDOM manipulation" problem, but I was not aware of these Zustand's advantages, thanks for the comparison! Let's use Zustand.

Hook vs Provider

I like the useSlatePlugins approach, let's go for this. I assume it would go in pair with Zustand store to access the components from the plugins.

Options vs Store

I think we can leverage the store to minimize passing options into React components and use* functions. I am first thinking to the element/mark types.

Next

Big thanks for reviewing my proposal, I was waiting for that before going forward as it's a first time for me.

How would you split this big task into smaller steps?

[MikeAliG]

Thanks for your kind words.

Docs & Blog Posts

Sure! I'll start the series very soon once we can restructure the API.

Packages

Great! I'll make sure I put that under my list.

Store

Fantastic! I'll look forward to it

Hooks

That's correct. The useSlatePlugins() can go hand to hand with the Zustand store to handle most things and what's great in this approach is that, in this case, the user doesn't need to wrap their EditableComponent with anything since Zustand works with observer pattern. As you rightfully said, we can leverage this pattern for any element/mark to make them highly reactive which gives the users the ability to change the elements on the fly without any performance cost and even in the project's docs!

Option vs Store

You are 100% correct on this as well! It is much easier with to pass the options via store rather than individual components.

Next

Thank you for you ideas! I am starting with a draft PR today. Here's my thought process:

1. We can start modularising each section to its individual workspace (common, marks, elements, etc.)
2. Add Zustand to handle high level options
3. Docs
4. Full Blog Posts

Of course this is how it is on the grand scheme of things but if you like we can make a new repo and start the development there or I can add you to a project on Github/Asana/Trello and we can prioritise the steps. Your call is what matters, so thanks again for listening to me and I will wait for you call.

Many Thanks.

[zbeyens]

For the new docs, Docusaurus can be a good choice https://v2.docusaurus.io/

[MikeAliG]

Sure. I'll put that on the schedule.

[zbeyens]

Thanks for your time! That's now clear enough to start. I invited you as a collaborator, please create a dedicated branch for each deliverable. I'll deeply review/edit your PRs against this branch, and I can assign to myself the migration to slate-plugins-components and slate-plugins-components-fluent once useSlatePlugins is implemented.

I also created a GitHub board, feel free to manage it. https://github.com/udecode/slate-plugins/projects/2

[santialbo]

Also it looks like the entire lodash library is being included despite using the treeshakeable imports. This could be because of this singe line where it's importing "lodash".
https://github.com/udecode/slate-plugins/blob/1cbc427eb46a963c372874d9e07ec2ccc001ad0f/packages/slate-plugins/src/common/plugins/node-id/withNodeID.ts#L1

[MikeAliG]

Hello! Thanks. I just started with the @udecode/slate-plugins-common and hopefully will test and push but I checked with the Github Project, I saw that apparently I don't have access to create cards and one more thing: the package __test-utils__ is being used by almost all workspaces do you prefer to have it as a global module or do you prefer having it as sub directory?

[zbeyens]

Great! Please accept the collaborator invitation then let me know if it's still an issue.

As for __test-utils__, it can indeed be a dev dependency.

[MikeAliG]

All fixed! Thanks.