Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(html-serializer): Disables tokenizers if marks/nodes are not found in the editor schema #86

Merged
merged 4 commits into from
Jan 6, 2023
Merged

fix(html-serializer): Disables tokenizers if marks/nodes are not found in the editor schema #86

merged 4 commits into from
Jan 6, 2023

Conversation

rfgamaral
Copy link
Member

@rfgamaral rfgamaral commented Jan 5, 2023 •
edited by craigcarlyle
Loading

Overview

This will help fix an internal issue where the RichTextKit is used with a bunch of extensions disabled, but since the HTML serializer does not take into account all marks/nodes that are unavailable (due to the disabled extensions), the Markdown input is still parsed as if most extensions were enabled. Because of this, the HTML output will contain markup that the editor cannot handle, thus not rendering all content as expected.

PR Checklist

  • Changes introduced here have been manually tested by someone other than the PR author
  • Pull request title follows the Conventional Commits Specification
  • Added/updated unit test cases and/or end-to-end test cases

Test plan

This PR is a bit hard and time-consuming to test because it requires changing Storybook code to replicate the behaviour that we have on Todoist, and it would take a lot of time to write a thorough test plan. However, I took some time to review the serializers unit tests to make sure that everything was tested properly.

Please focus your review mostly in the code changes (take a special look at the commit descriptions to better understand the "refactor" changes), and do a short smoke test in the deployed Storybook below:

  • Smoke test the default Plain-text editor
    • Type into the Typist editor field
      • Observe that the "Markdown Output" is as expected
    • Type into the content control (on the bottom)
      • Observe that the Typist editor content is as expected
  • Smoke test the default Rich-text editor
    • Type into the editor field
      • Observe that the "Markdown Output" is as expected
    • Type into the content control (on the bottom)
      • Observe that the Typist editor content is as expected

@rfgamaral
refactor: Review comments and code structure
2589d60 
This commit simply reviews some comments/JSDoc and rewrites the code
structure a bit (not introducing breaking changes) to allow for future
extensibility to the serializer object.
@rfgamaral
refactor: Consistent function styling for Markdown serializer plugins
4ea0b6e
@rfgamaral rfgamaral added the 🙋 Ask PR Used for PRs that need a review before merging. label Jan 5, 2023
@rfgamaral rfgamaral self-assigned this Jan 5, 2023
@netlify
Copy link

netlify bot commented Jan 5, 2023 •
edited
Loading

✅ Deploy Preview for doist-typist ready!

Name Link
🔨 Latest commit cf2f8c6
🔍 Latest deploy log https://app.netlify.com/sites/doist-typist/deploys/63b6ce30a0aed000087cd702
😎 Deploy Preview https://deploy-preview-86--doist-typist.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@rfgamaral
fix: Disables more tokenizers if marks/nodes are not found in the edi…
d2c1137 
…tor schema
@rfgamaral
refactor: Markdown tests structure and missing tests
cf2f8c6 
This commits refactors the `describe` structure for all test specs to be
more similar and consistent with the HTML serializer tests, and also
fixes and adds a few missing tests.
@rfgamaral rfgamaral force-pushed the ricardo/serializers-improvements branch from be40ae8 to cf2f8c6 Compare January 5, 2023 13:18
@rfgamaral rfgamaral requested review from a team and craigcarlyle and removed request for a team January 5, 2023 13:20
craigcarlyle
craigcarlyle approved these changes Jan 5, 2023
View reviewed changes
Copy link

@craigcarlyle craigcarlyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rfgamaral Code changes here look reasonable and no issues discovered during smoke testing 👍

@rfgamaral rfgamaral merged commit 0ed4a9b into main Jan 6, 2023
@rfgamaral rfgamaral deleted the ricardo/serializers-improvements branch January 6, 2023 11:07
doistbot added a commit that referenced this pull request Jan 6, 2023
@doistbot
chore(release): 1.0.12 [skip ci]
a311754 
## [1.0.12](v1.0.11...v1.0.12) (2023-01-06)

### Bug Fixes

* **html-serializer:** Disables tokenizers if marks/nodes are not found in the editor schema ([#86](#86)) ([0ed4a9b](0ed4a9b))
@doistbot
Copy link
Member

doistbot commented Jan 6, 2023

🎉 This PR is included in version 1.0.12 🎉

The release is available on:

Your semantic-release bot 📦🚀

@doistbot doistbot added the released Pull requests that have been released to production label Jan 6, 2023
jerrywcy added a commit to jerrywcy/memos-typist-editor that referenced this pull request Aug 12, 2023
@jerrywcy
chore(release): 1.0.0 [skip ci]
ab98bf0 
## 1.0.0 (2023-08-12)

### Features

* Add the `PasteHTMLTableAsString` extension ([Doist#290](https://github.com/jerrywcy/memos-typist-editor/issues/290)) ([ee90014](ee90014))
* Allow all marks to coexist with the Code mark ([Doist#309](https://github.com/jerrywcy/memos-typist-editor/issues/309)) ([ac06735](ac06735))
* Disallow suggestions inside inline code marks and code blocks ([Doist#154](https://github.com/jerrywcy/memos-typist-editor/issues/154)) ([7d75314](7d75314))
* Initial Typist implementation ([19451ab](19451ab))
* **serializers:** Add `get*` functions for reusable singular instances ([Doist#88](https://github.com/jerrywcy/memos-typist-editor/issues/88)) ([b2c77c3](b2c77c3))
* TaskItem and TaskList extension ([5db1900](5db1900))

### Bug Fixes

* `insertMarkdownContent` didn't insert Markdown correctly in plain-text documents ([Doist#13](https://github.com/jerrywcy/memos-typist-editor/issues/13)) ([74cc623](74cc623))
* Add required ProseMirror dependencies to package ([Doist#73](https://github.com/jerrywcy/memos-typist-editor/issues/73)) ([cd605c0](cd605c0))
* Add support for literal autolinks (GFM based) ([Doist#303](https://github.com/jerrywcy/memos-typist-editor/issues/303)) ([4537091](4537091))
* Add support for query params in `RichTextLink` extension ([Doist#4](https://github.com/jerrywcy/memos-typist-editor/issues/4)) ([9fac158](9fac158))
* Bold and Italic now use *** and ___ instead of ** and __ ([cc29ea9](cc29ea9))
* change to memos-typist-editor in package.json ([c8250b1](c8250b1))
* **deps:** Migrate ProseMirror dependencies to `@tiptap/pm` package ([Doist#151](https://github.com/jerrywcy/memos-typist-editor/issues/151)) ([d2a8eae](d2a8eae))
* **deps:** update dependency prosemirror-codemark to v0.4.2 ([Doist#34](https://github.com/jerrywcy/memos-typist-editor/issues/34)) ([58938a1](58938a1))
* **deps:** update dependency prosemirror-model to v1.18.2 ([Doist#25](https://github.com/jerrywcy/memos-typist-editor/issues/25)) ([5d1fc1b](5d1fc1b))
* **deps:** update dependency prosemirror-model to v1.18.3 ([Doist#30](https://github.com/jerrywcy/memos-typist-editor/issues/30)) ([54bfd56](54bfd56))
* **deps:** update dependency prosemirror-view to v1.29.1 ([Doist#26](https://github.com/jerrywcy/memos-typist-editor/issues/26)) ([9f86a5e](9f86a5e))
* **deps:** update dependency unist-util-remove to v4 ([Doist#380](https://github.com/jerrywcy/memos-typist-editor/issues/380)) ([6c5430a](6c5430a))
* **deps:** update gfm autolink literal packages to v2 (major) ([Doist#378](https://github.com/jerrywcy/memos-typist-editor/issues/378)) ([7ad9af8](7ad9af8))
* **deps:** update gfm strikethrough packages to v2 (major) ([Doist#379](https://github.com/jerrywcy/memos-typist-editor/issues/379)) ([b1e411d](b1e411d))
* **deps:** update tiptap packages to v2.0.0 ([Doist#198](https://github.com/jerrywcy/memos-typist-editor/issues/198)) ([fe4aa82](fe4aa82))
* **deps:** update tiptap packages to v2.0.0-beta.200 ([Doist#3](https://github.com/jerrywcy/memos-typist-editor/issues/3)) ([6e977a9](6e977a9))
* **deps:** update tiptap packages to v2.0.0-beta.202 ([Doist#9](https://github.com/jerrywcy/memos-typist-editor/issues/9)) ([ce43f74](ce43f74))
* **deps:** update tiptap packages to v2.0.0-beta.203 ([Doist#35](https://github.com/jerrywcy/memos-typist-editor/issues/35)) ([2188bc6](2188bc6))
* **deps:** update tiptap packages to v2.0.0-beta.204 ([Doist#38](https://github.com/jerrywcy/memos-typist-editor/issues/38)) ([cb5b359](cb5b359))
* **deps:** update tiptap packages to v2.0.0-beta.205 ([Doist#54](https://github.com/jerrywcy/memos-typist-editor/issues/54)) ([2074402](2074402))
* **deps:** update tiptap packages to v2.0.0-beta.206 ([Doist#59](https://github.com/jerrywcy/memos-typist-editor/issues/59)) ([27e0f26](27e0f26))
* **deps:** update tiptap packages to v2.0.0-beta.207 ([Doist#63](https://github.com/jerrywcy/memos-typist-editor/issues/63)) ([da9889f](da9889f))
* **deps:** update tiptap packages to v2.0.0-beta.209 ([Doist#75](https://github.com/jerrywcy/memos-typist-editor/issues/75)) ([97a41c5](97a41c5))
* **deps:** update tiptap packages to v2.0.0-beta.215 ([Doist#129](https://github.com/jerrywcy/memos-typist-editor/issues/129)) ([af22d22](af22d22))
* **deps:** update tiptap packages to v2.0.0-beta.216 ([Doist#135](https://github.com/jerrywcy/memos-typist-editor/issues/135)) ([3d31a2e](3d31a2e))
* **deps:** update tiptap packages to v2.0.0-beta.217 ([Doist#136](https://github.com/jerrywcy/memos-typist-editor/issues/136)) ([eebba15](eebba15))
* **deps:** update tiptap packages to v2.0.0-beta.218 ([Doist#148](https://github.com/jerrywcy/memos-typist-editor/issues/148)) ([0e9e179](0e9e179))
* **deps:** update tiptap packages to v2.0.0-beta.220 ([Doist#158](https://github.com/jerrywcy/memos-typist-editor/issues/158)) ([c7fc3d8](c7fc3d8))
* **deps:** update tiptap packages to v2.0.0-rc.1 ([Doist#192](https://github.com/jerrywcy/memos-typist-editor/issues/192)) ([26d090a](26d090a))
* **deps:** update tiptap packages to v2.0.0-rc.2 ([Doist#195](https://github.com/jerrywcy/memos-typist-editor/issues/195)) ([8753f13](8753f13))
* **deps:** update tiptap packages to v2.0.1 ([Doist#201](https://github.com/jerrywcy/memos-typist-editor/issues/201)) ([e31cb2f](e31cb2f))
* **deps:** update tiptap packages to v2.0.2 ([Doist#212](https://github.com/jerrywcy/memos-typist-editor/issues/212)) ([200b339](200b339))
* **deps:** update tiptap packages to v2.0.3 ([Doist#226](https://github.com/jerrywcy/memos-typist-editor/issues/226)) ([a1953b0](a1953b0))
* **docs:** correct the render function name ([Doist#50](https://github.com/jerrywcy/memos-typist-editor/issues/50)) ([45dd681](45dd681))
* Extra paragraph node inserted above an Horizontal Rule ([Doist#313](https://github.com/jerrywcy/memos-typist-editor/issues/313)) ([3852309](3852309))
* **factories:** Allow alphanumeric IDs for suggestion nodes in `createSuggestionExtension` ([Doist#66](https://github.com/jerrywcy/memos-typist-editor/issues/66)) ([a1726a6](a1726a6))
* **html-serializer:** Disables tokenizers if marks/nodes are not found in the editor schema ([Doist#86](https://github.com/jerrywcy/memos-typist-editor/issues/86)) ([0ed4a9b](0ed4a9b))
* **html-serializer:** Don't share instances between editors ([Doist#275](https://github.com/jerrywcy/memos-typist-editor/issues/275)) ([3aba8c7](3aba8c7))
* Invalid `hasCodeMarkBefore` check in `canInsertSuggestion` utility function ([Doist#156](https://github.com/jerrywcy/memos-typist-editor/issues/156)) ([21826c5](21826c5))
* **markdown-serializer:** Override Turndown escaping behaviour with custom rules ([Doist#102](https://github.com/jerrywcy/memos-typist-editor/issues/102)) ([6950afb](6950afb))
* **paste-markdown:** Incorrect paste behaviour when HTML source is VSCode ([Doist#260](https://github.com/jerrywcy/memos-typist-editor/issues/260)) ([3326796](3326796))
* Remove unused Tippy.js peer dependency ([Doist#56](https://github.com/jerrywcy/memos-typist-editor/issues/56)) ([85f87a5](85f87a5))
* Replace usage of useEvent with useCallback ([Doist#98](https://github.com/jerrywcy/memos-typist-editor/issues/98)) ([3b175f7](3b175f7))
* **rich-text-link:** More lenient regex for input/paste rule ([Doist#72](https://github.com/jerrywcy/memos-typist-editor/issues/72)) ([98e363f](98e363f))
* Task List now only toggle by typing `[ ]` in bullet list ([ae3103b](ae3103b))
* Unit-tests now support TaskItem and TaskList ([639abf6](639abf6))

### Reverts

* Revert "ci: Add support to publish experimental releases" (Doist#279) ([dc57964](dc57964)), closes [Doist#279](https://github.com/jerrywcy/memos-typist-editor/issues/279) [Doist#278](https://github.com/jerrywcy/memos-typist-editor/issues/278)
jerrywcy added a commit to jerrywcy/memos-typist-editor that referenced this pull request Aug 12, 2023
@jerrywcy
chore(release): 1.0.0 [skip ci]
26fb00f 
## 1.0.0 (2023-08-12)

### Features

* Add the `PasteHTMLTableAsString` extension ([Doist#290](https://github.com/jerrywcy/memos-typist-editor/issues/290)) ([ee90014](ee90014))
* Allow all marks to coexist with the Code mark ([Doist#309](https://github.com/jerrywcy/memos-typist-editor/issues/309)) ([ac06735](ac06735))
* Disallow suggestions inside inline code marks and code blocks ([Doist#154](https://github.com/jerrywcy/memos-typist-editor/issues/154)) ([7d75314](7d75314))
* Initial Typist implementation ([19451ab](19451ab))
* **serializers:** Add `get*` functions for reusable singular instances ([Doist#88](https://github.com/jerrywcy/memos-typist-editor/issues/88)) ([b2c77c3](b2c77c3))
* TaskItem and TaskList extension ([5db1900](5db1900))

### Bug Fixes

* `insertMarkdownContent` didn't insert Markdown correctly in plain-text documents ([Doist#13](https://github.com/jerrywcy/memos-typist-editor/issues/13)) ([74cc623](74cc623))
* Add required ProseMirror dependencies to package ([Doist#73](https://github.com/jerrywcy/memos-typist-editor/issues/73)) ([cd605c0](cd605c0))
* Add support for literal autolinks (GFM based) ([Doist#303](https://github.com/jerrywcy/memos-typist-editor/issues/303)) ([4537091](4537091))
* Add support for query params in `RichTextLink` extension ([Doist#4](https://github.com/jerrywcy/memos-typist-editor/issues/4)) ([9fac158](9fac158))
* Bold and Italic now use *** and ___ instead of ** and __ ([cc29ea9](cc29ea9))
* change description in package.json ([52a2535](52a2535))
* change to memos-typist-editor in package.json ([c8250b1](c8250b1))
* **deps:** Migrate ProseMirror dependencies to `@tiptap/pm` package ([Doist#151](https://github.com/jerrywcy/memos-typist-editor/issues/151)) ([d2a8eae](d2a8eae))
* **deps:** update dependency prosemirror-codemark to v0.4.2 ([Doist#34](https://github.com/jerrywcy/memos-typist-editor/issues/34)) ([58938a1](58938a1))
* **deps:** update dependency prosemirror-model to v1.18.2 ([Doist#25](https://github.com/jerrywcy/memos-typist-editor/issues/25)) ([5d1fc1b](5d1fc1b))
* **deps:** update dependency prosemirror-model to v1.18.3 ([Doist#30](https://github.com/jerrywcy/memos-typist-editor/issues/30)) ([54bfd56](54bfd56))
* **deps:** update dependency prosemirror-view to v1.29.1 ([Doist#26](https://github.com/jerrywcy/memos-typist-editor/issues/26)) ([9f86a5e](9f86a5e))
* **deps:** update dependency unist-util-remove to v4 ([Doist#380](https://github.com/jerrywcy/memos-typist-editor/issues/380)) ([6c5430a](6c5430a))
* **deps:** update gfm autolink literal packages to v2 (major) ([Doist#378](https://github.com/jerrywcy/memos-typist-editor/issues/378)) ([7ad9af8](7ad9af8))
* **deps:** update gfm strikethrough packages to v2 (major) ([Doist#379](https://github.com/jerrywcy/memos-typist-editor/issues/379)) ([b1e411d](b1e411d))
* **deps:** update tiptap packages to v2.0.0 ([Doist#198](https://github.com/jerrywcy/memos-typist-editor/issues/198)) ([fe4aa82](fe4aa82))
* **deps:** update tiptap packages to v2.0.0-beta.200 ([Doist#3](https://github.com/jerrywcy/memos-typist-editor/issues/3)) ([6e977a9](6e977a9))
* **deps:** update tiptap packages to v2.0.0-beta.202 ([Doist#9](https://github.com/jerrywcy/memos-typist-editor/issues/9)) ([ce43f74](ce43f74))
* **deps:** update tiptap packages to v2.0.0-beta.203 ([Doist#35](https://github.com/jerrywcy/memos-typist-editor/issues/35)) ([2188bc6](2188bc6))
* **deps:** update tiptap packages to v2.0.0-beta.204 ([Doist#38](https://github.com/jerrywcy/memos-typist-editor/issues/38)) ([cb5b359](cb5b359))
* **deps:** update tiptap packages to v2.0.0-beta.205 ([Doist#54](https://github.com/jerrywcy/memos-typist-editor/issues/54)) ([2074402](2074402))
* **deps:** update tiptap packages to v2.0.0-beta.206 ([Doist#59](https://github.com/jerrywcy/memos-typist-editor/issues/59)) ([27e0f26](27e0f26))
* **deps:** update tiptap packages to v2.0.0-beta.207 ([Doist#63](https://github.com/jerrywcy/memos-typist-editor/issues/63)) ([da9889f](da9889f))
* **deps:** update tiptap packages to v2.0.0-beta.209 ([Doist#75](https://github.com/jerrywcy/memos-typist-editor/issues/75)) ([97a41c5](97a41c5))
* **deps:** update tiptap packages to v2.0.0-beta.215 ([Doist#129](https://github.com/jerrywcy/memos-typist-editor/issues/129)) ([af22d22](af22d22))
* **deps:** update tiptap packages to v2.0.0-beta.216 ([Doist#135](https://github.com/jerrywcy/memos-typist-editor/issues/135)) ([3d31a2e](3d31a2e))
* **deps:** update tiptap packages to v2.0.0-beta.217 ([Doist#136](https://github.com/jerrywcy/memos-typist-editor/issues/136)) ([eebba15](eebba15))
* **deps:** update tiptap packages to v2.0.0-beta.218 ([Doist#148](https://github.com/jerrywcy/memos-typist-editor/issues/148)) ([0e9e179](0e9e179))
* **deps:** update tiptap packages to v2.0.0-beta.220 ([Doist#158](https://github.com/jerrywcy/memos-typist-editor/issues/158)) ([c7fc3d8](c7fc3d8))
* **deps:** update tiptap packages to v2.0.0-rc.1 ([Doist#192](https://github.com/jerrywcy/memos-typist-editor/issues/192)) ([26d090a](26d090a))
* **deps:** update tiptap packages to v2.0.0-rc.2 ([Doist#195](https://github.com/jerrywcy/memos-typist-editor/issues/195)) ([8753f13](8753f13))
* **deps:** update tiptap packages to v2.0.1 ([Doist#201](https://github.com/jerrywcy/memos-typist-editor/issues/201)) ([e31cb2f](e31cb2f))
* **deps:** update tiptap packages to v2.0.2 ([Doist#212](https://github.com/jerrywcy/memos-typist-editor/issues/212)) ([200b339](200b339))
* **deps:** update tiptap packages to v2.0.3 ([Doist#226](https://github.com/jerrywcy/memos-typist-editor/issues/226)) ([a1953b0](a1953b0))
* **docs:** correct the render function name ([Doist#50](https://github.com/jerrywcy/memos-typist-editor/issues/50)) ([45dd681](45dd681))
* Extra paragraph node inserted above an Horizontal Rule ([Doist#313](https://github.com/jerrywcy/memos-typist-editor/issues/313)) ([3852309](3852309))
* **factories:** Allow alphanumeric IDs for suggestion nodes in `createSuggestionExtension` ([Doist#66](https://github.com/jerrywcy/memos-typist-editor/issues/66)) ([a1726a6](a1726a6))
* **html-serializer:** Disables tokenizers if marks/nodes are not found in the editor schema ([Doist#86](https://github.com/jerrywcy/memos-typist-editor/issues/86)) ([0ed4a9b](0ed4a9b))
* **html-serializer:** Don't share instances between editors ([Doist#275](https://github.com/jerrywcy/memos-typist-editor/issues/275)) ([3aba8c7](3aba8c7))
* Invalid `hasCodeMarkBefore` check in `canInsertSuggestion` utility function ([Doist#156](https://github.com/jerrywcy/memos-typist-editor/issues/156)) ([21826c5](21826c5))
* **markdown-serializer:** Override Turndown escaping behaviour with custom rules ([Doist#102](https://github.com/jerrywcy/memos-typist-editor/issues/102)) ([6950afb](6950afb))
* **paste-markdown:** Incorrect paste behaviour when HTML source is VSCode ([Doist#260](https://github.com/jerrywcy/memos-typist-editor/issues/260)) ([3326796](3326796))
* Remove unused Tippy.js peer dependency ([Doist#56](https://github.com/jerrywcy/memos-typist-editor/issues/56)) ([85f87a5](85f87a5))
* Replace usage of useEvent with useCallback ([Doist#98](https://github.com/jerrywcy/memos-typist-editor/issues/98)) ([3b175f7](3b175f7))
* **rich-text-link:** More lenient regex for input/paste rule ([Doist#72](https://github.com/jerrywcy/memos-typist-editor/issues/72)) ([98e363f](98e363f))
* Task List now only toggle by typing `[ ]` in bullet list ([ae3103b](ae3103b))
* Unit-tests now support TaskItem and TaskList ([639abf6](639abf6))

### Reverts

* Revert "ci: Add support to publish experimental releases" (Doist#279) ([dc57964](dc57964)), closes [Doist#279](https://github.com/jerrywcy/memos-typist-editor/issues/279) [Doist#278](https://github.com/jerrywcy/memos-typist-editor/issues/278)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@craigcarlyle craigcarlyle craigcarlyle approved these changes

Assignees

@rfgamaral rfgamaral

Labels
🙋 Ask PR Used for PRs that need a review before merging. released Pull requests that have been released to production
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rfgamaral @doistbot @craigcarlyle