feat: support add custom inline nodes

[Flrande]

After this PR is merged, externals can implement customization for inline elements by overriding the service for specific blocks.

InlineSpec

InlineSpec is a format for defining inline elements. Externals can describe the rendering method and corresponding data structure for a particular inline element using this format.

type InlineSpecs<
  TextAttributes extends BaseTextAttributes = BaseTextAttributes,
> = {
  // Field name, corresponding to the key in delta attributes
  name: string;
  // Schema for this field
  schema: ZodTypeAny;
  // Matching function, if the function returns true, an inline element will be rendered internally using the renderer defined by this InlineSpec
  match: (delta: DeltaInsert<TextAttributes>) => boolean;
  // Rendering function, includes two parameters: delta and selected, where the latter indicates whether the element is selected
  renderer: AttributeRenderer<TextAttributes>;
  // An optional field, indicating whether the element is of embed type
  embed?: boolean;
};

Renderer

The renderer in InlineSpec needs to return a TemplateResult to implement the rendering of inline elements. For its implementation, you can refer to the following link:

affine-text is the most common non-embed type of inline node in blocksuite, responsible for bolding, underlining, and other basic styles.

blocksuite/packages/blocks/src/_common/inline/presets/nodes/affine-text.ts

Lines 45 to 78 in 8fd0f98

	@customElement('affine-text')
	export class AffineText extends ShadowlessElement {
	static override styles = css`
	affine-text {
	white-space: break-spaces;
	word-break: break-word;
	}
	`;
	@property({ type: Object })
	delta: DeltaInsert<AffineTextAttributes> = {
	insert: ZERO_WIDTH_SPACE,
	};
	override render() {
	const style = this.delta.attributes
	? affineTextStyles(this.delta.attributes)
	: styleMap({});
	// we need to avoid \n appearing before and after the span element, which will
	// cause the unexpected space
	if (this.delta.attributes?.code) {
	return html`<code style=${style}
	><v-text .str=${this.delta.insert}></v-text
	></code>`;
	}
	// we need to avoid \n appearing before and after the span element, which will
	// cause the unexpected space
	return html`<span style=${style}
	><v-text .str=${this.delta.insert}></v-text
	></span>`;
	}
	}

latex-node is an embed-type node used as an example in this PR, capable of rendering mathematical formulas.

blocksuite/packages/playground/apps/default/specs-examples/latex/latex-node.ts

Lines 17 to 65 in 8fd0f98

	@customElement('latex-node')
	export class latexNode extends ShadowlessElement {
	static override styles = css`
	.affine-latex {
	white-space: nowrap;
	word-break: break-word;
	color: var(--affine-text-primary-color);
	fill: var(--affine-icon-color);
	border-radius: 4px;
	text-decoration: none;
	cursor: pointer;
	user-select: none;
	padding: 1px 2px 1px 0;
	display: inline-block;
	}
	.affine-latex:hover {
	background: var(--affine-hover-color);
	}
	.affine-latex[data-selected='true'] {
	background: var(--affine-hover-color);
	}
	`;
	@property({ type: Object })
	delta: DeltaInsert<TextAttributesWithLatex> = {
	insert: ZERO_WIDTH_SPACE,
	};
	@property({ type: Boolean })
	selected = false;
	override updated() {
	const latexContainer = this.querySelector<HTMLElement>('.latex-container');
	assertExists(latexContainer);
	latexContainer.replaceChildren();
	katex.render(this.delta.attributes?.latex ?? '', latexContainer, {
	throwOnError: false,
	displayMode: false,
	output: 'mathml',
	});
	}
	override render() {
	return html`<span class="affine-latex" data-selected=${this.selected}
	><span class="latex-container"></span
	><v-text .str=${ZERO_WIDTH_NON_JOINER}></v-text
	></span>`;
	}
	}

InlineMarkdownMatch

Currently, we have preset a mechanism to trigger inline formatting changes using markdown syntax. For example, in "abc", if you want to change "b" to bold, you need to enter 'a' + '**b**' + space + 'c'. InlineMarkdownMatch is a format that describes such shortcuts. Apart from realizing inline formatting transformations through these preset shortcuts, it is more common to achieve this through extending the format-bar. However, this goes beyond the scope of this PR, so no further description will be provided.

type InlineMarkdownMatch<
  TextAttributes extends BaseTextAttributes = BaseTextAttributes,
> = {
  name: string;
  pattern: RegExp;
  action: (props: {
    inlineEditor: InlineEditor<TextAttributes>;
    prefixText: string;
    inlineRange: InlineRange;
    pattern: RegExp;
    undoManager: Y.UndoManager;
  }) => ReturnType<KeyboardBindingHandler>;
};

Latex Example

This PR uses new API to write a simplified example, which can be experienced at branch playground, and the main core code is as follows:

blocksuite/packages/playground/apps/default/specs-examples/latex/latex.ts

Lines 20 to 121 in 8fd0f98

	const latexSpec: InlineSpecs<TextAttributesWithLatex> = {
	name: 'latex',
	schema: z.string().optional().nullable().catch(undefined),
	match: delta => !!delta.attributes?.latex,
	renderer: (delta, selected) => {
	return html`<latex-node
	.delta=${delta}
	.selected=${selected}
	></latex-node>`;
	},
	embed: true,
	};
	const latexMarkdownMatch: InlineMarkdownMatch<TextAttributesWithLatex> = {
	name: 'latex',
	/* eslint-disable no-useless-escape */
	pattern: /(?:\$)([^\s\$](?:[^`]*?[^\s\$])?)(?:\$)$/g,
	action: ({ inlineEditor, prefixText, inlineRange, pattern, undoManager }) => {
	const match = pattern.exec(prefixText);
	if (!match) {
	return KEYBOARD_ALLOW_DEFAULT;
	}
	const annotatedText = match[0];
	const startIndex = inlineRange.index - annotatedText.length;
	if (prefixText.match(/^([* \n]+)$/g)) {
	return KEYBOARD_ALLOW_DEFAULT;
	}
	inlineEditor.insertText(
	{
	index: startIndex + annotatedText.length,
	length: 0,
	},
	' '
	);
	inlineEditor.setInlineRange({
	index: startIndex + annotatedText.length + 1,
	length: 0,
	});
	undoManager.stopCapturing();
	inlineEditor.formatText(
	{
	index: startIndex,
	length: annotatedText.length,
	},
	{
	latex: String.raw`${match[1]}`,
	}
	);
	inlineEditor.deleteText({
	index: startIndex,
	length: annotatedText.length + 1,
	});
	inlineEditor.insertText(
	{
	index: startIndex,
	length: 0,
	},
	' '
	);
	inlineEditor.formatText(
	{
	index: startIndex,
	length: 1,
	},
	{
	latex: String.raw`${match[1]}`,
	}
	);
	inlineEditor.setInlineRange({
	index: startIndex + 1,
	length: 0,
	});
	return KEYBOARD_PREVENT_DEFAULT;
	},
	};
	class CustomParagraphService extends ParagraphService<TextAttributesWithLatex> {
	override mounted(): void {
	super.mounted();
	this.inlineManager.registerSpecs([...this.inlineManager.specs, latexSpec]);
	this.inlineManager.registerMarkdownMatches([
	...this.inlineManager.markdownMatches,
	latexMarkdownMatch,
	]);
	}
	}
	class CustomListService extends ListService<TextAttributesWithLatex> {
	override mounted(): void {
	super.mounted();
	this.inlineManager.registerSpecs([...this.inlineManager.specs, latexSpec]);
	this.inlineManager.registerMarkdownMatches([
	...this.inlineManager.markdownMatches,
	latexMarkdownMatch,
	]);
	}
	}

CleanShot.2024-01-03.at.11.22.00.mp4

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
blocksuite	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 3, 2024 4:47am

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
blocksuite-docs	⬜️ Ignored (Inspect)	Visit Preview		Jan 3, 2024 4:47am

[doodlewind]

Briefing the API surface in PR description will be very helpful :)

[Saul-Mirone]

How to view the demo in playground?

[Flrande]

done @doodlewind @Saul-Mirone

[pengx17]

There are more features in AFFiNE requires us to extend the default blocksutie doc editor. For example, we want to let the inline page reference to have different ui/ux in some cases - like localization for journal pages.

If we try to extend "LinkedPage" reference, this is what we can do right now based on the changes in this PR:

1. define a customaffine-referencefor AFFiNE
   a. use Lit or plain HTML/web component as the renderer expects lit template return values
   b. use use Lit if we want to extend the existing reference component
2. override .mountedinParagraphService,ListService,DatabaseServiceto use the customizedAffineInlineSpec
3. find and replace the services in the default DocSpecs preset
4. assign the custom specs to the Editor component

It looks to me that this approach is still not mature enough for use in affine yet.

1. Lit issues:
   a. If we want to use React, then how to pass individual React component/element to Lit?
   b. If not, we may need to setup a Lit toolset in Affine (includingexperimentalDecorators?)
2. Extending existing behaviours:
   a. There is actually no way to extend the editor, but to replace the internal implementation. Not easy for behavior extension - the bs is a white box
   b. On the other hand, if blocksuite allows the AFFiNE side to define custom block renderers, the affine side may easily write bad data which breaks the data integrity given the blocksuite's schema. This makes it hard for the blocksuite side to revolve onwards.

[Flrande]

There are more features in AFFiNE requires us to extend the default blocksutie doc editor. For example, we want to let the inline page reference to have different ui/ux in some cases - like localization for journal pages.

If it's just about changing the UI, maybe more reasonable to provide some kind of interface directly in affine-reference? (just like #5954)