Visual editor

[wwwillchen]

Motivation

One of the challenges with developing Mesop applications for Python engineers with limited FE experience is that creating layouts and dealing with styling has a substantive learning curve.

By providing a visual editor, we can lower the barrier to entry and improve the velocity of Mesop app developers.

Prior Art

There's many similar attempts, but perhaps the most relevant example is Puck which is an open-source visual editor for React; it provides a very intuitive user experience. One of the differences, though, is that Puck emits JSON (a data structure of the component tree), but I'd like to emit Python code to close the gap between code produced by the visual editor vs. hand-written.

Design

High-level component structure

• Component library (left sidenav) - user can drag a component from the left sidenav and drop it into the center "canvas" area.
• Canvas area (main center panel) - this is the droppable area. Also, you can select a component to focus on it.
• Outline - similar to the existing component tree in Mesop Dev Tools, this displays the component hierarchy
• Editor (right sidenav) - this provides form controls for editing the focused component's value.

Interfaces

Server -> client

// Put in RenderEvent
message ComponentConfig {
  string component_name
  repeated EditorField fields
  string category
}

message EditorField {
  string name
  ParamType type
}

message ParamType {
   oneof type {
      BoolType bool
      IntType int
      StringType string
      StringLiteralType string_literal
      ListType list
      StructType struct
   }
}

message BoolType {
   bool default_value
}

message StructType {
   repeated EditorField fields
}

message IntType {
  int32 default_value
}

message StringType {
  string default_value
}

message StringLiteralType {
   repeated string literals // note: defaults to first element
}

message ListType {
  ParamType type
}

From client to server

• update_component_code(source_code_location, component_edit)

message SourceCodeLocation {
  string  module
  int32 line
  int32 col
}

message ComponentEdit {
  SourceCodeLocation location
  string keyword_argument (can be empty, if it's a positional argument)
  string new_value
}

Generating ComponentConfig proto from Python API

Example program:

import inspect
from typing import Callable, Any, Literal

# The button function (for reference)
def button(
    *,
    on_click: Callable[[ClickEvent], Any] | None = None,
    type: Literal["raised", "flat", "stroked", "icon"] | None = None,
    color: str = "",
    disable_ripple: bool = False,
    disabled: bool = False,
    key: str | None = None,
):
    pass

# Function to analyze and serialize the function signature
def serialize_function_to_proto(func):
    sig = inspect.signature(func)
    component_config = {
        "component_name": func.__name__,
        "fields": [],
        "category": "UI Components"  # Example category
    }

    for name, param in sig.parameters.items():
        editor_field = {"name": name}
        param_type = param.annotation

        # Map Python types to proto ParamType
        if param_type in [bool]:
            editor_field["type"] = {"bool": {"default_value": param.default}}
        elif param_type in [str]:
            editor_field["type"] = {"string": {"default_value": param.default}}
        elif param_type == Literal["raised", "flat", "stroked", "icon"]:
            editor_field["type"] = {"string_literal": {"literals": ["raised", "flat", "stroked", "icon"]}}
        # Add more type mappings as needed...

        component_config["fields"].append(editor_field)

    return component_config

# Serialize the button function
serialized_button = serialize_function_to_proto(button)
print(serialized_button)

Flows

When you drag a component, it will create a placeholder component and emit a user event. EditorEvent. Inside EditorEvent, there's three types: 1) create, 2) edit, and 3) delete. After the user event, we will do a hot reload.

• User drags component from component library into canvas
• User triggers a UserEvent of type EditorEvent (which has three sub-types: 1) create, 2) edit, and 3) delete)
• Server checks if dev mode is on. If it's off, then no-op.
• Server modifies source code.
• This triggers hot reload

Implementation notes

• Use Angular Material CDK drag and drop module.
• Need a way to emit the source code location of each component instance.

Tasks

• Refactor Mesop so that there's three modes/apps: dev, editor and prod. Create a flag for cli.py (--prod) which determines which binary to use.
• Rename DevToolsService -> EditorService and have two versions: a no-op version when in dev mode; and a real impl for editor mode (injected by the root apps).
• Create basic shell UI: ComponentLibrary component for left sidenav and ComponentFieldsEditor and ComponentOutline components for right sidenav
• Fix the top padding thing
• Create a nice background overlay of selected component
• Attach source location to component instance in editor mode (instrument component helper with hook to use inspect and get the call-site; the frame before mesop/components)
• Send repeated ComponentConfig as part of RenderEvent in editor mode.
• When clicking on a component instance in canvas; change "focused_component" in EditorService; based on this display in ComponentFieldsEditor the requisite information to modify it.
• Emit UserEvent with EditorEvent.
• Implement handlers in Python server for editor event.
• Handle struct type in editor panel.
• Allow creating an array element from editor panel.
• Provide drag and drop from ComponentLibrary to main canvas. #36

Nice to haves:

• Create a "plus sign" of new component when hovering
• Show simplified component tree - in particular, for the parent and descendants of the currently highlighted element

Open questions

• How do I create a visual placeholder when dropping an item?

[wwwillchen]

Source code manipulation

General process:

1. Parse source code into AST
2. Manipulate AST (e.g. visitor)
3. Print AST back into source code

We can use libcst or astor for this. LibCST is probably better because it'll preserve things like commenting and formatting

Example program
Input: me.box(columns=2)
Output: me.box(columns=4)

[wwwillchen]

Limitations

If there's any indirection, e.g. using a variable instead of assigning literal values, then this is going to be difficult to programmatically update from the UI. It's technically do-able, (e.g. look up alias reference), but this becomes quite complicated beyond the simplest indirections.

Instead, we'll show a disabled input with a tooltip to explain why it's not editable.

Instead of having shared values; we should steer app developers to instead use theming, which is dependent on Angular Material's new theming to be launched.

[wwwillchen]

Command Dialog

We can provide a flexible way of allowing users to search through commands (inspired by Raycast when they have focused on a component. (We can also later provide a hotkey, e.g. cmd+K, to open the dialog even without a focused component).

• v1 When creating a new component, we will open the command dialog.
• v2 When focused on a component, we can try to show it as part of a shortcut, e.g. command)
• v3 Pop-up when there's no focused component.

This is a flexible way of adding richer, assistive feature into Mesop editor over time while keeping a minimal and consistent UI

[wwwillchen]

Different cases to handle:

• Support struct (e.g. style API)
• Support list struct (e.g. radio option)
  • Edit option
  • Delete option
  • Add option
• Support add component
• Support delete component
• Support copy component