Proposal: Automated operator-demo video generator

[xuang7]

Motivation

Texera has around 160 operators, and most of them do not have a demo. The current operator descriptions are often limited, so users may still be unsure how to configure or use an operator in practice. Recording a video for every operator manually does not scale.

This proposal introduces a Playwright-based tool that automatically generates a short sample video for each operator, so operators can include an up-to-date example of how they are used.

These videos are not intended to be detailed tutorials. Instead, each video provides a concise example showing:

• what users are expected to enter in the operator form,
• how the operator is added to a workflow,
• and what result users can expect after running it.

As a side benefit, generating the videos exercises the full operator workflow end to end: logging in, creating a workflow, dragging an operator onto the canvas, filling in its properties, running the workflow, and viewing the result. This can also provide end-to-end coverage for common operator interactions.

Generation Pipeline

A developer can ask an agent to recommend sample field values for an operator based on a given dataset. The recommended values are then reviewed and stored as configuration data.

The remaining generation is deterministic. Each operator group has a defined script template that encodes how operators in that group are demoed: which sample workflow to load, where to drag the operator and how to connect it, and whether a dataset needs to be selected. OperatorScriptGenerator picks the template by the operator's group, fills in the operator's name/type and its field values from operator-field-values.json, and emits the per-operator script.

Sample video:

line-chart_demo.webm

Rollout plan

Horizontal split by code layer into ~10 small, independently-compiling PRs, ordered along the demo flow (frontend testids, flow setup, login/navigation, drag, fill, run, dataset, config framework, demo data, generator+entry). Each PR compiles on its own; only the last wires the entry point so the tool runs.

Maintenance Principle

A key goal is to keep this process lightweight and avoid turning operator demo generation into an additional maintenance burden.

Operator authors should only need to provide or update the sample field values required for their operators. They should not be expected to maintain Playwright scripts or video-generation infrastructure directly. Shared controllers and generation logic should be maintained centrally. When UI changes affect multiple operator demos, the corresponding updates should be made once in the shared controller layer rather than separately for every operator.

Video regeneration should also be an occasional release task rather than a required step for every operator change or pull request.

Maintenance Model

Role	Responsibility
Operator authors	Provide and validate the recommended sample values for their operators.
Tool maintainers	Maintain the shared Playwright controllers, script generator, and generation infrastructure.
Regeneration	A maintainer, possibly the release manager, runs a full regeneration before major releases.
Hosting	The generated videos are published publicly on YouTube.

Limitations

Requires a full running stack: Generating or regenerating videos requires a live Texera deployment, including the backend, frontend, seeded sample datasets, Playwright, and Chromium. This is not a lightweight process and cannot run in a minimal CI environment.

Playwright flows are timing-sensitive: End-to-end browser flows may occasionally fail because of timing issues rather than actual regressions. A retry or tolerance strategy may therefore be needed if these flows are used in automated checks.

UI coupling is reduced but not eliminated: The controllers rely on stable data-testid attributes where possible. However, some interactions still depend on ng-zorro component classes or canvas geometry. Certain elements rendered by the component library, such as dropdown popups, cannot easily be assigned test IDs. As a result, frontend or component-library changes may still break the generation flow and require updates to the shared controllers.

Open Questions

1. Should operator authors only maintain sample values and validate the generated result?
2. Who should own the shared controllers when UI changes affect the generation flow?
3. Should generation remain a best-effort maintenance task and therefore not block a release?
4. How much should video quality matter? The current default datasets and workflows are intentionally kept simple, but this may produce outputs whose colors, scales, or visual appearance are not always intuitive. Should we invest additional effort in improving the sample data and visualization settings for presentation quality?

Please feel free to share any thoughts, concerns, or suggestions. Thank you.

[chenlica]

Here are my thoughts:

1. The author of each new operator is responsible for the content of the corresponding video. We need to provide clear instructions and script for each author to use to generate the video and test the quality.

2. I don't fully understand the meaning of "controllers."

3. Due to the effort and storage cost for the generated videos, we can generate new videos with a reasonable frequency, e.g., for each major release.

4. We want to minimize the effort. Some "good enough" videos are sufficient for now.

[xuang7]

Agreed! Here, "controllers" refer to the shared, reusable code that drives the browser via Playwright through each step of the workflow (e.g., logging in, creating a workflow, dragging an operator, filling out forms, running the workflow, and viewing the results). Each operator's script simply invokes these shared controllers instead of reimplementing the browser interactions.

[chenlica]

Please modify the description to explain the meaning of "controllers."

[Neilk1021]

1. Full-length (1min+) videos might be a bit excessive; I agree with Chen that the author should be responsible for making their video.
2. Photoshop has something similar, which is a pretty clean implementation of this.

3. I think it depends on what you mean by "quality." Videos should likely be as concise as possible while still showing what an operator does.

[xuang7]

Yes, thanks for the thoughtful feedback! The current videos spend some time setting up the workflow, and a few steps include longer delays than necessary. I think we can adjust delay times so that each operator video is around 30–40 seconds.

The goal is to show users what results they can expect after running the operator, rather than provide a full tutorial explaining every field of how the operator works. This way, the videos can remain concise while still giving users a clear preview of the operator's functionality.