[Idea]: Support ComfyUI for Image Generation.

[PhillLittlewood]

What are you trying to do?

The system I use must be isolated from the internet so all services must be locally hosted. I want open-notebook to be able to call a ComfyUI workflow when it needs I ages.

What feels difficult or missing today?

I cannot currently create anything that requires images.

What outcome would help?

Allow open-notebook to have an image generation tab in settings that will accept a comfyui URL and a ComfyUI image workflow. This could also be expanded to use ComfyUI for TTS also.

How do you handle this today?

I need to manually generate any images I need.

Additional context, examples, or possible directions

No response

How would you like to participate?

• I can add use cases or join the product/design discussion.
• I can help explore the technical approach.
• I can help implement an approved work item.
• I can test or validate a proposed solution.

Before posting

• I searched existing Discussions and Issues for related proposals.

[lfnovo]

Direct answer first: Open Notebook has no image generation today — not "no ComfyUI", none at all. Our model layer (esperanto) covers language, embeddings, reranking, speech-to-text and text-to-speech; image generation isn't a modality there yet. #691 (podcast cover art) was closed for that reason, and #203 is where visual outputs are being thought through as a direction.

So the real need is image generation as a capability, and ComfyUI is one way to serve it. Hosted models (OpenAI, Google) and open-weight models served locally can all sit behind the same interface; for an air-gapped setup like yours, ComfyUI or a plain local HTTP runtime would be the backends. I've opened the modality request in esperanto: lfnovo/esperanto#277 — that's where the provider contract gets shaped, and a ComfyUI adapter ("workflow JSON with a designated prompt node, substitute and poll") is a natural entry there.

Two things that shape it from the Open Notebook side:

1. Where would images appear for you — podcast covers, illustrations in study guides, diagrams? The consumer defines the contract.
2. For TTS: esperanto already has providers, so a ComfyUI TTS backend would be a separate, smaller thread.

You ticked every participation box. The concrete first step is the esperanto modality design — if you want to drive it, say so there and here.

Status: parked until champion — capability tracked in lfnovo/esperanto#277, product direction in #203.