Live terminal output

[pythongosssss]

This updates the logging from using a MemoryHandler to overriding stdout/stderr in order to obtain all outputs, including progress bars. (Previously done here #4664)

Adds two new endpoints for the frontend
GET /internal/logs/raw - returns an array of the last 300 raw stdout/stderr entries and their timestamps, and the size of the terminal (required for accurate rendering)
POST /internal/logs/subscribe - Body: clientId: the socket id, enabled: if logs are being subscribed to send via websocket in a new event logs.

The format of /internal/logs has slightly changed as it is no longer formatted via logging but just the outputs.

The logs event contains any entries since last flush and the terminal size if it has changed.
Events are only sent via the websocket when a client is subscribed to reduce unnecessary traffic.

[huchenlei]

LGTM! @comfyanonymous PTAL

[comfyanonymous]

There's an issue with this on Linux where if I add some prints in the code I only see them after I ctrl-c comfyui.

[pythongosssss]

I'm not able to reproduce using WSL2, are you getting no log output at all?

[comfyanonymous]

I add a print here for example: https://github.com/comfyanonymous/ComfyUI/blob/master/comfy/ops.py#L67

[huchenlei]

Seems here the handling of progress messages is altering original terminal behavior.

[huchenlei]

Ref on ComfyUI-Manager's log setup:
https://github.com/Comfy-Org/manager-core/blob/e9d5397aa15d013d0ad4f23de2e8eb86e1a60fac/prestartup_script.py#L150-L177

[huchenlei]

I add a print here for example: https://github.com/comfyanonymous/ComfyUI/blob/master/comfy/ops.py#L67

Context:

During the debug session, we observed if we have print statement printing stuffs while the progress bar is being actively updated, some logs are not properly output. They are flushed out only when pressing Ctrl+C.

[pythongosssss]

After a bunch of investigating, the fix here was to set the line_buffering to match the underlying stream

[juntaosun]

@pythongosssss

Hello, when I call POST /internal/logs/subscribe using the ComfyUI API, the server denies access and I get a cross-origin error.

has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

However, I have added the following startup parameters:

--enable-cors-header

❌ /internal/logs/subscribe POST failed!

    const payload = {
                enabled: true, 
                clientId: this.clientId, // clientId
            };

            // POST
            const response = await fetch(`http://127.0.0.1:8188/internal/logs/subscribe`, {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json',
                },
                body: JSON.stringify(payload),
            });

This means the startup parameters are valid, but /internal/logs/subscribe is not working.
Can you fix this?