Description
The prompt_async handler forks the entire prompt operation, including user-message persistence, and immediately returns HTTP 204. A client can therefore observe a successful response before the prompt has been durably admitted. If the process or request scope ends in that window, the caller has no way to distinguish an accepted prompt from one that never reached storage.
Expected behavior
HTTP 204 should mean the user message has been persisted. Model execution can remain asynchronous.
Suggested change
Separate prompt admission from loop execution: await admission in the handler, then fork only the session loop when noReply is not set. A regression test can call promptAsync with noReply: true and immediately verify the message is present.
Description
The
prompt_asynchandler forks the entire prompt operation, including user-message persistence, and immediately returns HTTP 204. A client can therefore observe a successful response before the prompt has been durably admitted. If the process or request scope ends in that window, the caller has no way to distinguish an accepted prompt from one that never reached storage.Expected behavior
HTTP 204 should mean the user message has been persisted. Model execution can remain asynchronous.
Suggested change
Separate prompt admission from loop execution: await admission in the handler, then fork only the session loop when
noReplyis not set. A regression test can callpromptAsyncwithnoReply: trueand immediately verify the message is present.