Repository navigation
v0.5.0
Session resume, a cancellation chain from the socket through the provider to every tool call, tools that run on Vertex, and failing tool calls answered to the model. One thing is removed and several behaviours an existing consumer observes change: see Upgrading from 0.4.x below. Every release up to and including 0.4.0 has two defects that end the process (see Security and Fixed), so upgrade even if you use none of the new features.
Upgrading from 0.4.x
- Every client must offer
hal.v1beside its token,new WebSocket(url, ['hal.v1', token]). A bare-token offer, accepted with a deprecation warning since 0.3.0, now fails to connect, andcredentialFromSubprotocolreturnsundefinedfor it. - Closing a socket abandons its turn. The run stops reading from the model and abandons its tool calls, no
stream_endfollows the close, andafterModelResponseandonErrordo not fire for it, whileafterSessionstill does. A missed heartbeat andstop()close sockets too, so drain connections before a deploy. - Every tool call now has 30 seconds to settle, a
beforeToolCallpolicy included. A tool that legitimately runs longer needs a largertoolTimeoutMs, or0for the old unbounded wait. ToolRegistry.executeno longer throws for a tool name the model invented, or rejects when an executor throws: the model is answered instead. It throws aTypeErrorfor an input that is not an object, whatever the tool's schema.- Messages in one session are answered one at a time, so a message sent while another is streaming starts after that one's
stream_end. - Tools now run on Vertex, where a function-calling turn ends
tool_useinstead ofend_turn. If you readstopReasonyourself, expect the new value. - Two new
errorlog lines:authenticator failedfor aWsAuthenticatorthat throws or rejects, andtool executor threwfor a tool executor that does. - Resume is opt-in and changes nothing until you enable it. With it on, do not erase a conversation in
onDisconnect: the socket a resume replaced can reportonDisconnectafter the resume.
Added
- Session resume, opt-in via
transport: {resume: {enabled: true}}. A client that reconnects with?sessionId=<id>gets its conversation back, but only the session's own user; any other id gets a fresh session, indistinguishable from one that never existed. - Continue the latest session with
resume: {enabled: true, latest: true}: a refresh or a new tab rejoins the user's most recently active conversation, and?new=1starts a new one. Stores supply it through the optionalSessionStore.latestFor. - An expired session told apart from a missing one, through the optional
SessionStore.lookupandresumeFailureon theconnectedframe, which readsexpiredonly for the session's owner. - Cancel a model call with
SendMessageParams.signal. Bedrock closes the request;@google-cloud/vertexaicannot cancel, so on Vertex the engine stops reading while the vendor request runs on. - A deadline on every tool call,
toolTimeoutMs(default30000,0waits forever), and aToolContext.signalaborted at the deadline with aTimeoutError, so a tool that passes it tofetchstops its own request too. - Stored entries a tool suppressed carry
suppressed: true, so a resumed session never shows text a tool hid.
Removed
- The bare-token subprotocol offer. It put the access token in the 101 response headers; clients have offered
['hal.v1', token]since 0.3.0.
Changed
- Closing a socket abandons its turn, as above. The session keeps the answer as it stood, committed and flagged
truncated, so a resumed session shows it cut off.processMessageStreamtakes an optional{signal}that does the same for a caller of its own. credentialFromSubprotocolreturnsundefinedunless the offer includeshal.v1.- A failing
WsAuthenticatoris logged aterror, with its error type only, never its message.
Fixed
- Messages in one session are answered one at a time, in the order they arrived, instead of interleaving two runs.
- A Vertex stream that failed mid-way ended the process, through an unobserved promise inside
@google-cloud/vertexai. Every release up to and including 0.4.0 is affected. - No tool ever ran on Vertex: the provider reported a function-calling turn as
end_turn, so the call was dropped. Every release up to and including 0.4.0 is affected. - A failing tool call no longer fails the turn. An invented tool name or a throwing executor is answered to the model, and the round's other results survive. The thrown message is logged, cut to 500 characters, and never shown to the model; a throw after the user left is logged at
infoastool call abandoned. - An answer still streaming when a tool suppresses the response is committed and retracted like one already sent.
Security
- One malformed WebSocket upgrade could end the process. An empty or missing
Hostheader, or a request-target such as//, threw where nothing catches it, so a single request with no credentials ended the process and every conversation on it. It is now answered400 Bad Request. Every release up to and including 0.4.0 is affected.
Full detail is in CHANGELOG.md.