Skip to content

v0.5.0

Choose a tag to compare

@saadahmed95 saadahmed95 released this 04 Oct 21:44
b97e7b1

Session resume, a cancellation chain from the socket through the provider to every tool call, tools that run on Vertex, and failing tool calls answered to the model. One thing is removed and several behaviours an existing consumer observes change: see Upgrading from 0.4.x below. Every release up to and including 0.4.0 has two defects that end the process (see Security and Fixed), so upgrade even if you use none of the new features.

Upgrading from 0.4.x

  • Every client must offer hal.v1 beside its token, new WebSocket(url, ['hal.v1', token]). A bare-token offer, accepted with a deprecation warning since 0.3.0, now fails to connect, and credentialFromSubprotocol returns undefined for it.
  • Closing a socket abandons its turn. The run stops reading from the model and abandons its tool calls, no stream_end follows the close, and afterModelResponse and onError do not fire for it, while afterSession still does. A missed heartbeat and stop() close sockets too, so drain connections before a deploy.
  • Every tool call now has 30 seconds to settle, a beforeToolCall policy included. A tool that legitimately runs longer needs a larger toolTimeoutMs, or 0 for the old unbounded wait.
  • ToolRegistry.execute no longer throws for a tool name the model invented, or rejects when an executor throws: the model is answered instead. It throws a TypeError for an input that is not an object, whatever the tool's schema.
  • Messages in one session are answered one at a time, so a message sent while another is streaming starts after that one's stream_end.
  • Tools now run on Vertex, where a function-calling turn ends tool_use instead of end_turn. If you read stopReason yourself, expect the new value.
  • Two new error log lines: authenticator failed for a WsAuthenticator that throws or rejects, and tool executor threw for a tool executor that does.
  • Resume is opt-in and changes nothing until you enable it. With it on, do not erase a conversation in onDisconnect: the socket a resume replaced can report onDisconnect after the resume.

Added

  • Session resume, opt-in via transport: {resume: {enabled: true}}. A client that reconnects with ?sessionId=<id> gets its conversation back, but only the session's own user; any other id gets a fresh session, indistinguishable from one that never existed.
  • Continue the latest session with resume: {enabled: true, latest: true}: a refresh or a new tab rejoins the user's most recently active conversation, and ?new=1 starts a new one. Stores supply it through the optional SessionStore.latestFor.
  • An expired session told apart from a missing one, through the optional SessionStore.lookup and resumeFailure on the connected frame, which reads expired only for the session's owner.
  • Cancel a model call with SendMessageParams.signal. Bedrock closes the request; @google-cloud/vertexai cannot cancel, so on Vertex the engine stops reading while the vendor request runs on.
  • A deadline on every tool call, toolTimeoutMs (default 30000, 0 waits forever), and a ToolContext.signal aborted at the deadline with a TimeoutError, so a tool that passes it to fetch stops its own request too.
  • Stored entries a tool suppressed carry suppressed: true, so a resumed session never shows text a tool hid.

Removed

  • The bare-token subprotocol offer. It put the access token in the 101 response headers; clients have offered ['hal.v1', token] since 0.3.0.

Changed

  • Closing a socket abandons its turn, as above. The session keeps the answer as it stood, committed and flagged truncated, so a resumed session shows it cut off. processMessageStream takes an optional {signal} that does the same for a caller of its own.
  • credentialFromSubprotocol returns undefined unless the offer includes hal.v1.
  • A failing WsAuthenticator is logged at error, with its error type only, never its message.

Fixed

  • Messages in one session are answered one at a time, in the order they arrived, instead of interleaving two runs.
  • A Vertex stream that failed mid-way ended the process, through an unobserved promise inside @google-cloud/vertexai. Every release up to and including 0.4.0 is affected.
  • No tool ever ran on Vertex: the provider reported a function-calling turn as end_turn, so the call was dropped. Every release up to and including 0.4.0 is affected.
  • A failing tool call no longer fails the turn. An invented tool name or a throwing executor is answered to the model, and the round's other results survive. The thrown message is logged, cut to 500 characters, and never shown to the model; a throw after the user left is logged at info as tool call abandoned.
  • An answer still streaming when a tool suppresses the response is committed and retracted like one already sent.

Security

  • One malformed WebSocket upgrade could end the process. An empty or missing Host header, or a request-target such as //, threw where nothing catches it, so a single request with no credentials ended the process and every conversation on it. It is now answered 400 Bad Request. Every release up to and including 0.4.0 is affected.

Full detail is in CHANGELOG.md.