You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Attachments by reference: an attachment with ref: (the host's own
id for the file, a material id say) is stored in the session as its ref
and metadata, never its bytes (no image part; the ref in the envelope and
in the message's attachments; path is then optional). Each model
call fetches a referenced picture (when the model takes pictures) or PDF
(when it takes PDFs) through the new :resolve_attachment hook
(fn attachment, ctx -> {:ok, {mime, bytes}} | {:error, reason} end or
an MFA; ctx is %{session_id: id}), once per kind and ref, in the call's own
process, not the loop's, and sends it where an inline one would go; with
no hook, an error, a raise or a bad answer the model gets "(
couldn't be loaded, so it isn't sent.)" instead. A tool can answer with
pictures by reference, %{ref:, mime:, name:} in its :images, stored
as the tool result's attachments (one answer's pictures are all inline
or all by reference). Rho.Session.attachments/1 lists a user message's
or tool result's files (kind, name, mimeType, and ref / path / about / pages when present) for the host's chat. Sessions with inline
pictures are read as before, and pictures the model makes stay inline.
Rho.Session.fetch/2: Rho.Session.messages/2 leaves placeholders for
the files whose bytes aren't in the session (pictures by reference, PDFs)
with fetch: false, to be fetched later in another process; without it
(the default) it fetches them before returning, given session_id:.
Changed
Attached PDFs read from their paths are read by the model call's
process, not the loop's, and each attachment counts against the
100-page / 20 MB budget (one attached twice used to go twice on one
place in it). Rho.Loop.context/1 fetches in the caller's process.
Rho.Loop.prompt/3, steer/3 and follow_up/3 raise ArgumentError
in the caller for an attachment with neither :path nor :ref.