Repository navigation
v0.6.0
Added
-
agents: a bundled
openai-embeddingsLLM config — OpenAI's
text-embedding-3-small(override withOPENAI_EMBEDDING_MODEL) with
OPENAI_API_KEY, for vector stores and file ingestion when no local
embedding server is running. Choose it in a store template or via
mindconnect.vector-store.embedding-config; the localembeddingsconfig
stays the default. -
agents: transcription over the REST API, as a job.
POST /api/transcriptionstakes a recording as multipart, stores it, queues
the work on the task queue and answers202with the job id plus both URLs
to follow it: one to poll, one to stream.GET /api/transcriptions/{id}
reports the status and, once done, the transcript with the detected language
and the recording's length;?wait=holds the request until the job ends,
for a caller who wants the synchronous feel without a loop.
GET /api/transcriptions/{id}/eventsis Server-Sent Events from the job's
channel — queued, running, the transcript as it forms indeltaevents,
then completed with the whole text — opening with the state as it is now,
so a late arrival is told what it missed, and?afterSeq=resumes a dropped
connection without a gap. The queue retries a provider hiccup, andmodel
names an LLM config whose alias is followed. The recording stays in the file
store, so a transcript keeps the audio behind it: its id and URL come back
with the job, andDELETE /api/files/{id}disposes of it when the caller
says so. Whether the text arrives word by word depends on the model —
OpenAI's transcribe models stream it,whisper-1answers in one piece and
sends one delta — and the events are the same either way, so a client does
not branch on the model. -
agents: speech to text — a Whisper model as a normal LLM config, and a
microphone in the chat. A config's type can now beSPEECH_TO_TEXTnext to
CHATandEMBEDDING, with the same provider, key, encryption and admin
form as every other model.LlmTranscriptionis the port callers use: they
name a config and get a transcript, so an alias is followed exactly as it is
for a chat model — pointspeech-to-textat whichever config should serve
and swap it later without touching a caller. Behind it,
OpenAiTranscriptionGatewaycalls the OpenAI-compatible
/v1/audio/transcriptionsendpoint, so OpenAI, Groq and a local Whisper
server all work by pointing the base URL at them; language, prompt and
response format are provider parameters on the config, and the LM Studio
model picker says it serves no transcription models rather than offering
chat ones.In the chat, the composer's new microphone records in the browser and puts
the transcript into the input, after anything already typed — nothing is
sent without the person who spoke it pressing Send. Dictating while a turn
is still streaming brings the words back as a sticky toast, since there is
no input to fill at that moment. The admin UI tests a
speech config the same way: drop an audio file into the test dialog, or
press Record, speak and press Stop. Both need a microphone and a
secure context (localhostor HTTPS); where the browser refuses, uploading
a file still works.A new installation is seeded with a
speech-to-textconfig (whisper-1
behind${OPENAI_API_KEY}, overridable throughSPEECH_TO_TEXT_MODELand
SPEECH_TO_TEXT_BASE_URL); an existing one gets it on the next start, since
seeding checks each entry by id.
Changed
- agents:
AgentLoop.runtakes the tokens spent by earlier attempts as a
further argument, andStreamEventhas a newTurnUsagevariant. Code
embeddingmc-agent-runtime-corethat callsrundirectly, or that
switches exhaustively overStreamEvent, needs the extra argument and the
extra arm.StreamEvent.Doneis unchanged.
Fixed
-
agents: the Responses API reports what a turn cost.
usageon
/v1/responseswas always zero: the runtime summed the providers' token
counts per round but dropped them at the turn boundary, so every response
claimed nought input and nought output tokens. The counts now ride with the
turn — across a suspend on a tool call and back — and reach the response
object and theresponse.completedframe. They arrive with the new
StreamEvent.TurnUsage, sent whenever the loop hands back control, so a
turn that fails or is cancelled reports what it burned instead of zero,
and the extra model call that answers after the round cap is counted too.
The same numbers ride themc-agent-api-restsession stream as a
turn_usageframe; a client talking to an older server sees none, which
reads as untracked rather than as a measured zero. Nothing on that API
carried a turn's cost before — the/memoryendpoint'stotalTokens
sizes the next prompt's context window, which is not what a turn spent,
and the per-call traces that do record it are rendered only in the admin
UI. -
agents: the Responses API stream announces and closes its content
parts. It sentoutput_item.added, the text deltas andoutput_item.done,
but none ofcontent_part.added,output_text.doneorcontent_part.done
— so a client had nothing telling it that the deltas and the text in the
finished item are one content part, and one that assumed otherwise rendered
the answer twice. A tool call now also gets the
function_call_arguments.donea client may wait for before dispatching, an
item opensin_progressand empty rather than already complete, and
sequence_numbercounts the frames actually written, so it no longer skips
the numbers of protocol events this layer does not send. -
agents: a Responses API stream no longer closes a content part it
never opened. When text reaches the runtime finished rather than as tokens
— a reviewer's replacement answer — the item was closed without ever being
announced, and a client resolving those frames against
response.output[output_index]aborted the stream on the missing entry
(the official OpenAI SDK raisesIndexError). Such an item is now
announced before it closes. -
agents: a Responses API stream can no longer start out of order. A
subscriber was registered while the backlog it had yet to receive was
handed over outside the lock, so an event emitted in between overtook it: a
reader could see a later delta before an earlier one, and a terminal event
that won the race closed the stream on a backlog never written. The backlog
is now delivered under the same lock that registers the subscriber. -
agents: two messages appended at the same moment no longer claim the
same place in a conversation. The sequence number was read and written in
two steps, so a turn finishing several tool calls together could hand the
same number to two messages and skip the next, which throws off anything
cutting by it: delete-from-here, regenerate, and the tool cards a turn is
folded into. Handing out the number is now the store's own doing, in one
step: Postgres reserves it in a counter row, the file and in-memory stores
take a lock per conversation.MessageRepositorygainsappendand loses
countByConversationId, whose only caller this was. -
agents: a chat longer than 200 messages no longer hides its newest
ones. A session's history was loaded as the first 200 messages, so once
a conversation passed that the chat kept showing its opening and silently
dropped everything after it, the message just sent included. It looked
like the message disappearing and the view jumping back a turn, while the
agent went on answering it, because the runtime had always read the
conversation entire. The history now comes back entire too, in the chat,
the admin UI's chat, the attached-files list and regenerate. -
agents: editing the built-in
defaultvector-store template in the Admin
UI no longer pretends to save. The built-in template is the
mindconnect.vector-store.*properties; its edit form now says so and saves
the values as a copy under a name of your own, and saving or deleting it
under its own name reports why nothing happened instead of staying silent.
What's Changed
- A long chat shows its newest messages, and two appends get two numbers by @beisdog in #50
- An OpenAI embedding config, and a template form that admits what it cannot save by @beisdog in #51
- Fix token streaming and accounting on the Responses API by @beisdog in #52
- Speech to text: a Whisper config, and a microphone in the chat by @beisdog in #53
Full Changelog: v0.5.2...v0.6.0