Skip to content

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 10 Sep 02:18
· 477 commits to main since this release

Added

  • agents: a bundled openai-embeddings LLM config — OpenAI's
    text-embedding-3-small (override with OPENAI_EMBEDDING_MODEL) with
    OPENAI_API_KEY, for vector stores and file ingestion when no local
    embedding server is running. Choose it in a store template or via
    mindconnect.vector-store.embedding-config; the local embeddings config
    stays the default.

  • agents: transcription over the REST API, as a job.
    POST /api/transcriptions takes a recording as multipart, stores it, queues
    the work on the task queue and answers 202 with the job id plus both URLs
    to follow it: one to poll, one to stream. GET /api/transcriptions/{id}
    reports the status and, once done, the transcript with the detected language
    and the recording's length; ?wait= holds the request until the job ends,
    for a caller who wants the synchronous feel without a loop.
    GET /api/transcriptions/{id}/events is Server-Sent Events from the job's
    channel — queued, running, the transcript as it forms in delta events,
    then completed with the whole text — opening with the state as it is now,
    so a late arrival is told what it missed, and ?afterSeq= resumes a dropped
    connection without a gap. The queue retries a provider hiccup, and model
    names an LLM config whose alias is followed. The recording stays in the file
    store, so a transcript keeps the audio behind it: its id and URL come back
    with the job, and DELETE /api/files/{id} disposes of it when the caller
    says so. Whether the text arrives word by word depends on the model —
    OpenAI's transcribe models stream it, whisper-1 answers in one piece and
    sends one delta — and the events are the same either way, so a client does
    not branch on the model.

  • agents: speech to text — a Whisper model as a normal LLM config, and a
    microphone in the chat. A config's type can now be SPEECH_TO_TEXT next to
    CHAT and EMBEDDING, with the same provider, key, encryption and admin
    form as every other model. LlmTranscription is the port callers use: they
    name a config and get a transcript, so an alias is followed exactly as it is
    for a chat model — point speech-to-text at whichever config should serve
    and swap it later without touching a caller. Behind it,
    OpenAiTranscriptionGateway calls the OpenAI-compatible
    /v1/audio/transcriptions endpoint, so OpenAI, Groq and a local Whisper
    server all work by pointing the base URL at them; language, prompt and
    response format are provider parameters on the config, and the LM Studio
    model picker says it serves no transcription models rather than offering
    chat ones.

    In the chat, the composer's new microphone records in the browser and puts
    the transcript into the input, after anything already typed — nothing is
    sent without the person who spoke it pressing Send. Dictating while a turn
    is still streaming brings the words back as a sticky toast, since there is
    no input to fill at that moment. The admin UI tests a
    speech config the same way: drop an audio file into the test dialog, or
    press Record, speak and press Stop. Both need a microphone and a
    secure context (localhost or HTTPS); where the browser refuses, uploading
    a file still works.

    A new installation is seeded with a speech-to-text config (whisper-1
    behind ${OPENAI_API_KEY}, overridable through SPEECH_TO_TEXT_MODEL and
    SPEECH_TO_TEXT_BASE_URL); an existing one gets it on the next start, since
    seeding checks each entry by id.

Changed

  • agents: AgentLoop.run takes the tokens spent by earlier attempts as a
    further argument, and StreamEvent has a new TurnUsage variant. Code
    embedding mc-agent-runtime-core that calls run directly, or that
    switches exhaustively over StreamEvent, needs the extra argument and the
    extra arm. StreamEvent.Done is unchanged.

Fixed

  • agents: the Responses API reports what a turn cost. usage on
    /v1/responses was always zero: the runtime summed the providers' token
    counts per round but dropped them at the turn boundary, so every response
    claimed nought input and nought output tokens. The counts now ride with the
    turn — across a suspend on a tool call and back — and reach the response
    object and the response.completed frame. They arrive with the new
    StreamEvent.TurnUsage, sent whenever the loop hands back control, so a
    turn that fails or is cancelled reports what it burned instead of zero,
    and the extra model call that answers after the round cap is counted too.
    The same numbers ride the mc-agent-api-rest session stream as a
    turn_usage frame; a client talking to an older server sees none, which
    reads as untracked rather than as a measured zero. Nothing on that API
    carried a turn's cost before — the /memory endpoint's totalTokens
    sizes the next prompt's context window, which is not what a turn spent,
    and the per-call traces that do record it are rendered only in the admin
    UI.

  • agents: the Responses API stream announces and closes its content
    parts. It sent output_item.added, the text deltas and output_item.done,
    but none of content_part.added, output_text.done or content_part.done
    — so a client had nothing telling it that the deltas and the text in the
    finished item are one content part, and one that assumed otherwise rendered
    the answer twice. A tool call now also gets the
    function_call_arguments.done a client may wait for before dispatching, an
    item opens in_progress and empty rather than already complete, and
    sequence_number counts the frames actually written, so it no longer skips
    the numbers of protocol events this layer does not send.

  • agents: a Responses API stream no longer closes a content part it
    never opened. When text reaches the runtime finished rather than as tokens
    — a reviewer's replacement answer — the item was closed without ever being
    announced, and a client resolving those frames against
    response.output[output_index] aborted the stream on the missing entry
    (the official OpenAI SDK raises IndexError). Such an item is now
    announced before it closes.

  • agents: a Responses API stream can no longer start out of order. A
    subscriber was registered while the backlog it had yet to receive was
    handed over outside the lock, so an event emitted in between overtook it: a
    reader could see a later delta before an earlier one, and a terminal event
    that won the race closed the stream on a backlog never written. The backlog
    is now delivered under the same lock that registers the subscriber.

  • agents: two messages appended at the same moment no longer claim the
    same place in a conversation. The sequence number was read and written in
    two steps, so a turn finishing several tool calls together could hand the
    same number to two messages and skip the next, which throws off anything
    cutting by it: delete-from-here, regenerate, and the tool cards a turn is
    folded into. Handing out the number is now the store's own doing, in one
    step: Postgres reserves it in a counter row, the file and in-memory stores
    take a lock per conversation. MessageRepository gains append and loses
    countByConversationId, whose only caller this was.

  • agents: a chat longer than 200 messages no longer hides its newest
    ones. A session's history was loaded as the first 200 messages, so once
    a conversation passed that the chat kept showing its opening and silently
    dropped everything after it, the message just sent included. It looked
    like the message disappearing and the view jumping back a turn, while the
    agent went on answering it, because the runtime had always read the
    conversation entire. The history now comes back entire too, in the chat,
    the admin UI's chat, the attached-files list and regenerate.

  • agents: editing the built-in default vector-store template in the Admin
    UI no longer pretends to save. The built-in template is the
    mindconnect.vector-store.* properties; its edit form now says so and saves
    the values as a copy under a name of your own, and saving or deleting it
    under its own name reports why nothing happened instead of staying silent.

What's Changed

  • A long chat shows its newest messages, and two appends get two numbers by @beisdog in #50
  • An OpenAI embedding config, and a template form that admits what it cannot save by @beisdog in #51
  • Fix token streaming and accounting on the Responses API by @beisdog in #52
  • Speech to text: a Whisper config, and a microphone in the chat by @beisdog in #53

Full Changelog: v0.5.2...v0.6.0