Skip to content

Releases: Particle-Academy/prism-harness

v0.11.1

Choose a tag to compare

@github-actions github-actions released this 14 Sep 17:37

v0.11.1 - An empty provider state is stored as a map

An assistant turn with no provider state was stored with "additional_content": [], an empty list. The TypeScript and Python harnesses store {}. A typed reader of a thread shared between them expects a map and refuses the list. It is now stored as {}.

Rows already stored read exactly as before.

prism-parity's new harness-thread-rows corpus found it. The corpus pins the rows a run stores and replays in all three languages, and is now part of this package's tests.

v0.11.0

Choose a tag to compare

@github-actions github-actions released this 14 Sep 17:18

v0.11.0 - Approvals work through a real provider handler

Requires particle-academy/prism 0.123.0, which fixes the approval resume in Prism and GHSA-jc3v-v77r-9p38 (an approved call ran again on later requests).

What was broken

Every approval test used Prism::fake(). Through a real provider handler:

  • a run that stopped on a tool needing approval reported awaitingApproval() as false, so if ($response->awaitingApproval()) never ran;
  • the thread did not store the approval request, or the results of the other tools in that step;
  • approve() did not run the approved tool;
  • resuming recorded the whole conversation again.

Nothing ran without approval. The approval step itself did not work.

Fixed

  • awaitingApproval() is true for a run that finishes on tool calls with approvals pending. A real handler finishes that way. Only a fake reported Pause.
  • What a turn adds is recorded once, also when Prism rewrites the tool result messages on resume.
  • Consecutive tool result rows are replayed as one message. Each tool output reaches the provider once, and Anthropic never gets an empty user turn for a decision-only row.
  • An approved call runs once, and does not run again on later turns.

New: decide()

approve() and deny() each continue the run. Prism refuses any call that has no answer yet. So with several calls pending, answering them one at a time refuses the rest. Answer them together:

if ($response->awaitingApproval()) {
    $session->decide(array_map(
        fn (ToolApprovalRequest $pending) => new ToolApprovalResponse($pending->approvalId, approved: true),
        $response->pendingApprovals(),
    ));
}

The README's earlier example approved in a loop, which refused every call after the first. It also called ->first() on pendingApprovals(), which returns an array.

Upgrading

Threads keep their rows. Rows recorded by earlier versions replay as before.

v0.10.0

Choose a tag to compare

@github-actions github-actions released this 14 Sep 07:47

v0.10.0 - Turns carry attachments; modes set provider options

Attachments on a turn

send() and stream() take an optional trailing list of media to send with the prompt:

$session->send('What is wrong with this layout?', null, [
    Image::fromBase64($request->string('screenshot'), 'image/png'),
]);

An attachment is stored with the turn and replayed on later turns. It must be an Image, Document, Audio or Video that carries its bytes, a provider file id, or document chunks.

Refused before a run starts, with an UnacceptableAttachment that carries a code:

Code When
attachment_by_reference built from a URL, or from a local or storage path. This includes a URL whose content was already fetched.
attachment_not_media anything other than those four media types
attachment_empty no bytes (including fromBase64('')), an empty file id, or an empty chunk list
attachment_without_prompt attachments with an empty prompt, which would otherwise vanish without an error

The TypeScript and Python harness ports apply the same rules with the same codes. prism-parity's harness-turn-attachments corpus pins that across all three.

Provider options per mode

A mode can declare provider_options. They are passed to Prism's withProviderOptions() unchanged on every run in that mode, sent or streamed:

'overseer' => [
    'provider_options' => ['thinking' => ['type' => 'adaptive'], 'effort' => 'medium'],
],

That is Anthropic's adaptive thinking, which current Claude models require. They refuse ['thinking' => ['enabled' => true, 'budgetTokens' => 4000]] with a 400; that shape is for older models only. (An earlier version of these notes showed the older shape.)

A value that is not a map is refused when the mode resolves. Extended thinking survives a stored thread under either shape: the signature Anthropic requires on a later tool-use turn is stored and sent again. That is tested against the real Anthropic handler.

Compatibility

Every new argument is optional and comes last, so existing calls are unaffected. AgentMode gains a trailing $providerOptions constructor argument, which defaults to [].

v0.9.1

Choose a tag to compare

@github-actions github-actions released this 14 Sep 07:20

v0.9.1 - Threads record each turn once

Fixed: send() copied the whole conversation into the thread on every turn

Session::send() (and so AgentRuntime::send()) recorded the provider response's full message list. With a real provider, that list starts with the entire history the request carried, so every turn appended the whole conversation again before the new messages. The thread doubled with each turn.

What that did:

  • Storage: a conversation of n turns stored about 2^(n+1) rows instead of 2n. One thread in our own lab reached 2,046 rows holding 20 distinct messages after 10 turns.
  • The model: the next turn replays whatever is stored. Unless a context window (keep_recent or your own strategy) was bounding it, the model saw the conversation repeated many times over, and paid for it.

It has been present since v0.2.0. stream() was never affected, and a thread whose runs all used stream() is correct.

The test suite didn't catch it because Prism::fake() returns only the messages a test supplies. The new tests drive Prism's real OpenAI handler over a faked HTTP transport instead.

Threads you already have

This release stops the growth. It does not repair rows already written.

To find an affected thread, compare its row count with its distinct payloads:

select t.scope, count(m.id) as rows, count(distinct m.payload) as distinct_payloads
from harness_threads t
join harness_thread_messages m on m.thread_id = t.id
group by t.id
having count(m.id) > count(distinct m.payload)
order by rows desc;

Some repetition is legitimate. A user can really send "ok" twice. So a gap between the two numbers is a signal, not proof.

An affected thread's rows for a run begin with a copy of the rows before that run. If you need to repair threads, raise an issue: a repair has to be careful about legitimate repeats, so it is not shipped blind.

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 14 Sep 03:52

v0.9.0 - Threads replay their attachments faithfully, on prism v0.120.0

Requires particle-academy/prism v0.120.0

The minimum prism is now v0.120.0. From that release, prism stores media with its bytes, a kind, and no file paths. On an older prism, an attachment saved before its message was sent stored no bytes at all, and a thread holding it could not be replayed. This package can't put those bytes back.

Fixed: four ways a replayed attachment came back wrong

  • A url or local-file document came back titled with its mime type, and lost its real title. Document's factories take the title where Media's take the mime type, and the rebuild called them the Media way.
  • A text document lost its title.
  • A chunked document could not be replayed at all. It has no file id, url, path or bytes, so replay threw UnmappableContent.
  • A filename set with as() was dropped.

Rows you already have

Thread rows written before this release keep working. A row that carries a local_path or storage_path and no bytes still replays from that path, as before. New rows carry the bytes instead and replay without touching the filesystem.

The thread table is still trusted storage: an older row's path is still read as a file on replay. Don't let request input write to it directly.

v0.8.2

Choose a tag to compare

@github-actions github-actions released this 10 Sep 18:45

Correct what refusing a referenced Audio actually stops

Docs and tests only. No behaviour change, and nothing to do on upgrade.

v0.8.1 described a request-derived Audio::fromLocalPath() as "arbitrary file
read" that the harness refuses. The refusal is right. That description of it
was not, and an overstated security claim is worse than no claim, because an
operator can build on it.

MEASURED, NOT REASONED ABOUT

fromLocalPath() and fromStoragePath() read the file INSIDE the constructor,
in the host's own code, before the harness is ever called. Construct one,
delete the file, and the bytes are still on the value object. The read cannot
be prevented from here and v0.8.2 no longer says it can.

What refusing a path DOES stop is the step that turns a read into a breach: the
bytes being uploaded to a third-party transcription provider, and the file
coming back to the caller as text. Worth having. Not the same claim.

A URL is genuinely different, and the docs now separate the two. fromUrl() is
lazy — nothing is fetched until the request is built — so refusing it means the
request is never made. That one is SSRF prevention in the ordinary sense.

A TEST THAT WOULD HAVE GONE RED ON A DEPENDENCY BUMP

v0.8.1 pinned that Prism's hasBase64() returns true for a URL — the trap this
guard was written around. That was true when written and is no longer: it was
reported and fixed upstream (Particle-Academy/prism#40, where the reference
turned out to be the outlier; both ports already had it right).

This package supports a RANGE of Prism versions, so an assertion about a
dependency's internals makes the suite green or red depending on which
resolution CI picked, while the security property is identical either way. It
now pins the property instead: the refusal does not depend on that predicate
and stays correct whichever answer a resolved Prism gives.

The guard itself still uses isUrl()/isFile() rather than the now-honest
hasBase64(). A security guard should not silently change meaning with the
resolved version of a dependency.

A new test pins the eager read, so if Prism ever makes it lazy the suite fails
and these docs get to be stronger.

The v0.8.1 release notes have been amended in place with the same correction.

v0.8.1

Choose a tag to compare

@github-actions github-actions released this 10 Sep 18:24

Voice: referenced audio is refused, and the failure path is measured

Two findings from the v0.8.0 pre-publish audit. One had a fix; the other
turned out not to, and saying so precisely is the other half of this release.

BREAKING, narrowly: VoiceExchange now REFUSES an Audio it would have to
dereference

transcribe() and exchange() throw Prism\Harness\Exceptions\UnsafeAudioSource
(code unsafe_audio_source) when handed audio built with fromUrl(),
fromLocalPath() or fromStoragePath(). Inline audio — fromBase64(), which is
what a browser microphone produces — is unaffected, so an application that
takes a recording from the client needs no change.

The safe construction and the dangerous one are one method name apart, and
nothing at the call site made anyone pause.

CORRECTION, made in v0.8.2: this section originally said a request-derived
fromLocalPath() was "arbitrary file read". That is wrong about the mechanism.
fromLocalPath() and fromStoragePath() read the file INSIDE the constructor,
in the host's own code, before the harness is ever called — measured by
constructing one, deleting the file, and finding the bytes still there. The
read cannot be prevented from here.

What refusing a path stops is the step that turns a read into a breach: the
bytes being uploaded to a third-party transcription provider, and the file
coming back to the caller as text. Worth having, and not what this said.

A URL is genuinely different. fromUrl() is lazy, so nothing is fetched until
the request is built and the refusal means the request is never made. That one
is SSRF prevention in the ordinary sense.

If you transcribe recordings your own application wrote, that is legitimate
and stays available:

new VoiceExchange(allowReferencedAudio: true);

The flag is an assertion about where the audio came from, which only you can
make, so it is off by default.

If you write a provenance guard of your own: do not use Prism's hasBase64()

It delegates to hasRawContent() and returns TRUE for a URL nobody has fetched
and a path nobody has read — it answers "can bytes be obtained", not "are
bytes in hand". A guard written with it admits every case it was meant to
refuse. The first draft of this one did. Use isUrl() and isFile(). Filed
upstream as Particle-Academy/prism#40.

Documented, not fixed: a failed voice call can put the recording in a log

With zend.exception_ignore_args=0, PHP records frame arguments, and an error
reporter that walks them (Flare and Sentry both do, by reflection) can capture
the audio from a failed turn's stack trace.

The package cannot close this, and that was measured rather than assumed.
Prism's own frames are clean — speech-to-text attaches a stream, not a base64
string. The frames that hold the recording are VoiceExchange's own, because a
method taking an Audio has the Audio in its arguments; and rethrowing a
tidier exception does not help, because the replacement is constructed inside
that same frame. Tests now pin all of it, so a future Prism that carries the
payload deeper turns the suite red instead of the documentation quietly wrong.

Two remedies work, and both are the operator's:

  • zend.exception_ignore_args=1 strips frame arguments. php.ini-production
    ships this; a PHP with no ini file at all does NOT, so "we never changed it"
    is not the safe answer.
  • Scrub Prism\Prism\ValueObjects\Media\Audio in your error reporter. It also
    reaches the protected rawContent, not just the public base64.

SummarisingCompaction carries the same note — a conversation transcript is in
scope there for the same reason.

Also

Voice shipped in v0.8.0 with no README section at all. It has one now, which
is where a security default belongs.

Closes #11, closes #12.

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 10 Sep 02:40

v0.8.0 — a scope can hold more than one conversation, and a turn can be spoken

Two things every application building a chat on this package runs into
immediately, and neither was possible.

A thread is addressed by participant and scope and resolved with
firstOrCreate, so a scope held exactly ONE conversation for ever. An
application offering "new chat" had no way to provide it except by minting a
scope per conversation — which defeats the addressing that lets a restarted
worker find the same session again.

$session->newConversation();   // the whole API

retired_at is the seam. A retired thread stops being the one a session
resolves; the next resolve creates a fresh one at the same address.

Nothing is deleted, and that is the point rather than an implementation
detail.
"Clear my context" and "erase my history" are different requests that
one button is very often asked to mean at once, and only one of them is
recoverable if the user meant the other. The retired thread keeps every message,
still readable, still addressable by id — the same rule compaction follows,
where the view changes and the storage does not.

Retirement is idempotent and per-address: retiring one scope's conversation
leaves every other scope, and every other participant, untouched.

Prism\Harness\Voice\VoiceExchange — transcribe an utterance, run it as an
ordinary turn, speak the answer back.

$reply = (new VoiceExchange)->exchange($session, $utterance);
$reply->heard;   // what the transcriber heard
$reply->text;    // the agent's answer
$reply->audio;   // the answer as speech, or null

Prism already did both directionsPrism::audio()->withInput($audio) ->asText() and ->withVoice(...)->asAudio(), across OpenAI, ElevenLabs,
Gemini, Groq, Mistral and Replicate — and its test fake already covered
speechToText and textToSpeech. Nothing needed adding there and nothing was;
that was checked rather than assumed.

What the harness could not do was hold a spoken conversation, because
Session::send() takes a string. So every application wanting voice wrote the
same glue, and each invented its own answer to the question of what gets stored.

The thread stores TEXT. A transcript is what replays to a model, what a human
reads back, and what compaction and recall operate on. Audio in the message
table would be unreplayable by anything but the original provider, and would put
minutes of PCM in a table designed for messages.

Two decisions a caller would otherwise get wrong, both tested:

  • An empty transcript is not a turn. Silence, a mis-fired button, a dead
    microphone — sending "" would record an empty user message in the thread for
    ever and bill a turn answering nothing. Reported as empty so the caller can
    say "I didn't catch that".
  • A turn that produced no prose is not synthesised. A tool-only turn
    legitimately says nothing, and asking a TTS provider to read an empty string
    is a billed request for a silent file.

heard is carried separately from text because the two fail differently: a
wrong answer to the right transcription is a model problem, a right answer to
the wrong transcription is a microphone problem, and only showing both lets
anyone tell which just happened.

Turn-based, not a live duplex stream. Press-to-talk: one utterance in, one
answer out. A continuously open bidirectional socket with barge-in is a
different product, and letting a caller discover that from latency rather than
from the type would be the more expensive mistake.

The thread does not record that a turn was spoken. Thread::record() has no
per-message provenance, so a transcription is indistinguishable from something
typed once it is stored — which matters, because a misheard word reads as a user
who said something odd. The honest fix is a provenance field on the message
rather than something bolted onto the voice layer. Until then VoiceReply::heard
carries it for the length of the turn, and an application that needs it durably
must store it itself.

One migration, adding a nullable retired_at and an index. Existing threads have
no retirement and resolve exactly as before, so nothing changes until an
application calls newConversation().

257 tests, phpstan clean.

v0.7.1

Choose a tag to compare

@github-actions github-actions released this 09 Sep 16:16

v0.7.1 — the release v0.7.0 never got

Same feature as v0.7.0, re-cut so it has a release page. Nothing in the package
changed between them except this repo's AGENTS.md gaining the release procedure.

WHY v0.7.0 HAS NO NOTES, since that is the reason this tag exists. The release
workflow refuses to publish a tag whose tests have not already succeeded for
that exact commit — a good guard. It also means pushing main and the tag in the
same breath fails, because the tests for that SHA are still queued when the tag
lands. v0.6.0, v0.6.1 and v0.7.0 were all cut that way. All three release runs
failed; nobody noticed, because pushing a tag looks like success and Composer
resolves from tags, so the package installed fine while the releases page went
on showing v0.5.0 as Latest. A consumer had to diff the source tree to find out
what changed.

v0.6.0 and v0.6.1 were recovered by re-running their release jobs once tests had
gone green. v0.7.0 could not be: a NIGHTLY strict factcheck run reddened its
commit hours after tagging — on ecosystem-wide claim staleness, nothing to do
with the package — and the guard counts any historical failure on the SHA.
Deleting that run to get past the gate would destroy the evidence, so the tag
keeps its history and the release moves here.

What the feature is

Prism\Harness\Contracts\SummaryBudgethow the summary word budget is
enforced is the application's choice, not this package's.
Two dials,
deliberately separate: HOW BIG (summary_words) and HOW ENFORCED (the
contract).

budget what it does costs
RetryOnce (default) counts, and asks once more when over a second call on turns that overshoot
AskOnly nothing — the behaviour before v0.6.0 one call, and the summary may be 4x what you asked
TruncateTo guarantees the bound by cutting free, and can hand the model a fragment that reads whole
$this->app->bind(SummaryBudget::class, fn () => new TruncateTo);
  • RetryOnce is the default and is the v0.6.0 behaviour, so upgrading
    changes nothing. It took a 60-word budget from 205 words to 61. It is allowed
    to miss, and does — measured arms still finished at 118, 102 and 114 against a
    budget of 60.
  • AskOnly is kept as a NAMED choice rather than deleted. One model call per
    compacting turn is a legitimate policy when cost is the binding constraint.
  • TruncateTo prefers a sentence boundary inside the budget and marks the
    cut when there is none, because "the customer agreed to the refund provided"
    is not merely shorter than the truth — it is a different claim. Bind an
    EvictionSink alongside it; the cut text is unrecoverable from the summary by
    construction.

Resolved from the container only when the application bound one, so the
constructor default stays reachable rather than duplicated where it would drift.

And the strategy no longer trusts the contract

A budget is application code. One returning an empty string would evict the
older turns and leave an empty marker in their place — history gone with nothing
standing in for it, the worst outcome this class has. An empty result is now
treated exactly like a failed model call, so a third-party budget cannot do more
damage than the provider being down. Found by writing the test for it.

Why any of this exists

The budget was never enforced before v0.6.0. summary_words reached the model
as "in at most N words" inside a prompt — a request, not a bound — and nothing
looked at the answer. Measured live from prism-labs:

stated budget came back at over
15 92 6.1x
15 346 23.1x
60 (the default) 205 3.4x

The 346-word summary had re-stated every exchange in the conversation one by
one, which is exactly the unbounded growth this strategy rewrites rather than
appends in order to avoid. Requirement 2 of the short-term-memory design — the
summary keeps compacting, bounded rather than growing — was not met, and it was
invisible because the summaries read plausibly.

v0.6.1

Choose a tag to compare

@github-actions github-actions released this 09 Sep 16:10

Correct the published config comment about what the summariser costs