Skip to content

Releases: solmyrhq/ai-workflow

7.0.0

Choose a tag to compare

@pocketarc pocketarc released this 09 Oct 00:11
Immutable release. Only release title and notes can be modified.
335d285

What's Changed

  • feat!: Replace Prism with the Laravel AI SDK behind the package's own types. by @pocketarc in #16
  • ci: Run Claude Code Review on pull requests. by @pocketarc in #17
  • docs: Note that the package does not find 6.x cache entries for requests with URL attachments. by @pocketarc in #18

Prism is replaced by the Laravel AI SDK

The package now uses the Laravel AI SDK (laravel/ai) internally to send requests, and Prism PHP is no longer a dependency. Every public method signature uses only the package's own types, in the AiWorkflow\ namespace. Only the internal AiWorkflow\Gateway classes refer to Laravel\Ai classes.

6.x depended on prism-php/prism at the dev-feature/openrouter-reasoning branch of the pocketarc/prism fork. Because of this, you had to add the fork to your app's composer.json as a vcs repository. In 7.0, Composer installs every dependency from Packagist, and composer require pocketarc/ai-workflow works without a repositories block.

Remove the prism-php/prism requirement and the pocketarc/prism repository from your composer.json. Keep them only if your app uses Prism itself.

7.0 adds no migrations.

Requirements

  • PHP 8.3 or later, as in 6.x.
  • Laravel 12.62 or later, or Laravel 13.15 or later. Laravel 11 is no longer supported.
  • laravel/ai ^1.0.
  • pocketarc/laravel-integrations ^6.3. The 6.x constraint was ^6.0.

The OpenRouter API key comes from the Integration row

In 7.0, the package reads the OpenRouter API key from the api_key credential of the openrouter Integration row. It ignores OPENROUTER_API_KEY. In 6.x, Prism read OPENROUTER_API_KEY, and the Integration row only had to exist. AiWorkflowReplayer and the eval runner also read the key from the Integration row. If the row has no api_key, the package throws a RuntimeException for every OpenRouter request.

Before you deploy 7.0, make sure that the production Integration row holds the real OpenRouter key.

The package reads the OpenRouter URL and extra headers from config('ai.providers.openrouter'). Put the API keys for other providers, such as anthropic: and openai:, in the Laravel AI SDK's config/ai.php. To publish that file, run php artisan vendor:publish --tag=ai-config.

Prism types are replaced by package types

Replace every import from Prism\ with the package type in this table:

6.x (Prism) 7.0
ValueObjects\Messages\UserMessage and the other messages AiWorkflow\Messages\UserMessage and the other messages
ValueObjects\Media\Image, Document, Audio, Video, Text AiWorkflow\Messages\Attachment with an AttachmentKind
ValueObjects\ToolCall, ToolResult AiWorkflow\Messages\ToolCall, ToolResult
Schema\ObjectSchema and the other schemas AiWorkflow\Schema\ResponseSchema with a JSON Schema array
Tool::as(...)->using(...) new AiWorkflow\Tools\Tool(name, description, parameters, handler)
Text\Response, Structured\Response AiWorkflow\Responses\TextResponse, StructuredResponse
ValueObjects\Usage (promptTokens, completionTokens, cacheReadInputTokens, cacheWriteInputTokens) AiWorkflow\Responses\Usage (inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens)
ValueObjects\Meta AiWorkflow\Responses\ResponseMeta
Enums\FinishReason AiWorkflow\Enums\FinishReason, with the same values
Streaming\Events\*Event AiWorkflow\Streaming\*
PrismException and its subclasses AiWorkflow\Exceptions\ProviderException and its subclasses
Prism::fake() AiWorkflow\Testing\OpenRouterFake

Also make these changes to your code:

  • Write a tool's parameters as a JSON Schema object. Write its handler with one array $arguments parameter. In 6.x, the handler had one parameter for each tool parameter.
  • Change the signature of your judges to judge(AiWorkflowRequest $originalRequest, TextResponse|StructuredResponse $response).
  • Pass a ResponseSchema to sendStructuredMessages() and sendStructuredMessagesWithTools(). SchemaBuilder::fromDataClass() returns a ResponseSchema, and AiWorkflowContext::$schema is a ResponseSchema.
  • Use the package's Usage and FinishReason types with the events, AiWorkflowContext, StructuredDataResult and the usage() method of AiWorkflowException.
  • Handle the stream events StreamStart, TextDelta, ReasoningDelta, ToolCallEvent, ToolResultEvent and StreamEnd. There is no error event. If the provider sends an error during a stream, the generator throws an UpstreamErrorException.

When the retries are exhausted, AiService throws a subclass of ProviderException. The subclasses are RateLimitedException, InsufficientCreditsException, ProviderOverloadedException, ProviderConnectionException, ProviderRequestException, UpstreamErrorException, UnexpectedFinishReasonException and StructuredDecodingException. ProviderException does not extend AiWorkflowException, so sendStructuredData() throws these exceptions wrapped in StructuredDataRequestException. If your code catches Prism exceptions, change it to catch these exceptions. If the model calls a tool that is not in the request's tools, the package throws UnknownToolException.

Each step of a tool loop is a separate request

The package runs the tool loop itself. It sends each step as a separate request through the laravel-integrations executor, so each step has its own integration_requests row and its own retries. If a step fails, only that step is sent again. In 6.x, the whole tool loop was one executor call with one integration_requests row. On a retry, the executor sent the whole loop again from the first step.

TextResponse::$messages holds only the assistant and tool-result messages from the call. toolCalls and toolResults contain the tool calls and results of every step. In 6.x, they contained only those of the last step. usage is the sum over every step.

Unknown, Error and Other finish reasons are retried

In 6.x, AiService checked the finish reason after the request returned. It threw a PrismException for Unknown, Error or Other, and it did not retry the request. In 7.0, the package checks the finish reason of an OpenRouter response inside the executor. OpenRouterProvider classifies the failure as an upstream fault, so the executor retries the request. The delay is the attempt number multiplied by server_error_multiplier_ms. If every attempt is rejected, the package throws UnexpectedFinishReasonException. Its usage property holds the tokens of every rejected attempt.

The package sends requests to providers that have no registered integration, such as anthropic:, without the executor. It also sends replays and streams without the executor. For these requests, the package throws UnexpectedFinishReasonException on the first rejected response. The eval runner retries a rejected replay up to ai-workflow.eval.replay_tries times.

A response that is rejected for its finish reason is no longer logged as the call's response. The row's response_text, structured_response and finish_reason are empty. response_body holds the body of the last rejected response. The token columns hold the sum of the tokens of every rejected attempt.

Token counts for direct providers come from the Laravel AI SDK

For providers other than OpenRouter, such as anthropic:, openai: and gemini:, the package now uses the token counts of the Laravel AI SDK. In these counts, input tokens include cache reads and cache writes, and output tokens include thought tokens. In 6.x, OpenRouter's counts were already calculated this way.

Compared with 6.x, the logged input_tokens of Anthropic and OpenAI requests now include cache reads and writes. The logged output_tokens of Gemini requests now include thought tokens. The package does not change rows that 6.x logged. The cost in eval reports is calculated with one formula for every provider.

System messages in a conversation are added to the system prompt

The package appends each SystemMessage in a conversation to the system prompt, in order, with a blank line between them. The position of the message in the conversation is lost. In 6.x, Prism sent the system message at its position in an OpenRouter conversation, and it threw a PrismException for an Anthropic request.

Video attachments work only on OpenRouter and Gemini

For a video attachment, the Anthropic and OpenAI gateways in the Laravel AI SDK throw an InvalidArgumentException.

The package now sends an AttachmentKind::Media attachment as an image, audio, video or document, according to its MIME type. In 6.x, the package left it out of OpenRouter requests. For a Media attachment without a MIME type, the package throws UnsupportedAttachmentException.

client_options

In 6.x, the package passed ai-workflow.client_options to Prism's withClientOptions() for every provider. In 7.0, the package uses timeout as the request timeout for every provider, rounded up to whole seconds. It passes the other Guzzle options, such as curl, only to OpenRouter requests.

The package no longer downloads URL attachments when it logs a request

In 6.x, MessageSerializer called Prism's base64() and mimeType() methods on every attachment. For an attachment with a URL and no base64 content, base64() downloaded the URL with Laravel's Http::get(). It returned the base64 of the response body, and the serializer stored that base64 next to the URL.

If the attachment had no MIME type, mimeType() detected one from the downloaded bytes.

MessageSerializer ran when the package logged a request. It also ran when AiWorkflowCache built the cache key for a prompt with cache_ttl. In both cases...

Read more

6.0.0

Choose a tag to compare

@pocketarc pocketarc released this 27 Sep 17:46
4905cd0

What's Changed

  • fix!: Replay structured requests with their original schema and count the tokens of every attempt. by @pocketarc in #14
  • fix: Return the response when a cache write fails. by @pocketarc in #15

ai_workflow_requests has a new schema_name column

The nullable column holds the schema name that each structured request was sent with. AiWorkflowReplayer uses it to replay the request with the same schema name. For requests logged before the migration, which have no schema_name, AiWorkflowReplayer still uses the name schema.

If ai-workflow.logging.enabled is true, AI calls fail with a database error until the migration has run. Run the migration before the upgraded app handles any AI calls.

Copy the new migration into database/migrations with the current timestamp as its filename prefix. Then run the migration:

cp vendor/pocketarc/ai-workflow/database/migrations/0001_01_01_000010_add_schema_name_to_ai_workflow_requests.php \
   "database/migrations/$(date +%Y_%m_%d_%H%M%S)_add_schema_name_to_ai_workflow_requests.php"
php artisan migrate

With the current timestamp, the new migration runs after the ai-workflow migrations that you published before. If those filenames start with the date on which you published them, the original 0001_01_01_000010 prefix would make the new migration run before the migration that creates ai_workflow_requests, so migrate:fresh would fail.

Do not run vendor:publish --tag=ai-workflow-migrations for this upgrade. With Laravel's default database.migrations.update_date_on_publish setting, the command publishes every ai-workflow migration again with a new timestamp. migrate then fails on the first copy, which creates tables that already exist.

sendStructuredData() wraps request errors in StructuredDataRequestException

sendStructuredData() now throws StructuredDataRequestException when an attempt fails with an exception that does not extend AiWorkflowException, such as a provider error, a rate-limit error or a connection failure. getPrevious() returns the original exception, and getCode() returns its code, or 0 if that code is not an integer. The attempts property holds the number of the attempt that failed. usage() returns the tokens of the earlier attempts plus those of the failed attempt's response, if one was received. An exception that extends AiWorkflowException, such as GuardrailViolationException, is rethrown as the same instance.

If you catch Prism or laravel-integrations exceptions around sendStructuredData(), catch StructuredDataRequestException instead and check getPrevious(). The other AiService methods still throw the original exceptions.

Other behaviour changes

  • StructuredDataResult->usage now holds the tokens of every attempt, including attempts that failed validation. No tokens are counted for an attempt served from the cache.
  • Every exception that extends AiWorkflowException has a usage() method. It returns the tokens of the responses received before the exception was thrown, or null if no response was received. For an exception thrown by sendStructuredData(), usage() is never null and covers every attempt.
  • When a middleware such as an output guardrail rejects a response, or the finish reason is Unknown, Error or Other, the row logged for the failed request now contains that response (the text or structured output and the finish reason) and its tokens.
  • When the structured output cannot be decoded and the prompt has a fallback_model, sendStructuredMessages() and sendStructuredMessagesWithTools() now log the failed primary request and dispatch AiWorkflowRequestFailed for it. They do the same for the fallback request if it fails. In sendStructuredMessages(), the fallback request now runs through the middleware, and its response is cached.
  • If an AiWorkflowRequestCompleted listener throws an exception after a successful request, the request is no longer logged a second time as failed, and AiWorkflowRequestFailed is no longer dispatched.
  • If a cache write fails, AiService passes the exception to Laravel's report() and still returns the response. Before, AiService threw the exception, so the caller lost the response, the request was logged a second time as failed, and AiWorkflowRequestFailed was dispatched.
  • sendStructuredData() no longer caches a response that fails validation.
  • A response is now cached under a key built from the request as it was before middleware ran, which is the key used for cache reads. Before, a request was never served from the cache if a middleware had rewritten its messages or system prompt.
  • AiWorkflowReplayer now rebuilds the recorded schema exactly, including anyOf. On MySQL, which reorders JSON object keys, it restores each object's property order from its required list when that list names every property.

Full Changelog: 5.1.0...6.0.0

5.1.0

Choose a tag to compare

@pocketarc pocketarc released this 26 Sep 00:30
36cecaa

What's Changed

  • feat: Correct token costs in eval reports and reject structured answers containing INF or NaN. by @pocketarc in #13

Full Changelog: 5.0.1...5.1.0

5.0.1

Choose a tag to compare

@pocketarc pocketarc released this 09 Aug 23:38
8d5199b

What's Changed

  • fix: Stop the verdict migration failing on MySQL and MariaDB. by @pocketarc in #12

Full Changelog: 5.0.0...5.0.1

5.0.0

Choose a tag to compare

@pocketarc pocketarc released this 09 Aug 23:00
8c7b6fd

What's Changed

  • feat!: Record only the answer that a reviewer confirmed as correct, not a separate verdict. by @pocketarc in #11

Structured responses are checked against their Data class rules

sendStructuredData() now builds the response with validateAndCreate() instead of from(). from() ignored any validation rules the Data class declared, so ranges, enums and formats were never applied. They are now. This changes runtime behaviour for existing code: a response that used to be accepted can now fail validation if it breaks a declared validation rule. Review the validation rules on any Data class you pass to sendStructuredData() before upgrading. sendStructuredMessages() is unaffected.

Full Changelog: 4.0.0...5.0.0

4.0.0

Choose a tag to compare

@pocketarc pocketarc released this 07 Aug 10:57
b4c12e9

What's Changed

  • feat: Retry an eval replay once when the provider fumbles the response. by @pocketarc in #9
  • fix!: Require every schema property for OpenAI's strict mode. by @pocketarc in #10

Full Changelog: 3.2.0...4.0.0

3.2.0

Choose a tag to compare

@pocketarc pocketarc released this 02 Aug 13:10
3b482a3

What's Changed

  • feat: Show the host app's links and last comment in the eval report. by @pocketarc in #8

Full Changelog: 3.1.0...3.2.0

3.1.0

Choose a tag to compare

@pocketarc pocketarc released this 01 Aug 19:41
bcb9ccb

What's Changed

  • fix: Replay evals against today's prompt and its reasoning setting. by @pocketarc in #7

Full Changelog: 3.0.0...3.1.0

3.0.0

Choose a tag to compare

@pocketarc pocketarc released this 26 Jul 23:43
67736f5

What's Changed

  • chore!: Require pocketarc/laravel-integrations ^6.0. by @pocketarc in #6

Full Changelog: 2.2.0...3.0.0

2.2.0

Choose a tag to compare

@pocketarc pocketarc released this 19 Jul 18:24
679387c

What's Changed

  • feat: Persist and cost thought tokens on eval scores. by @pocketarc in #5

Full Changelog: 2.1.0...2.2.0