Skip to content

0.4.0

Pre-release
Pre-release

Choose a tag to compare

@airowe airowe released this 12 Nov 21:27
6eee709

A minor release: the tool outcome log changes shape, so .upToNextMinor(from: "0.3.1") pins don't pick it up.

.package(url: "https://github.com/airowe/syrup-ai-kit", .upToNextMinor(from: "0.4.0"))

Changed

  • ToolOutcomeLog keeps every call in a run, not only the last one. calls and summary are new, last still works, and the log keeps at most 20 calls (callLimit). SyrupAIEvalKit launches the app with --siri-ai-eval-clear-outcomes, so each run starts empty. The label is now "Tool calls: …", and the old "Last tool call: …" label is still read. Before, a tool called after a failure replaced the failure in the log. (#42)
  • Storage key is now syrupAIKit.toolOutcomes. Outcomes stored by 0.3.x aren't carried over.

New

  • Expo plugin: generated tools honour the bad-host EvalSwitch. With it on, every generated request goes to https://siri-ai-eval-bad-host.invalid, so a generated tool can now have a negative control. (#43)
  • Expo plugin: pass SIRI_AI_EVAL_SWITCHES=YES to xcodebuild to compile EvalSwitch checks into a Release build. Guard them with #if DEBUG || SIRI_AI_EVAL_SWITCHES. Builds without that setting, such as eas build and archives, leave the checks out. (#43)

Checked on a phone

The checks used a Release Expo app with SIRI_AI_EVAL_SWITCHES=YES:

  • A generated value-only tool passed 5 of 5 runs.
  • Its bad-host negative control failed the reply check in 4 of 4 complete runs, as intended.
  • A failed call followed by a second one showed both calls in the outcome.
  • The bad-host URL isn't in a Release build made without the setting.

Docs

  • Results of the 0.3.1 device run: value-only tools, an enum parameter, a failure, and a write action with confirmation. (#41)
  • A reply check can miss because Siri writes typographic apostrophes ("couldn’t"). Match a word without one.