Repository navigation
0.4.0
Pre-release
Pre-release
A minor release: the tool outcome log changes shape, so .upToNextMinor(from: "0.3.1") pins don't pick it up.
.package(url: "https://github.com/airowe/syrup-ai-kit", .upToNextMinor(from: "0.4.0"))Changed
ToolOutcomeLogkeeps every call in a run, not only the last one.callsandsummaryare new,laststill works, and the log keeps at most 20 calls (callLimit). SyrupAIEvalKit launches the app with--siri-ai-eval-clear-outcomes, so each run starts empty. The label is now "Tool calls: …", and the old "Last tool call: …" label is still read. Before, a tool called after a failure replaced the failure in the log. (#42)- Storage key is now
syrupAIKit.toolOutcomes. Outcomes stored by 0.3.x aren't carried over.
New
- Expo plugin: generated tools honour the
bad-hostEvalSwitch. With it on, every generated request goes tohttps://siri-ai-eval-bad-host.invalid, so a generated tool can now have a negative control. (#43) - Expo plugin: pass
SIRI_AI_EVAL_SWITCHES=YEStoxcodebuildto compileEvalSwitchchecks into a Release build. Guard them with#if DEBUG || SIRI_AI_EVAL_SWITCHES. Builds without that setting, such aseas buildand archives, leave the checks out. (#43)
Checked on a phone
The checks used a Release Expo app with SIRI_AI_EVAL_SWITCHES=YES:
- A generated value-only tool passed 5 of 5 runs.
- Its
bad-hostnegative control failed the reply check in 4 of 4 complete runs, as intended. - A failed call followed by a second one showed both calls in the outcome.
- The bad-host URL isn't in a Release build made without the setting.
Docs
- Results of the 0.3.1 device run: value-only tools, an enum parameter, a failure, and a write action with confirmation. (#41)
- A reply check can miss because Siri writes typographic apostrophes ("couldn’t"). Match a word without one.