Tests Show Failed When Passed

[lonnylot]

When running agent tool tests through the CLI they return as failed. However, when looking in the UI they all passed.

I believe it's trying to run tests that I deleted via the UI and never had locally.

node ➜ /workspaces/call-agents (master) $ npx elevenlabs agents test *snip*
Running 4 test(s) for agent '*snip*'...

Test invocation started (ID: * snip*)
Waiting for tests to complete...

Error running tests: Error: response -> test_runs -> [0] -> test_info: Missing required key "success_condition"; response -> test_runs -> [0] -> test_info: Missing required key "success_examples"; response -> test_runs -> [0] -> test_info: Missing required key "failure_examples"; response -> test_runs -> [1] -> test_info: Missing required key "success_condition"; response -> test_runs -> [1] -> test_info: Missing required key "success_examples"; response -> test_runs -> [1] -> test_info: Missing required key "failure_examples"; response -> test_runs -> [2] -> test_info: Missing required key "success_condition"; response -> test_runs -> [2] -> test_info: Missing required key "success_examples"; response -> test_runs -> [2] -> test_info: Missing required key "failure_examples"; response -> test_runs -> [3] -> test_info: Missing required key "success_condition"; response -> test_runs -> [3] -> test_info: Missing required key "success_examples"; response -> test_runs -> [3] -> test_info: Missing required key "failure_examples"

[adiixs]

I can confirm, I have the same problem running tests in CI/CD pipeline.

The problem also occurs when using the elevenlabs test pull command.

Errors about missing keys succes_condition, success_examples, and failures_examples are displayed, even though they are marked as optional in the UI. Furthermore, these are tool invocation tests.

↻ Updating config for 'Webhook_InvocationTriggered_Variant1'...
  ✗ Error updating test 'Webhook_InvocationTriggered_Variant1': Error: response: Missing required key "success_condition"; response: Missing required key "success_examples"; response: Missing required key "failure_examples"
↻ Updating config for 'Webhook_InvocationTriggered_Variant2'...
  ✗ Error updating test 'Webhook_InvocationTriggered_Variant2': Error: response: Missing required key "success_condition"; response: Missing required key "success_examples"; response: Missing required key "failure_examples"
↻ Updating config for 'Webhook_InvocationTriggered_Variant3'...
  ✗ Error updating test 'Webhook_InvocationTriggered_Variant3': Error: response: Missing required key "success_condition"; response: Missing required key "success_examples"; response: Missing required key "failure_examples"

[PennyroyalTea]

Released in 0.4.1.