Skip to content

1.1.0 TestBench AI Service

Choose a tag to compare

@BatoWa BatoWa released this 30 Jul 12:39
· 2 commits to main since this release
1cc8a22

Added

  • Minimum TestBench version check: all built-in agents now verify the connected server during
    precheck and fail with 409 Conflict and a localized message if it is older than
    TestBench 4.1.
  • check_min_testbench_version(context, conn) helper, available to custom agents. Custom agents
    that support older servers can skip it or check conn.server_version themselves.
  • The Defect Explainer now rejects XML-based test object versions. Only JSON-based TOVs are
    supported — either set directly on the TOV or inherited from the project's default exchange
    format.
  • TestBench API support for reading project and TOV metadata: get_project_details(),
    get_tov_details() and is_json_based_tov(), backed by the new ProjectDetails, TOVDetails
    and ProjectContext models and the ProjectStatus, ProjectExchangeFormat and
    TOVExchangeFormat enums.
  • English and German messages for both new precheck failures.
  • Dependabot configuration to keep GitHub Actions up to date.

Changed

  • The Defect Explainer reads failure messages and inserts explanations using the data-tb-*
    anchors of the current testbench2robotframework 2.x comment format. Comments written by
    tb2rf 1.1 and older are still handled by a legacy fallback.
  • Re-running the Defect Explainer on the same test case now replaces the previous explanation
    instead of appending another one — including when only its heading remained in the stored
    comment.
  • LLM output is HTML-escaped before it is written into an execution comment, so explanations
    containing < or & no longer corrupt the comment markup.
  • Dependencies now require final releases instead of pre-releases:
    testbench-cli-reporter>=3.0.0,<4.0.0 and testbench2robotframework>=2.0.0,<3.0.0.
  • Documentation states the TestBench version requirements per agent and the TOV format
    restriction of the Defect Explainer.

Fixed

  • An empty execution comment is no longer replaced by a bare AI disclaimer heading.
  • Execution traces passed to the LLM keep table cells on one line and no longer contain
    non-breaking spaces from tb2rf's comment indentation.
  • Test cases without an explanation are skipped instead of producing an empty annotation.