Skip to content

v0.3.85 — Expected test errors no longer raise console error pressure, and editor quit is confirmed by process exit

Choose a tag to compare

@FoxsterDev FoxsterDev released this 03 Nov 22:40
· 78 commits to main since this release

Why this matters

A PlayMode suite where every test passed could still report
console_error_pressure_detected: true, the flag meaning "Unity logged console
errors during this run", because a test deliberately triggered an error and
asserted it with LogAssert.Expect. Suites longer than about 5 minutes could end
in request_lifecycle_reset after the Play Mode domain reload, even with a
larger timeoutMs. And when the Unity editor was in the background in Edit
Mode, unity.editor.quit replied "acknowledged" while the editor stayed open.
By default, request-editor-quit returned at that acknowledgement. With
--wait-for-exit, it waited 20-30 s and then reported
editor_quit_ack_without_exit.

What changed

  • Long test runs keep the caller's timeout. Entering Play Mode reloads the
    script domain, which drops the request inside the editor. The host (the
    launcher and MCP server that run outside Unity) then waits for the run to
    finish and recovers its result; this is the post-reset recovery wait.
    unity.tests.run_editmode and unity.tests.run_playmode no longer stop that
    wait at a fixed 300 s cap. The longest live run in this release took 250 s;
    waits longer than 300 s are covered by host unit tests only. A runtime config
    post_reset_recovery_cap_ms for these two operations is ignored and listed in
    runtime-config-show ignored_overrides. Recovery now reports
    post_reset_recovery_mode, post_reset_recovery_budget_ms,
    post_reset_recovery_elapsed_ms and post_reset_recovery_stop_reason.
  • Console errors are attributed per test. Through Unity Test Framework's log
    scope, an error counts as expected_by_test only when the test passed and
    either matched it with LogAssert.Expect or had
    LogAssert.ignoreFailingMessages set. Those errors no longer set
    console_error_pressure_detected. Errors without that evidence (off the main
    thread, outside a test, during an interrupted test, before a domain reload, or
    on a Test Framework without a readable log scope) still raise pressure.
    test_verdict is unchanged. New fields:
    console_error_expected_by_test_count, console_error_not_expected_count,
    console_error_classification_basis and up to 3 console_error_samples with
    the active test name. console_error_count_since_request_start keeps its raw
    meaning and still includes expected errors.
  • Final status of a finished run uses that run's counts. Counts are stored
    with the result (console_error_count_at_completion and related fields)
    instead of being read from the live editor counter that later runs change. A
    result stored without them reports lower_bound_from_recorded_attribution or
    unavailable.
  • Test results record the test setup. A new test_environment object holds
    the Game View render size, preset label, preset_kind, pixel size,
    render_matches_preset, size group, build target and capture_status. It is
    captured without opening, focusing or waiting on a window.
  • The compact CLI summary keeps the verdict. When the compact CLI envelope
    (the short JSON summary the launcher prints) exceeds 8 KB, it drops
    console_error_samples, then test_environment, then shortens
    first_failure text before falling back to the reduced form. It lists what it
    dropped in compact_omitted_fields. The reduced form keeps test_verdict,
    the pass/fail/skip totals, failure_count and the console error pressure
    fields.
  • Editor quit works when the editor is unfocused. On Unity 6000.0.58f2 on
    macOS, Unity skipped EditorApplication.delayCall in Edit Mode while the
    editor was not the active app. The quit now fires from a one-shot
    EditorApplication.update tick. The quit intent is kept for 120 s so that it
    survives a domain reload; that part is covered by EditMode and unit tests
    only. The acknowledgement (the bridge's reply that it accepted the quit) adds
    quit_mechanism: editor_update_one_shot, editor_pid,
    editor_application_focused, is_playing and scene_dirty. Older packages
    report quit_mechanism: legacy_delay_call.
  • Behaviour change: request-editor-quit waits for the process to exit by
    default
    (--exit-timeout-ms default 30000). If the editor survives, the
    command exits non-zero with editor_quit_ack_without_exit, the live editor
    PIDs and a recovery command; it sends no signal. --no-wait-for-exit restores
    the old ack-only result as closeout_classification: quit_ack_unverified
    with process_exit_verified: false. A quit the bridge rejects now fails at
    once with editor_quit_request_rejected (closeout_classification: quit_request_rejected).
  • unity.game_view.configure selects only fixed-resolution sizes. 9x16
    used to select Unity's built-in 16:9 Portrait aspect-ratio preset and report
    resolution_applied. It now returns an error that explains the match is an
    aspect-ratio preset, or creates a fixed custom size when
    allowCreateCustomSize is set. The resolved game_view adds preset_kind
    (fixed_resolution, aspect_ratio, free, unknown).
  • Printed recovery commands use the absolute launcher path. Recovery,
    cleanup and refresh commands now start with the quoted absolute launcher path
    (PowerShell gets a & prefix) and include a quoted project root where
    needed. An unknown, missing or mismatched launcher path falls back to the
    bare launcher name, which does not run from an arbitrary directory.
  • ensure-ready reports a declared package correctly.
    package_import_state.dependency_declared now reads the manifest, so a
    git-pinned dependency no longer shows false while status-summary reports
    it ready.
  • Upgrade together. Update the Unity package pin and the host launcher
    together. A project still pinned to v0.3.84 keeps the old quit path. With the
    editor unfocused in Edit Mode, that quit does not run, so
    request-editor-quit fails after about 30 s with
    editor_quit_ack_without_exit; a focused editor still quits. Older packages
    also keep raw-count console error pressure.
  • Unsaved changes on quit. EditorApplication.Exit is forced and discards
    unsaved scene and Prefab Mode changes without a prompt. That is unchanged, but
    the quit now actually happens while the editor is unfocused. Check
    scene_dirty in the acknowledgement.
  • Restart clients. Restart MCP clients after updating. After upgrading an
    installed launcher, restart twice, because the first start still runs the
    previous refresh launcher.

What this gives developers

  • Fewer false alarms from expected errors. With the new package and a Test
    Framework that has a readable log scope, main-thread errors that a passing
    test expected no longer set pressure. Errors logged off the main thread,
    outside a test, during an interrupted test or before a domain reload still
    set it. When pressure is true, the samples name the test that was running.
  • Long PlayMode suites are no longer cut off by a fixed 5-minute recovery cap;
    they wait up to the timeoutMs you gave (live runs in this release reached
    250 s, so the longer case rests on host unit tests).
  • Final status you read later describes the run you asked about, even after
    other runs.
  • A failing layout test shows the resolution it ran at, and preset_kind tells
    you whether the Game View used a fixed size or an aspect-ratio preset.
  • By default, a successful request-editor-quit means the editor process
    exited. If the editor is still running after the timeout, the command names
    the live PIDs and prints a recovery command. With --no-wait-for-exit, a zero
    exit code only means the quit was acknowledged (quit_ack_unverified).
  • Large test results keep their verdict and totals in compact output.

Validation

  • Host Python suite: 1279 tests OK, 14 skipped on macOS. Release safety, docs
    freshness and version consistency checks passed.
  • Remote CI (Integration Tests on Windows, Ubuntu and macOS, and Discovery
    Checks) passed for the release commit, and the Release Tag Gate passed.
  • Live checks on macOS with Unity 6000.0.58f2 in two consumer projects:
    • Package EditMode self-tests: 186/186 passed, console_error_pressure_detected
      false.
    • A 428-test PlayMode suite: 426 passed, 2 skipped, 0 failed in 237 s,
      pressure false. Before this release it reported pressure true from errors
      the tests expected.
    • A 39-test PlayMode fixture whose test expects one exception: 39/39,
      pressure false, expected_by_test 1. Before this release it reported
      pressure true.
    • test_environment was correct at the 1080x1920 and 1440x3200 fixed
      presets and under an aspect-ratio preset (render_matches_preset empty). A
      layout test that fails at 1440x3200 shows the 20:9 render size in its own
      result.
    • Request final status for a completed run returned that run's own counts
      after later runs.
    • Printed recovery commands ran unchanged from the home directory.
    • Editor quit while unfocused in Edit Mode: on v0.3.84 the quit was
      acknowledged but the editor did not exit (20-30 s, then ack-without-exit).
      With this release the editor exited in about 1.4-1.8 s without SIGTERM on a
      fresh editor, after a PlayMode run, after a refresh and after a forced
      domain reload (6 runs). In the domain reload case the quit was sent after
      the reload finished.
    • One more quit run used --no-wait-for-exit. That mode does not check
      process exit, so it is not part of the exit times above.
    • unity.game_view.configure with 9x16 returns the explanatory error;
      1080x1920 and 1440x3200 still select fixed presets.

Known limitations

  • Remote Unity Package CI remains waived because its runner has no Unity
    license.
  • A post-reset recovery wait longer than 300 s was not verified live; the
    longest live suite took 250 s, and the longer case is covered by host unit
    tests only.
  • A quit armed before a domain reload and fired after it was not shown live; it
    is covered by EditMode and unit tests only.
  • The EditorApplication.delayCall behaviour behind the quit fix was observed
    only on Unity 6000.0.58f2 on macOS.
  • Unity 2021.3 was only compiled offline at C# 9 against 2021.3 assemblies, not
    run.
  • Windows and Ubuntu editors were not tested live.
  • Printed commands were run live only on macOS from the home directory; cmd and
    PowerShell quoting on Windows hosts were not run live.
  • allowCreateCustomSize: true after an aspect-ratio-only match and quitting
    with a dirty scene were not verified live.