Skip to content

v0.3.87 — Tests no longer run stale code; cancel test runs; Android device lane

Choose a tag to compare

@FoxsterDev FoxsterDev released this 06 Nov 04:19
· 60 commits to main since this release

Why this matters

The MCP could report the editor as current while Unity had not compiled the
latest script edit, so a test run executed the previous assemblies and failed
on bugs that were already fixed. A long test suite could not be stopped, and
every Android device step (install, cold launch, logcat, frame pacing,
comparing two builds) had to be scripted by hand outside the MCP. The package
also did not compile on Unity 6000.5 and 6000.6.

What changed

  • Editor-domain currency is based on file content. The MCP keeps a
    snapshot of every .cs, .asmdef, .asmref and .rsp file under
    Assets/, embedded packages and local file: packages, taken when the
    loaded assemblies were compiled. Touching a file without changing it stays
    current; a real edit that was not compiled reads stale and names the
    file. unity_project_refresh requests the missing compile, and
    unity_tests_run_editmode / unity_tests_run_playmode refuse with
    editor_domain_stale (override: allowStaleEditorDomain=true).
  • unity_tests_cancel {requestId} stops a running EditMode or PlayMode run
    (Test Framework 1.4 or newer). The run completes as cancelled with the
    counts so far and not_run; a PlayMode run returns to Edit Mode.
  • Android device lane: nine tools that call adb from the host, so they
    keep working while the editor is busy: unity_device_list,
    unity_device_install, unity_device_app, unity_device_logcat,
    unity_device_files, unity_device_screen, unity_device_frame_pacing,
    unity_device_metrics and unity_device_coldstart_ab. State changes need
    allowMutating; uninstall and clear data also need approve and a package
    listed in the runtime config; serial is required when several devices are
    attached.
  • Project action payloads are checked before dispatch. Undeclared keys
    and values of the wrong type for the catalog token (for example a string
    for an int_array) are reported as payload_validation warnings, or
    refused with strictPayload.
  • Smaller responses. Transport errors no longer carry the final status
    twice, compact project action responses keep one currency verdict, and a
    test run without filters waits 1.5 times the longest recent run of the same
    kind (at least 300 s) instead of a fixed 300 s.
  • Scenarios. A refused scenario lists every validation error at once;
    project_defined_hook_poll_until no longer needs startPayload and
    failWhen for a plain wait.
  • Compatibility. The package compiles on Unity 6000.5 and 6000.6 again,
    import worker processes no longer overwrite the bridge state there, and a
    uGUI self-test now runs on Unity 2021.3.

What this gives developers

  • A test verdict reflects the code on disk, or the MCP says it does not.
  • A wrong or stuck test run can be stopped in seconds instead of waited out.
  • Install, cold-start timing, frame-pacing and A/B comparisons on an Android
    device are single tool calls with recorded results.
  • A typo or wrong type in a project action payload is reported instead of
    silently changing the hook's output.

Validation

  • Host Python suite: 1325 tests passed, 14 platform skips.
  • Package self-tests in clean generated projects, package resolved from local
    source, macOS, on Unity 2021.3.58f1, 2022.3.62f3, 6000.0.58f2, 6000.1.13f1,
    6000.2.14f1, 6000.3.23f1, 6000.4.4f1, 6000.5.10f1 and 6000.6.0b3: EditMode
    core 216/216 and PlayMode core 6/6 on every version; on 2021.3.58f1 also
    TextMeshPro 6/6, uGUI 57/57 and PlayMode uGUI 19 passed with 6 skipped.
  • Currency, live on Unity 6000.0.58f2: a touch stayed current; an edit made
    during a running EditMode test read stale after the run; the next test run
    was refused; a refresh compiled and the test then ran the new code.
  • unity_tests_cancel, live on Unity 6000.0.58f2: an EditMode run completed
    as cancelled about 2.6 s after the call (1 passed, 2 not run); a cancelled
    PlayMode run returned the editor to Edit Mode.
  • Device lane on one low-end Android 12 phone: one unity_device_coldstart_ab
    call installed two builds, confirmed the dex at verify and ran 3 cold
    starts per build. The longest splash freeze matched an earlier manual
    measurement (584–606 ms against 595–607 ms for the faster build). Ready times
    were about 1.2 s higher than that earlier measurement on the day of the run,
    with or without frame sampling.

Known limitations

  • Unity Package CI remains waived (no runner Unity license); the local
    licensed lane above is the package evidence.
  • Live Unity runs were on macOS only; Windows and Linux are covered by host
    tests and CI.
  • The forced compile of the currency fix is covered by unit tests; in the live
    runs Unity started the compile itself.
  • Test cancellation needs Test Framework 1.4 or newer.
  • The device lane is Android only; profiler capture, the editor Profiler
    connection to a player, native dialog input and iOS are not included.