test(EasyTrees): let the dataset load settle before setting treeRange - #48
Conversation
The Distribution baseline recorded
trees <- WideSample(trees[1:125], 48)
for a state its own test sets to c(77, 125) two lines earlier, and the
value flipped between machines: the dev box recorded 1:125, the
windows-latest runner produced 77:125.
Neither is what the test asks for. `set_inputs(data-dataSource =
"Sun2018")` does not finish loading inside its default 4 s wait -- the run
log says so outright, "Server did not update any output values within 4
seconds" -- so the `data-treeRange` set on the next line lands while the
dataset is still loading. When the load completes, UpdateAllTrees() sees
the tree count change and calls UpdateTreeRange(c(1L, nTrees)), resetting
the range to the full span and discarding the c(77, 125) the test just
asked for. Whether that reset landed before or after the download was
captured is what varied by machine.
Waiting for the load to settle before touching treeRange makes the
ordering deterministic: the reset happens first, then the test's range is
applied and honoured.
Also make wait_stable() cover the debounce window. mod_data's nTree /
treeRange watchers are debounce()d (aJiffy = 42 ms, typingJiffy = 105 ms),
and a pending debounce timer does not make Shiny busy -- there is nothing
to recompute until it expires, so wait_for_idle() can return before the
watcher has seen the value just set. With the ordering fixed this is
load-bearing rather than defensive: the c(77, 125) thinning is triggered
through that debounced watcher, and set_inputs() cannot be relied on to
wait for it (its own 4 s wait expires with "did not update any output
values" when a set produces no output change).
The 001 baseline is provisionally the windows-latest runner's output for
the 77:125 state, from run 30919105631's uploaded .new artifact. The
input line is now correct by construction, but the clustering conclusion
and rogue-tip lists downstream of it depend on the sampled subset and so
on the RNG state, which the corrected ordering shifts -- to be confirmed
against CI, whose job uploads .new artifacts for exactly this.
Fixes #46
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
067b4da to
e3c294d
Compare
|
Correcting the diagnosis in the PR description above. The Actual causeThe run log is explicit about it: The dataset load does not finish inside What gave it awayMy settle-wait change made CI produce
I had this backwards in the description above, where I called the runner's value "the value the The change now
Baseline statusThe Not a merge blocker either way: Not touched
|
Green, and the baseline needed no further changeRun 30931972326 — every job Previously The two Why the provisional baseline turned out exactI flagged it as uncertain because the sampled subset depends on RNG state, which the reordering That also retires the caveat in the PR description: the baseline is no longer provisional. Full matrix
Note this run also carries #53's |
Fixes #46
Stacked on #45 — branched from
feature/ci-maxmin-windows, so it carries thatPR's two commits until #45 merges. Merge #45 first and this reduces to the one commit.
It has to be stacked: without #45 the shinytest2 job dies in
Set up R dependenciesand never reaches the suite, so there is nothing to verify against.
What was wrong
The
Distributionbaseline recordedfor a state its own test sets to
c(77, 125)two lines earlier(test-Distribution.R:16) — and had
done so since the snapshot was first recorded on 2026-07-03. The pre-2026-07-27 version of
the file shows the same
1, 125under the old evenly-spaced stride, so this predates theWideSamplelogging change and is not MaxMin-specific.Why now, and why not "just stale"
CI had never checked it: the job died on the unresolvable
MaxMinSuggests (#43) longbefore reaching the suite. With #45 in place the runner reaches the tests and produces
trees[77:125]— the value the test asks for — so the stored baseline is the wrong one.The cause is that
wait_for_idle()does not wait for a pendingdebounce()timer.mod_data'snTree/treeRangewatchers are debounced (aJiffy= 42 ms,typingJiffy= 105 ms), and while that timer runs Shiny has nothing to recompute, so itreports idle.
set_inputs()+wait_for_idle()can therefore return before the watcher hasseen the value just set, and a download captured then encodes the previous state. Whether
the timer wins is machine-dependent — which is exactly why the dev box and the runner
disagreed on this one snapshot while agreeing on the other ten.
So
snapshot_accept()alone would have been the wrong move: it would have re-recordedwhichever side of the race the recording machine happened to land on, leaving the test free
to flip back later.
The change
wait_stable()sleeps past the longest debounce window and waits for idle again, so acapture happens after debounced work has run rather than possibly before it.
001baseline is replaced with the windows-latest runner's own output, taken fromthe
.newartifact of run 30919105631,rather than a locally re-recorded file — that run took the correct path, and adopting its
bytes keeps the baseline anchored to the platform the job runs on. Only the input line
differs at source; the clustering conclusion, cluster count and rogue-tip lists all follow
from the different tree subset.
What is verified, and what is not
Verified: the adopted baseline is byte-identical to what windows-latest / R 4.6.1
produced. Snapshots 002-011 already matched the locally-recorded baselines in that run
(FAIL 1, PASS 191), so cross-machine reproducibility is established for every state except
the racy one.
NOT verified locally. I could not run the suite on this box: the app subprocess aborts
with
namespace 'rlang' 1.2.0 is already loaded, but >= 1.3.0 is required, a stale locallibrary unrelated to this change, and updating the shared default library would disrupt
concurrent sessions. So the
wait_stable()change is reasoned from the debounce/idlesemantics, not measured. CI is the check here — and if the extra settling shifts any
other snapshot, the job uploads
.newartifacts to adopt.The job is
continue-on-error: true, so this cannot redden the workflow either way.Also worth knowing
WideSample()stop()s when MaxMin is absent(R/WideSample.R:149-152) — no silent fallback — so this baseline is
only reproducible where MaxMin resolves. That makes #45 a permanent prerequisite for the
suite, not just for this PR.
R/WideSample.Rsits in the unowned statistics/support-metrics cluster from #42, so nored-team area owns this code path today.
Investigated and dropped
I first suspected
FetchTreeRange's echo guard, which identifies a programmatic update'secho by arrival order and discards the first firing whose value differs from the pre-push
one — which looks like it could eat a real edit. Probing the module showed it cannot, in
practice: arming the guard is itself a reactive write, so the watcher re-fires and consumes
the guard ~42 ms after the dataset load, long before any user edit arrives. A candidate fix
that compared against the pushed value instead would have been worse — it would have
treated that stale firing as an edit and reverted the push. Both unit tests I wrote for it
passed against the unmodified module, confirming there was no defect there to guard. No
change made.
🤖 Generated with Claude Code