Release/packaging - #3
Merged
Merged
Conversation
electron-updater checks GitHub Releases (startup + manual "Check for Updates..."), never auto-downloads without a click; the update channel (stable vs. beta, driven by the tag suffix in release.yml) is selectable in-app and persisted both Electron-side and in ~/.spyde/settings.json. Help -> GPU Status surfaces the torch/CUDA diagnostics that vector_orientation_gpu already computed but never exposed in the UI. release.yml now creates a draft release before the platform matrix builds (avoiding electron-builder's concurrent-publish duplicate-draft race), publishes real installers + latest*.yml update feeds, and only un-drafts once all three legs succeed.
The 16-48px frames baked into Spyde.ico (and the matching icon.iconset PNGs icon.icns was built from) were a different, wrong icon -- only the 64px+ frames had the real spider-web logo. Since Windows/macOS use the small sizes for the title bar and dock, the app appeared to be showing a generic icon. Regenerated every size from the single correct source (Spyde.png) so all frames are now consistent.
- Move _STAGED_HANDLERS out of the backend into spyde/actions/registry.py (STAGED_HANDLERS + resolve_staged/register_staged) so adding an action only touches the actions package; document the WindowController protocol. - Session gains _window_controllers: bare-figure windows (strain map, IPF refine/views) register a controller for dispatch + teardown; _forget_window closes it and evicts per-window kept-alive figures (figure_registry). - ✕-closing a controller-backed window now routes through _forget_window instead of only emitting window_closed. - Delete dead duplicates: MDIManager.signal_trees (never populated), MDIManager.close_signal_tree (no callers; Session._close_tree is the one teardown route), PlotWindow._spyde_tree_teardown (no writers). - figure_registry: per-window keep-alive replacing the append-only leak lists (migration of the five _ALIVE sites lands with the lifecycle phase).
- lifecycle.py: the shared idioms every heavy action re-rolled — run_on_worker (daemon thread + _dispatch_to_main marshal, inline test fallback), bump_generation/is_current (StrictMode/latest-wins guard), resolve_vectors + fv_batch_running + wait_for_vectors (the find-vectors attach gap), replace_tree_attr (overlay swap), paint_signal_plots, progress_emitter, live_fill_poller (progressive shm fill). - commit.py: the two tree-spawning lifecycles with provenance stamping — open_result_tree (early-open progressive window: FV vectors tree, OM blank IPF) and commit_result_tree (THE Commit action: primary map + chip views + locked symmetric levels + attrs + on_tree hook). - Rewired by replacement: StrainController.commit() -> commit_result_tree (_commit_strain_tree deleted); vector_orientation_om result windows + _build_ipf_heatmap -> commit_result_tree; find-vectors _start_batch + build_vectors_result_tree and orientation _create_blank_ipf_window -> open_result_tree. - The five append-only _ALIVE figure lists (views, strain_display, ipf_view, ipf_density, ipf_refine_render) replaced by figure_registry.keep_alive keyed by window_id, evicted by _forget_window — figures no longer leak for the process lifetime. IPF-refine window now registers its controller.
Bug fixes (the click-a-button-and-nothing-happens classes): - Bare-figure windows: StrainController registers in the session window- controller registry (module _CONTROLLERS global deleted); the IPF-refine controller likewise; X-closing them now tears down overlays/nav hooks. - The find-vectors attach gap: vom_generate_library, vom_run and Vector VI self-wait via lifecycle.wait_for_vectors (strict per-tree mode) instead of hard-erroring; strain uses the same shared helper (bespoke copy deleted). - StrictMode double-fire: fv_preview/fv_stop get the run/stop generation guard (check -> attach -> re-check -> undo if superseded); CZB is synchronous and needs none. - update_vi latent no-op: generic toolbar dispatch now stores the Action instance with its artifacts. - A failed toolbar action emits action_active:false so its button un-lights. - tree.close() gets a re-entrancy guard and tears down ALL interactive action state (strain controller, wizards, overlays, IPF caches). - requires_vectors wired in toolbars.yaml: vector actions appear only after diffraction_vectors attach (toolbar re-sent by _finalize). - Silent no-ops (czb_manual_start, IPF point selector, overlay attach on a missing plot2d) now emit an error / warn instead of vanishing. Full migration onto the framework (compute cores untouched): - wizard.py: WizardController base (gen guard, own_window, worker marshal, overlay swap, close/remove/commit hooks). - StrainController subclasses it; _recompute rides run_on_worker. - OM and VOM wizard-state dicts become OmWizard/VomWizard controllers (attribute access; wholesale replace on library regen; teardown owns the refine-IPF window too). _set_overlay + tests updated. - Every hand-rolled worker thread rides lifecycle.run_on_worker; the two progressive-fill pollers ride lifecycle.live_fill_poller (identical intervals); overlay swaps ride replace_tree_attr. - rebin2d becomes Rebin2DAction, the first TransformAction subclass. - _template_action.py: copyable skeletons for all three action shapes + smoke test so they cannot rot.
…ooks
Staged-message naming standardized to <key>_open/_close/_tune/_run/_commit
(atomic across backend, registry, renderer, tests, specs - no aliases):
strain_run/strain_stop -> strain_open/strain_close
fv_preview/fv_stop -> fv_open/fv_close
czb_manual_start/czb_manual_stop/czb_manual/czb_auto
-> czb_open/czb_close/czb_pick/czb_run
Wizard-specific stages (om_generate_library, vom_refine, ...) keep their
<key>_ prefix.
wizardHooks.tsx - the shared renderer wizard lifecycle:
- useWizardLifecycle: mount->open / unmount->close, StrictMode-safe (open
deferred one tick + cancelled by the synchronous remount cleanup, so
exactly ONE open reaches the backend; its gen guard stays as backstop).
- useDebouncedAction: the debounced live-tune sender, cancelled on unmount.
- useWizardEvent: spyde:* CustomEvent subscription filtered per window.
- CommitButton: the standard Commit affordance -> <key>_commit
(WizardController.commit() -> commit_result_tree on the backend).
All five wizards (Strain, FindVectors, Orientation, VectorOrientation,
CenterZeroBeam) refactored onto the hooks; strain Submit becomes the
standard CommitButton (testid strain-commit; specs updated).
- spyde/actions/README.md: taxonomy, the two dispatch paths + full YAML gate reference, staged verb convention, lifecycle + ownership map, step-by-step for each action shape, renderer wizard-hook contract (StrictMode rule), pitfalls (each a real bug class), and how to test an action. - CLAUDE.md: Actions section rewritten to point at the README + the framework modules; find-vectors timing-trap note updated to wait_for_vectors and the requires_vectors gate.
- Renderer FIGURE reducer: a named chip view (view_label) no longer renames
its window - a committed Strain tree ended up titled by the LAST-emitted
chip ("omega") instead of "Strain" (screenshot-caught).
- actions_lifecycle.spec.ts (new): requires_vectors gate end-to-end (vector
actions hidden while the batch computes, appear on attach), Commit-to-new-
tree via the strain caret, and caret teardown leaving the committed tree.
- Both strain specs: wait on the REAL attach signal (the requires_vectors-
gated button appearing) instead of a dead waitForLog("Found") - status
strings travel the PLOTAPP protocol, invisible to the harness log; and
filter windows by the em-dash "- Vectors" title (a bare "Vectors" filter
also matches the open Find-Vectors wizard text in the source window).
- test_composition: drop the obsolete threading.Thread inline stub -
cod_search/cod_pick ride run_on_worker, which is inline for session=None.
- find_vectors_action: INFO timing logs around the batch (compute vs
finalize) so the chronic "batch only completes after clicking a plot"
stall can be localized from the app log.
ROOT CAUSE (measured, not guessed - see repro_batch_stall.py and
tests/_probe_fv_stall.spec.ts): Windows starves the hidden Electron-spawned
backend of timer interrupts. Timer waits (time.sleep, Event.wait, event-loop
timers - including dask scheduler task-delivery flushes) freeze INDEFINITELY
and wake only when process I/O arrives. Measured: a submitted batch sat
{waiting, processing} with all workers idle forever; a time.sleep(0.05) poll
loop slept 15 s; LocalCluster() took exactly 60.0 s; the same code from a
console completes in seconds. Every "the vectors appeared when I clicked the
plot" was the CLICK's stdin write waking the process.
THE FIX - Electron ticks the backend: runner.ts sends a no-op "tick" action
down stdin every 2 s (Electron's timers are healthy). Validated end-to-end:
scheduler up 60.0s -> 1.3s, batch NEVER-completes -> 5.8s hands-off, the
full lifecycle e2e 10+ min/flaky -> 56 s green.
Defense in depth (each earned by a failed experiment):
- compute_dispatch.poke_scheduler + no-progress watchdogs in the dispatcher
and both orchestrate wait loops (the proven unstick trio, fired every 5 s
of no progress; no-op on a healthy cluster).
- reliable_sleep (Event.wait-based) for progress-critical poll loops.
- process_guard.unthrottle_windows_timers (EcoQoS/timer-resolution opt-out,
backend + every worker via _WorkerTuningPlugin) + net-io stub now forced
(the 2 s probe passed once on a box where later ticks blocked for 60 s).
- heavy_imports.ensure_heavy_imports: the full-speed startup exposed a
hyperspy/pyxem circular-import race between the prewarm thread and the
first load ("partially initialized module") - single-flighted.
- dump_dask_state test action (+ per-call "only" filter): scheduler/worker
state + call stacks at WARNING - THE stuck-compute forensics tool.
- _probe_fv_stall.spec.ts (SPYDE_PROBE=1): the hands-off batch-health
regression probe that drove this whole investigation.
Two structural problems with the hover toolbar, both screenshot-verified: 1. It rendered in its window stacking context, so an overlapping sibling window buried it - "the actions only show up after I click the plot". Now the window raises itself (temporary z bump, focus order untouched) while its toolbar is revealed or engaged (caret open / action live), reported up by FloatingToolbar via onEngagedChange. 2. It hung OUTSIDE the window bottom, landing exactly on the titlebar of the window beneath - and once hover-raised it intercepted every pointer path into that window (users and Playwright alike). The bar now floats INSIDE the window bottom edge (video-player-controls style), so it can never shadow a neighbor and never covers its own titlebar. spyde.spec click-to-front test parks the mouse on neutral ground before comparing focus z-order (a hovered window is deliberately above siblings).
- tests/README.md: the suite map - two speed tiers (rule: never pay the distributed batch to test something that is not the batch), what each spec answers, how to read a failure (screenshots -> error-context -> logBuffer -> dump_dask_state), and the hard-won locator/interaction rules (select windows by what they HAVE, never text negatives - hasNotText is case-insensitive and matches wizard text; park-hover-click when switching windows). - _harness.cjs: loadTestVectors (the seconds-fast vectors result tree), waitForVectorActions (the REAL attach signal - the requires_vectors-gated buttons appearing), dumpDaskState. - strain_lazy absorbs strain_zrnb (deleted): commit + exact "Strain" title (the omega-retitle regression), toggle-off teardown with the committed tree surviving, reopen idempotency - all on the fast no-dask tier. - actions_lifecycle: robust locators + the park-hover-click idiom; owns the slow tier (requires_vectors gate timing + batch + commit on real dask). Verified: full journey suite 44/44 in 57 s (was 10+ min and hanging).
NOTEBOOK_PARITY_PLAN.md: the design + feasibility study for running every scientific action equally from a script (spyde.api), a Jupyter notebook (signal-first nb.action(signal) calls with full interactive wizard parity via anywidget/ipywidgets), and the SpyDE toolbar - same handlers, same payloads, same protocol messages. Key audited findings baked in: the compute cores are already headless (session params are dask-client lookups only); a real headless Session already runs in every pytest, so the notebook host is a presentation-seam problem (drawing/host.py + ipc.set_sink), not a re-implementation; anyplotlib is host-agnostic and CDN-free (one self-contained 263 KB ESM) - the JS weight is JupyterLab's own local labextensions; anywidget ships _esm per-instance (no dedupe) and the honest lever is an optional href-ESM upstream feature; Strain/VOM/CZB lack declared parameter schemas (hard-coded React forms) - schema completion is a named work item. Includes the per-action three-way contract table, five-phase implementation plan (script layer -> host seam -> notebook session -> Strain/FV exemplar wizards -> polish), per-host testing strategy, and risks.
The design-practice foundation for three-host action parity
(NOTEBOOK_PARITY_PLAN.md phase 1; the doc status header tracks this).
spyde.api - the script layer: find_vectors / orientation_map /
vector_orientation_map / strain_map / center_zero_beam / virtual_image /
vector_virtual_image as typed, headless wrappers over the SAME compute cores
the toolbar actions dispatch to. Never imports spyde.backend/spyde.drawing
(AST-checked + fresh-subprocess-checked); heavy imports are lazy; every
result carries a provenance record {"action","params","spyde_version"} (the
commit._stamp_provenance dict convention, so scripted results and committed
trees are interchangeable). Supporting seams: client= kwarg on the three
batch cores (explicit client wins; main_window/signal_tree lookups stay the
in-app path), provenance field on the four result dataclasses,
spyde/signals re-exports, and the pure reference physics
(zero_beam_filtered / default_reference) moved from strain_action into
strain_mapping so script strain uses the wizard-identical numbers.
Wizard parameter schemas - one source of truth per wizard: parameters
classattrs on StrainController and VomWizard, module PARAMETERS for CZB
(no controller class), FV/OM resolve from their existing toolbars.yaml
blocks; all reachable host-agnostically via the new
registry.wizard_parameters(key). This closes the audit gap where
Strain/VOM/CZB forms existed only as hard-coded React carets - a notebook
form generator (phase 4) can now render every wizard from the schema. The
wizard template skeleton and actions README document the new REQUIRED step.
Guarded by test_api_layer.py (import graph, headless smoke on synthetic 4D
data through find-vectors -> strain/VI/VVI, CZB centering correctness,
client= seam stability) and test_wizard_schemas.py (schema completeness,
entry validity, and lock-step with the handler DEFAULTS - including the
fv/om YAML blocks). Suite: 563 passed (2 known baseline failures);
strain_lazy e2e green.
- add kernel/dnd.ts and actions/navigator_views.py, which 8bf3e3e references but did not include (typecheck / e2e build / renderer all broke on the unresolved import) - Session now honours SPYDE_NO_DASK when constructed directly (tests bypass app.py's branch), so a load thread never blocks _await_dask's 120 s timeout on a cluster that will never start - test_open_file_emits_busy_then_clears polls for the busy clear instead of a fixed 0.8 s sleep - the cold first .hspy read takes seconds on CI runners, which failed the job on every OS - add ui_fixes.spec.ts, the e2e verification for the UI fix batch
read_messages read sys.stdin's text layer, whose encoding is the platform default - cp1252 on Windows. A UTF-8 payload from Electron carrying any non-ASCII char (strain labels εxx/εyy, the Å unit) was mojibake'd: bytes 0xCE 0xB5 (UTF-8 ε) decoded to "ε", so tile_views' view-label lookup silently dropped the view and the combined strain figure never appeared (vector_om_lazy CI failure). Now read the binary stream and decode UTF-8 explicitly; fall back to text stdin when a harness supplies a stream without .buffer. Adds test_ipc_read.py. Also: - views.build_tiled_figure matches labels NFC-normalised (defence in depth against composed/decomposed round-trips) - tile_views falls back to payload window_id for bare-figure windows (VOM strain / IPF views have no registered Plot) - strain_lazy / vector_om_lazy: raise the source window via the app's own spyde_focus channel before clicking a caret the live result window now covers (toolbar shares the window z-level, no hover-raise); chip multi-select uses ControlOrMeta; IPF toggle asserts toBeAttached (the cascade can cover it - assert wired, not on-top)
test_strict_mode_double_mount_builds_only_one_controller flaked on fast CI runners (macOS py3.12/py3.13): it fires strain_open/close/open synchronously to mimic React StrictMode, but on a fast box the FIRST worker's 6x6 compute could finish and build a gen-1 window BEFORE the main thread issued close/open #2 - racing the gen-3 window to two figures. Production never sees this (StrictMode remounts before any worker lands), so it was a test-timing artifact, not a real bug. Gate compute_strain_field on a barrier the main thread releases only after the full open/close/open sequence is issued, so both workers always land into the final generation like they do in the real app. Deterministic 8/8 locally.
vector_om_lazy flaked on the loaded CI e2e runner at the Refine-tab strain readout: the assertion waited for the vom_fit event to have already streamed, but on a slow box it lagged the tab switch, so the readout still showed "No fit yet". (The test data has 4 disks at every nav position, so it was never a too-few-vectors problem - purely timing.) Nudge the strain-cap slider to FORCE a fresh vom_refine -> vom_fit at the current crosshair, and retry via toPass until the readout populates. Deterministic locally.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add in release and packaging as well as fix lingering bugs