feat(fit): the Fit wizard — build a model, fit the scan, commit component maps - #86
Merged
Conversation
Five waves, each independently shippable behind its own optional extra. Tracked in #48 with a sub-issue per task. Decisions recorded up front, because each one shapes the code: - spyde/models/ is already the neural disk detector, so the HyperSpy model port lives in spyde/fitting/. Unrelated meanings of "model". - exspy/kikuchipy/atomap ship as optional extras gated by a new requires_package key, mirroring requires_vectors. - The fitting engine is a batched GPU Levenberg-Marquardt over the whole nav grid (the vector_orientation_gpu.py playbook), with SAMFire's neighbour-seeding kept as the seed source rather than the scheduler. Measured baseline that motivates it: multifit on 1024 spectra x 1024 channels runs 10.83 s = 95 spectra/s single-threaded, which extrapolates to 11.6 min for a 256x256 SI. Parity with multifit is the acceptance test, not convergence. Also found while measuring: numexpr is not installed, so every hyperspy Expression component silently falls back to numpy.
Waves 1-4 need something loadable per modality from day one, before the real datasets land (#80). These are also the better thing to test against: the ground truth is known exactly, so a fit or an indexing run is scored against the numbers the data was built from rather than a golden file that quietly encodes yesterday's bug. eels_si power law + C/N/O K edges at real onsets, intensities following known concentration maps eds_si bremsstrahlung + Fe/Ni/Cu K families at real energies, with Fe-Kb/Ni-Ka overlapping ON PURPOSE so family-aware fitting is actually exercised rather than assumed ebsd_patterns real gnomonic Kikuchi-band geometry for a cubic crystal over a two-grain orientation field; the exact Euler angles are stamped on the signal, so dictionary indexing can be checked against truth instead of against a fixture of its own output Every spatial map is distinguishable from its transpose and both mirrors, so a flipped axis fails a test instead of looking fine — the si_grains/movie precedent. Ground truth is read with spyde.data.ground_truth(), not straight off the metadata: hyperspy turns nested metadata dicts into DictionaryTreeBrowser nodes, which have no .values(). None of this needs exspy or kikuchipy. set_signal_type is attempted and its "not understood" warning demoted to debug, so a user without the extras gets usable data and no scary warning. Reachable as load_test_data_eels / _eds / _ebsd.
Closes the framework half of #49. requires_package: hides a toolbar action until its optional extra is importable, mirroring requires_vectors — so a user without spyde[eels] never sees a button that would raise ImportError on click. Resolved with find_spec, so it never imports the package and never pays its import cost; cached, since an extra cannot appear mid-session. Applied in BOTH filter paths. get_toolbar_actions_for_plot and _action_matches_plot duplicate the gate list, and adding a key to one renders a button that never dispatches. A test asserts both contain it. Extras: eels (exspy), ebsd (kikuchipy), atoms (atomap), all. Deliberately not core deps — each pulls a large tree and the frozen app must stay installable for someone who only does 4D-STEM. A CI job installs spyde[all] so the gated code actually executes; the default job proves the app still works with none of it. NOT adding numexpr, and pyproject now says why. Hyperspy warns "Numexpr is not installed ... which is slower to calculate model" on every model build, but at spectrum sizes that advice is backwards. Measured on a 1024-channel multifit: absent 110 spectra/s numexpr, 1 thr 77 numexpr, 4 thr 75 numexpr, 48 thr 70 numexpr's per-call setup and thread dispatch dominate arrays of ~1k elements. The warning will keep inviting someone to add it, hence the comment.
Ports hyperspy's model concept and makes it fast. #51, #52, #53. Lives in spyde/fitting/, NOT spyde/models/ — that name is already the neural disk detector. Unrelated meanings of "model". spec.py ModelSpec: serialisable, round-trips against hyperspy's BaseModel.as_dictionary(), so a model built here opens in plain hyperspy. Packed parameter order IS the API — column j always means the same parameter. components.py batched autograd-differentiable ports of hyperspy's components; no Python loop over pixels anywhere engine.py batched Levenberg-Marquardt over the whole nav grid Why it works: n (free parameters) is tiny even when P and C are large, so the normal equations are (P, n, n) — a batch of small solves torch.linalg does natively. The one large object is the (P, C, n) Jacobian, so P is chunked. Jacobian by jacfwd not jacrev: forward-mode costs one pass per INPUT (n~10), reverse one per OUTPUT (C~2000). One implementation, not two. The "CPU fallback" is this code with device="cpu"; a separate scipy path would be a second thing to keep in agreement with hyperspy, and hyperspy already IS the reference — the acceptance test asserts parameter parity with multifit directly, because "it converged" proves nothing when you are replacing a reference implementation. Measured, 1024 spectra x 1024 channels, PowerLaw + 3 Erf (11 free params): hyperspy multifit 214.80 s 4.8 spectra/s batched cpu 27.02 s 37.9 spectra/s 7.9x batched cuda 3.97 s 257.8 spectra/s 54.1x Two numerical traps found by measurement, both documented at the code: - Column scaling is load-bearing, not a refinement. A PowerLaw fits A ~ 1e6 alongside r ~ 3, so cond(JtJ) ~ 1e12 and float64 Cholesky loses nearly every digit in the small-eigenvalue direction. Unscaled: 247 iterations for a TWO parameter fit, stopping 1.5-3.5% short of multifit. Scaled: 46 iterations to chisq ~1e-34. - Convergence must be a GRADIENT test. Testing step size alone reads "stuck against a wall with large lambda" as converged; requiring an ACCEPTED step instead never fires at all, because at the optimum nothing improves. Convergence on hard many-parameter models is still seed-limited (hyperspy hits maxfev on the same model) — that is what seeded propagation (#54) is for.
Answers "is PyTorch really the right way to fit?" — the honest answer was
"yes as a vehicle, no as I was using it". Profiling the solver:
residual evaluation 3.4 ms
jacfwd Jacobian 51.6 ms <- 94% of the solver
Forward-mode AD costs one forward pass per parameter, and these components have
closed-form derivatives that mostly reuse the value already computed
(df/dA IS the shape for every linear amplitude). So components now carry
analytic gradients and the engine builds the Jacobian directly; autodiff stays
as the fallback for any component without one.
batched cpu 37.9 -> 105.9 spectra/s (2.8x)
batched cuda 257.8 -> 361.6 spectra/s (1.4x)
The GPU gains less because it is now dispatch-bound, not compute-bound — the
remaining lever there is kernel fusion, not more math.
Also: the chunk heuristic was starving the GPU. Capping the Jacobian at 2**22
elements chopped a 1024-spectrum batch into pieces of 372 and cost a third of
the throughput (178 vs 267 spectra/s). Raised to 2**26 so typical scans run
unchunked.
Correctness is unchanged and still defined by hyperspy: every analytic gradient
is asserted equal to autodiff's, which is the ideal oracle since it is derived
mechanically from the value function and cannot share a mistake with a
hand-written formula.
seeding.py implements #54 — coarse strided fit, propagate to nearest neighbour,
one batched refine. SAMFire's insight (a neighbour's result is the best start)
without SAMFire's per-pixel scheduler. Only CONVERGED coarse fits propagate: a
failed one has wandered somewhere unphysical and seeding from it spreads the
failure rather than the answer.
Two benchmark corrections, both of which had been quietly measuring the wrong
thing:
- With exspy installed, create_model() returns an EELSModel that ALREADY has a
background, so the reference model had TWO PowerLaws while the batched side
had one. Both sides must fit the same model or the comparison means nothing.
- The EELS benchmark model cannot represent a hydrogenic edge, so neither
engine converges and it was timing the iteration cap. Added a well-posed EDS
case (gaussian peaks, which the library expresses exactly) as the default;
EELS stays as the deliberately-hard timing-only case, documented as such.
On the well-posed case: 57.6% converged, 361 spectra/s, 11.6x multifit.
Gaussian2D through the SAME engine, not a second code path. An image is flattened to H*W sample points exactly as a spectrum is C channels; the only difference is that the "axis" carries (x, y) PAIRS instead of scalars, handled by one branch in TorchComponent._split. Packing, the Jacobian, bounds, fixed parameters and the LM solve are all the 1-D machinery unchanged — and the tests assert that rather than assume it (bounds and fixed-parameter tests are repeated in 2-D precisely because a different shaping path is where they would break). image_coordinates() builds the coordinate grid, row-major so it lines up with a .ravel()'d image. The test image is deliberately NON-SQUARE (24x28): a transposed coordinate grid would run happily on a square one and put every peak in the wrong place. As in 1-D, A is the VOLUME under the surface rather than the peak height — transcribed from hyperspy's expression, with parity, batching and analytic-vs-autodiff gradient tests to keep it honest. This also unblocks Wave 4: refining atom positions IS a batched 2-D gaussian fit (#77).
Both shaped their output with `expand_as` / `expand`, which forces the
parameter block's P down to the axis's leading 1 and raises for P > 1:
RuntimeError: The expanded size of the tensor (1) must match the
existing size (4) at non-singleton dimension 0
Plain broadcasting (`p + 0.0 * x`) gives (P, C) for both a shared axis and a
per-position one.
Found by fitting a real EDS composition model, whose background is a
Polynomial — not by the component tests, because every one of them used a
SINGLE spectrum, where expand_as happens to work. That is the actual defect
here: a batched component library whose tests never batch.
Closed with TestEveryComponentBatches, which runs every component (and
Polynomial) with P=4 and asserts the value shape, the gradient shape, and that
row 1 reflects ITS OWN parameters — a broadcasting slip that returns row 0 for
everything would still have the right shape.
Wave 2's headline: type which elements are present, get a populated model. exspy owns the spectroscopy — edge and line energies, GOS edge shapes, family relationships — because reimplementing it would mean diverging from the numbers the community publishes. This module owns the plumbing: - one call for EELS and EDS alike returning a ModelSpec, so the result feeds the batched engine, the wizard and the renderer through one type; - pruning to the measured range. A component with no data under it is NOT harmless: its amplitude is unconstrained, so the optimiser trades it against everything else and degrades the parameters that ARE measurable. exspy will happily add a Cu-L line at 0.93 keV to a spectrum whose useful range starts at 2 keV; - an honest report of whether the batched engine can fit the result. That last point is asymmetric today and worth knowing: EDS -> Polynomial + Gaussian -> fully supported, gets the GPU path NOW EELS -> EELSCLEdge (GOS table) -> no batched port yet (#63), falls back to hyperspy, which is correct but slower Reported in the returned info rather than left as a surprise. Two exspy behaviours that do not do what their names suggest: - `add_lines` APPENDS, so restricting needs `set_lines`. - Even then, the EDS model expands each ELEMENT into its whole family (selecting Fe_Ka still builds Fe_Kb, Fe_La, Fe_Ln, ...) regardless of metadata.Sample.xray_lines — so honouring only_lines means filtering the built spec by name. The source signal is never mutated; a user may be exploring several compositions.
Two bugs of the same shape, found by round-tripping a composition model. to_model() rebuilt every component BARE, which breaks for anything that takes constructor arguments: - EELSCLEdge raises outright (element_subshell is required), so no EELS model could be turned back into hyperspy at all. - Polynomial is worse because it does NOT raise. Rebuilt bare it comes back at its DEFAULT order, so exspy's order-6 EDS background silently lost a3..a6 — to_model skipped the parameters that no longer existed and returned a different model that still fits and still looks plausible. ComponentSpec now carries init_args, captured by from_model via a small _INIT_ARGS table and used when rebuilding. An unrebuildable component gets an error naming the table to add to, rather than a bare TypeError from inside hyperspy. The original round-trip tests missed both because they only ever went model -> spec, never spec -> model with a component of this kind. Fixing that surfaced a third: EELSCLEdge.fine_structure_coeff is a VECTOR parameter with 8 elements, and ParameterSpec stored a single float, so hyperspy rejected the assignment on length. Vector parameters now round-trip faithfully (value becomes a list) but are excluded from the PACKED vector via ComponentSpec.scalar_parameters — the solver puts one scalar per column, and no component with such a parameter is fittable by the batched engine anyway (components.supports already says no). components.py uses the same scalar view so the two cannot drift apart.
…oo (#63) An EELS core-loss edge is not a formula — EELSCLEdge looks its shape up in a GOS table, which is why it had no batched port and why an EELS model fell back to hyperspy's one-pixel-at-a-time fitting while an EDS model (gaussians) got the GPU. The way out is to notice what is actually being fitted. Across a spectrum image the edge SHAPE is the same everywhere — atomic physics, fixed by the element and the microscope. What varies pixel to pixel is HOW MUCH of the element there is and exactly WHERE its edge starts. So the edge is sampled once on the signal axis and replaced by a tabulated component fitting intensity (linear) and onset_shift, batched like everything else. The approximation, stated rather than buried: fine structure and effective angle are FROZEN at the values the table was sampled with. They are what make an edge a lookup in the first place, and refitting them would mean re-sampling the table every iteration — the per-item Python work the batched engine exists to avoid. tabulate_model() reports what it froze. Right trade for quantification; wrong one for fine-structure analysis, which is why it is an explicit call and not automatic. Two deliberate choices in the component: - Outside the table the value HOLDS at the end sample rather than extrapolating. Extrapolating a GOS tail would invent signal where the measurement has none. - The onset-shift derivative is a central difference over one channel, not the exact piecewise-linear slope. The exact one is a step function that jumps at every segment boundary, so LM chatters as the shift crosses a channel. It differs from autodiff by ~1.5% of the gradient's own scale, only within a channel of a kink, and the test states that as the property rather than asserting an equality that is not true. Tested end to end: the table reproduces hyperspy's edge to 1e-9, a fit recovers both a known intensity and a known 12 eV onset shift, and on the synthetic SI the fitted O_K intensity tracks the concentration map the data was built from.
Two steps, kept separate because they fail differently. element_intensity_maps() is the BRIDGE: fitted parameters -> one intensity map per element, reading the component names the model was built with (Fe_Ka, O_K) and summing a family's lines. Pure array work, no physics, no optional extra. One bridge serves EDS and EELS because the naming shape is the same and the intensity parameter is looked up per component kind (Gaussian's A, TabulatedShape's intensity) — otherwise the EELS path silently produces no maps. quantify() is the PHYSICS: relative, Cliff-Lorimer, or EELS cross-sections. The split matters because the bridge is where a naming or ordering mistake sends iron's intensity into copper's map, and that is SILENT — every downstream number stays plausible. A wrong k-factor is at least a number a microscopist can sanity-check. They deserve separate tests, and the bridge gets more of them. Three things that would otherwise be quiet: - Negative fitted intensities are clamped at SOURCE. A fit can drive a line slightly negative on noise, and a negative "amount of an element" is not just unphysical for that element — it corrupts the normalisation for every other one. - Missing k-factors default to 1.0 and are REPORTED, because defaulting silently makes an uncalibrated number look calibrated. - The result records that composition is RELATIVE to the fitted elements. Cliff-Lorimer cannot know about an element you did not fit, so leaving one out does not cause a small error — it redistributes that element's share across everything else. End to end on the synthetic EDS SI: composition -> model -> batched fit -> quantify recovers the generator's concentration maps at r > 0.9 per element. "Quantification works" means it recovers what we put in, not that the numbers sum to 1.
…55, #56, #58) The first UI on top of spyde.fitting. Two tabs, matching the Orientation caret's shape: Model builds the model component by component, Run fits it and commits the maps. Layout is horizontal by preference — a component's parameters sit side by side so the caret grows wider rather than taller. #56, the picker: components are added from a "+ Component" POPUP that shows each one's SHAPE, not just its name. The backend samples every offerable component at defaults over the CURRENT signal axis and sends a normalised polyline the renderer draws as a sparkline. Seeding matters — a default Gaussian sits at 0 with sigma 1, which is off-screen on a 200-800 eV axis, so every peak shape would otherwise preview as an identical flat line. #58, the maps: Commit ships one integrated-area map per component through commit_result_tree, which already gives the strain-style toggle (click one, cmd-click to tile). No new display code. The preview is ONE spectrum, not the grid: a model edit has to feel instant, and fitting 65k positions per keystroke would not. fit_run is the only thing that touches the whole scan. The caret rebuilds itself from `fit_state` and never edits a local copy — the backend owns the model, and a caret that mutated its own would drift out of step the moment an edit was rejected, then show a model that is not the one being fitted. TWO REAL BUGS that only running the app could find; both handler-test suites pass either way: 1. A click on a caret control was SILENTLY LOST. Capture-phase listeners showed `mousedown -> the button's span, mouseup -> mdi-area`: the mousedown bubbled to the window, re-rendered the subwindow subtree, and replaced the control between down and up, so React never saw a click. The add simply did not happen, with no error anywhere. Fixed by keeping mousedown inside the caret. (dispatchEvent worked throughout, which is exactly what makes this look like a test problem rather than a UI one.) 2. `emit` was imported as `from ipc import emit`, which binds the original at import time — the test fixture patches `ipc.emit`, so every message assertion would have silently observed nothing. Imported as a module now. Backend: 26 handler tests (including the StrictMode open/close/open contract). UI: fit_wizard.spec.ts drives the real app end to end — toolbar button, palette sparklines, model building, a parameter edit echoed back from the backend, the fit, teardown/reopen, and Commit opening a window with a chip per component. Screenshots at every stage.
#57 — every positioned component gets on-plot handles: a POINT handle driving its centre and height, and a RANGE band driving its width. Dragging writes straight back into the model and redraws, and the numeric fields refresh from the same state, so both directions agree. Ported from anyplotlib's interactive-fitting example. The conversions are the substance. For most components the fitted amplitude is an AREA, not a height, so storing a dragged y straight into `A` would jump the curve by sigma*sqrt(2*pi) the instant it was touched. A width drag likewise holds the peak HEIGHT fixed while changing sigma — otherwise the curve moves away under the cursor, which reads as it fighting the drag. PERSISTENCE: the model and the fit now live on the TREE (`tree.fit_spec`, `tree.fit_result`), not on the caret controller. They are results, and the ownership map in actions/README.md §3 puts results on the tree. A model costs real effort to build and a fit costs minutes; closing the caret to get it out of the way must not discard either. tree.close() still disposes both. THREE MORE BUGS THE RUNNING APP FOUND, all invisible to the handler tests: 1. The live preview NEVER DREW. anyplotlib's add_line takes `label`, not `name`, and the TypeError was swallowed by this method's own except. The previous spec passed end to end while the plot showed nothing — I only caught it by looking at the screenshot. 2. Preview lines ACCUMULATED: remove_line takes an id or a Line1D handle, not a label, so every redraw stacked another line and the legend filled with `fit_preview` entries. The handle is kept now. 3. A new component arrived with amplitude 1 against counts of 1e5 — the curve was a flat line on the axis, the handles sat at y=0, and the fit started five orders of magnitude from its answer. All three read as "the model does nothing". scale_to_data() now sets the linear amplitude from the spectrum itself, generically: evaluate at unit amplitude, scale the peak to a fraction of the data's range. 39 backend tests; the e2e now also asserts the model survives close/reopen, and its screenshot is where the preview curve and the handles are actually checked — nothing else in the suite can see them.
Owner
Author
#57 drag handles + a persistent model#57 — every positioned component now gets on-plot handles: a point handle driving its centre and height, and a range band driving its width. Dragging writes straight back into the model and redraws; the numeric fields refresh from the same state, so both directions agree. Ported from anyplotlib's interactive-fitting example. The conversions are the substance. For most components the fitted amplitude is an area, not a height — storing a dragged Persistence — as you asked. The model and the fit now live on the tree ( Three more bugs the running app foundAll invisible to the handler tests, and the first is the one worth dwelling on:
Verified39 backend tests, and the e2e screenshot now shows what it should: one |
…background Three things, two of them from using it. "FIT SPECTRUM" (requested): fits only the spectrum on screen and writes the result back into the model, so the next nudge starts from a fitted position. Building a model is a loop — place a component, look, adjust — and fitting the whole scan to check one guess is the wrong unit of work: it costs seconds to minutes and answers a question about one pixel. Same engine, same spec, one row of data. It deliberately does NOT set a scan result: one spectrum is not a map, and offering Commit after it would export a component map built from a single pixel repeated everywhere. The other button is now "Run scan", so the two read as what they are. HANDLES MOVE IN PLACE instead of being rebuilt (the "gaussians don't move" report). fit_set_param used to call sync_widgets(), which tears down and recreates every widget — on every keystroke, several times a second. That makes the handles flicker and can pull one out from under the cursor mid-reach. update_widgets() now moves them via the widget's own set(), and rebuilding is reserved for a changed component LIST. A drag skips the handle being dragged, because writing a position back to the widget under the user's finger fights the drag. I could not reproduce a parameter edit failing to move the CURVE — in the app, typing a new centre moves both curve and handles — so the churn above is my best explanation for what was seen. If it persists, which parameter and how many components would narrow it. #65: fit_from_composition — type the elements, get a populated model, then drag the lines. EELS models are tabulated on the way in (#63), or the model is correct but falls back to hyperspy's per-pixel fitting, which is the difference between seconds and minutes on a real scan. #64: background_action — drag a window over a background-only region; the model is fitted to THAT SPAN ONLY and extrapolated across the whole axis, which is the entire point of a background model (it says what the background would have been under the peaks, where it cannot be measured). Fitted per navigation position, because a spectrum image has a different background per pixel and subtracting one pixel's fit everywhere would imprint a thickness map on the result. Seeded from the data INSIDE the window: a pre-edge window sits far below the peak, so scaling to the global max starts it an order of magnitude high. 46 backend tests; the e2e passes end to end.
…erly The drag did nothing to the model, and the reason is embarrassing: anyplotlib hands the dragged widget back on `event.source`, so the position is `event.source.x`. I read `event.x`, which is None — so every drag silently updated nothing while the handle still moved on screen, because the widget draws itself. That is exactly why it looked like "the curves don't move". Rewritten to follow the interactive-fitting example rather than approximate it: - Lines are updated with Line1D.set_data IN PLACE. Removing and re-adding a line per drag frame is heavy and does not repaint mid-drag. - ONE LINE PER COMPONENT plus a dashed sum, instead of a single summed curve. With several overlapping peaks, a sum tells you the total is wrong but not which component to grab. - The `_syncing` guard, because moving the handles in response to a drag re-enters the handler and the widget and model chase each other. - The range widget uses style="fwhm" at y=height/2, and the point widget drops its crosshair — the example's geometry, which reads as a peak handle and a width band rather than two unrelated markers. Verified with a REAL pointer drag in the app, not a synthesised event: grabbing the handle and dragging left moves centre 500 -> 296 eV, and the curve, both handles and the caret's fields all follow. A screenshot confirms it. "Fit spectrum" moved to the Model tab (requested). That is where the loop is — add, look, nudge, fit this one, look again. Run scan stays on its own tab as the separate expensive thing you do once the model is right. The drag test now pins the `event.source` contract directly, so this specific mistake cannot come back silently.
…d spectrum Reported as "Fit spectrum doesn't work at all", and the way it failed is worse than not working: it said CONVERGED and drew a model at about half the height of the data. current_spectrum() reconstructed the on-screen spectrum from signal.data plus a navigator index. The index was not where it expected, so it fell through to the mean over navigation — and a fit against the mean converges perfectly well. The status was true; the answer was about a spectrum nobody was looking at. plot.current_data is the authority: it IS the array the plot is displaying, already resolved through whatever navigator, region-integration or derived-view path produced it. Read that. The navigation mean survives only as a stand-in for the PREVIEW before the first frame lands, and it logs when it is used. Why nothing caught this: the handler tests build their own signal and never have a navigator, so the fallback WAS the displayed spectrum for them. It needed the real app, a real navigator position, and an assertion about the fitted VALUES rather than about a status string. fit_two_gaussians.spec.ts is that test, on the dataset this is meant for (tutorial_spectroscopy = hyperspy's two_gaussians). It asserts the model's peak height, computed from the fitted parameters — a HyperSpy Gaussian's A is an AREA, so the height is A/(sigma*sqrt(2*pi)) summed over components. Fitting the navigation mean gave ~460 against data peaking at ~1050; the fix gives ~1050. The threshold separates those two outcomes without pinning an exact fitted value, and a second assertion stops a degenerate fit that zeroes one component from passing on the other's back. Four backend tests pin the same contract directly, including that a painted spectrum always beats the mean and that a stale/odd-shaped current_data (a dask Future mid-transition) is ignored rather than fitted.
… icon PER-POSITION MEMORY. Each navigator position now remembers its own fitted parameters, on the tree beside the model. Scrubbing back to a pixel shows what was found THERE rather than whatever the last pixel left behind. The store is sparse — a dict keyed by position, not a (P, n) array — because it fills in as the user explores, and a full allocation would claim every position had been fitted. It is cleared whenever the component list changes: the vectors are POSITIONAL, so after an add or remove a stored sigma would arrive as an amplitude and nothing would look obviously wrong. ADAPTIVE FITTING (an "Adaptive" toggle). Moving the navigator recalls a stored fit if there is one, and otherwise — with adaptive on — fits the new spectrum seeded from the model as it stands, which is a neighbouring position's answer and therefore a good starting point (the same reason seeded propagation works for a whole scan, #54). Recall wins over refit: same computation, already done. Driven from pointer_UP on the navigator, not pointer_move, or it would queue fits faster than they complete. This also fixes the thing that made the earlier "Fit spectrum" bug possible: current_indices lives on the navigation SELECTOR, not on the Plot. Reading it off the Plot always returned None. There is now a real accessor and a test that exercises it against a REAL session rather than only a fake. SMOOTHER DRAGS. pointer_move and pointer_up are wired to separate handlers because they should do different amounts of work. A move now redraws the curve and nothing else; moving the partner handle and re-sending the whole model both cross the IPC boundary, and doing either at pointer rate is what made the curve lag the cursor. Both happen once, on release. Remove Background gets its OWN icon — it was borrowing Fit's, so the two buttons were indistinguishable. Drawn for 20 px rather than for the artboard: only about three strokes survive at button size, so it spends them on a straight baseline and a down-arrow, a different silhouette from Fit's peak. The first attempt was a dashed background curve — handsome at 150 px, mush at 20, which was the whole complaint. Checked by rendering at 20/28/150 px on the toolbar's own background before committing. 66 backend tests, both e2e specs green.
THE DRAG REGRESSION WAS MINE. Splitting the preview into one line per component went from ONE line update per pointer frame to N+1, and Line1D.set_data is not cheap bookkeeping: it recomputes the axis range and pushes the WHOLE plot state to the renderer. Two gaussians therefore meant three full state pushes per frame where there had been one. That is why it got worse after the last change, and the "smoothness" work in that commit did not touch it. refresh_lines now writes every line's data and pushes ONCE, so it is one push per frame regardless of component count — fewer than before the per-component rewrite, not merely back to parity. It reaches into anyplotlib's line entries because there is no public batched update; the public per-line path stays as the fallback, so an upstream change costs speed rather than correctness. FIT COVERAGE, so the store is visible: the caret shows "N/M fitted" with a filled marker when the CURRENT position has an answer. Without it there is no way to tell a position you skipped from one you fitted, and "why didn't adaptive fill this in" has no answer on screen. WHAT I COULD NOT MEASURE, stated plainly. I tried to quantify drag smoothness through Playwright and produced a number that was mostly harness: a bare page.mouse.move costs ~16 ms here, so 30 synthetic moves take ~600 ms with NO caret open at all, against ~850 ms with one. My earlier "~28 ms per move" was therefore ~70% Playwright. On that bad number I added a redraw coalescer; it showed no improvement, and I have removed it rather than ship an unverified optimisation that also drops frames. The batched push stays because it is a strict reduction in work that is true independent of any timing. Measured for real, on the side: a single-spectrum fit is 33 ms (14 LM iterations). float32 is SLOWER than float64 here (203 vs 109 ms) — worth recording so nobody tries it again. The 109 ms in that comparison was dominated by BUILDING the spec, which goes through sympy in hyperspy's Expression components; reusing a spec is 33 ms.
Two things the caret got wrong, both reported from the app. **Specs were rebuilt constantly.** Constructing a hyperspy component costs 20-110 ms — Expression components lambdify their formula through sympy on every instantiation, with no caching upstream — and the picker, "add component" and the background wizard each built their own. The palette therefore cost 6.2 s on first open and ~130 ms on every open after, which is what "the fitting is slow" actually was: the caret opening, not the fit. One prototype per kind now, one ComponentSpec read off it, deep-copied per use (`new_component_spec`), and the sampled palette cached per axis. With imports already paid, as the app pays them: first open 681 -> 3 ms on a new dataset, 0.04 ms reopening the same one. The remaining first-open cost is sampled off the main thread, so the caret appears immediately and the sparklines fill in. **Handles came off the component.** Backgrounds had none at all — a PowerLaw or an Offset was editable only through the caret's number boxes. They now carry ANCHOR points, and dragging one solves the component through it in closed form, so the curve passes exactly under the cursor (the other anchors are held, so it does not pivot out from under them). And the partner handle is moved every pointer frame instead of only on release: a widget `set` is a targeted push of a few fields, unlike `fit_state`, which stays on release. Two real bugs surfaced while verifying the anchors in the app, neither visible to the handler tests: * A PowerLaw's `origin` was seeded to the axis midpoint, because the rule that centres a gaussian also matched the name. The background was then identically zero over the left half of the spectrum, with its anchors outside its own domain. * Scaling a background by its PEAK matches a power law's singular left edge, leaving ~3e-5 across the spectrum; by its median LEVEL it puts 3.7e10 at that same edge. Neither is a scaling problem — a background needs its shape solved from the data, which `seed_background` now does through the same closed form, sampling a low percentile over a wide window so it lands on the low envelope rather than on whatever peak sits under an anchor. Verified in the app: `fit_handles.spec.ts` reads each handle out of the panel state and requires it to sit on its component's own line, drives a live pointer_move and asserts the width band tracked it, and drags a background anchor. Adds a `_spyde_test_panel_json` hook so a spec can read the lines as well as the widgets — a screenshot cannot tell "on the curve" from "a few pixels off it".
`exspy`, `kikuchipy` and `atomap` (plus `ase`, which atomap pulls in) are declared as optional extras in pyproject but were never resolved into the lockfile, so `uv sync --extra ebsd` had to re-resolve them every time.
…ehind Three things wrong in `fit_run`'s completion, all of the same shape as the handle drift: the result landed in the model but nothing else was told. * The store was never filled, so the caret said "0/1024 fitted" immediately after fitting all 1024, and navigating afterwards refit positions it had just solved. `record_run` now records every CONVERGED position — a failed fit is not an answer to remember, and leaving it out lets a later adaptive pass retry from a better neighbouring seed. * The handles stayed put while the curves jumped to the fitted values. * The post-run preview read `current_indices` off the PLOT, where it does not live, so `idx` was always None and it silently showed position 0's parameters instead of the displayed one. Same mistake that made "Fit spectrum" fit the navigation mean. `position_of` is the exact inverse of the `ravel_multi_index(reversed(idx))` the run path uses; it is tested on a NON-square scan, because a transposed key is invisible on a square one. `record_run` is a method rather than inline in `_done` so it is reachable without the worker marshal, which does not turn under pytest — the existing run test sidesteps the same way. Verified in the app: the spec now runs the scan and asserts the coverage readout moved off zero and that the handle positions changed (347/1024 on the tutorial spectroscopy dataset, handles on the fitted curves).
…t separate Every component is seeded mid-axis, so a second gaussian arrived with the same centre AND the same sigma as the first: two IDENTICAL Jacobian columns (measured corr(centre1, centre2) = 1.000000, cond(J) = 1e16). A perfectly degenerate pair can never separate — the solver moves both the same way forever — so it grows one and shrinks the other. On two genuinely separated peaks it was worse than cosmetic: the pair ran away together to sigma = 170 on a 100-wide axis, chisq 6.3e7, with neither peak found. Two changes, and the measurement says both are needed: * `spread_repeats` places repeats of a kind symmetrically about the seed, spaced by ONE of their own widths. Symmetric so a genuinely co-centred pair still starts co-centred; one width because wider is worse, not better (at two widths the co-centred case fails outright, chisq 6.3e5 against 1.3e-21). * `clamp_to_axis` bounds a peak's position to the axis and its width to the axis span. A component centred off the data or wider than all of it is not modelling anything the user can see, and those are exactly the runaway directions. Backgrounds are exempt: a PowerLaw's `origin` is a reference point deliberately placed outside the axis. Neither alone is enough. Bounds without the spread only pin the degenerate pair at the cap (chisq 6.3e7, unchanged). The spread without bounds still diverges on a noisy near-overlapping pair (chisq 1.1e8 against 127). Measured over six synthetic arrangements plus two positions of hyperspy's real `two_gaussians`: three catastrophic failures fixed, and the two real positions reproduce their previous fits EXACTLY (chisq 5.245e5 and 1.795e5, unchanged) — which is the point of spacing by one width rather than relocating. No seed is universally right on a non-convex problem; a wider spread trades one failure for another. That is what the drag handles are for.
"Once you try to fit a spectrum then you can't move either component."
`spyde:figure_event` carries `{figId, event}` and NO window_id — SpyDEContext's
re-broadcast never had one. The Fit caret's navigator listener guarded with
`detail.window_id === windowId`, which compared `undefined` and so never fired.
Every pointer_up on the caret's OWN plot therefore sent `fit_navigated`,
including the ones from its own drag handles.
Before a fit that was invisible: `fit_navigated` finds nothing stored and just
redraws. After one, the position IS stored, so it RECALLED the fitted values
and overwrote the drag the instant it landed — the handle snapped straight
back and the component could not be moved at all. Which also made the fit
un-improvable, since nudging a component and refitting is the entire workflow.
Filtered by figId instead, against this window's own figures.
Also reorders `_on_widget_drag`: handles first, then lines. `refresh_lines`
ends in a FULL panel push that serialises every widget's geometry, while
`update_widgets` issues TARGETED per-widget pushes that travel on the event
channel. Lines-first meant the frame's full push carried the OLD partner
position and the new one went out targeted-only. Handles-first is one push
per frame instead of one plus one per moved handle, and it is what lets a test
see the partner track at all.
Verified in the app by fit_drag_after_fit.spec.ts, which drags a component
before a fit (the control), fits, drags again, and requires the model to both
move AND stay moved.
Two gaussians are exchangeable — nothing in the model says which is the broad one — so fitted position by position they land in whichever slot each fit happens to pick. Measured on hyperspy's two_gaussians: the first component was the broad peak at only 43% of positions, its sigma ranging 2.2 to 25.7 across the scan. Which is what "it's literally suppressing one component to 0" was. Scrubbing the navigator made the SAME named component jump between the broad peak and the narrow one, so its amplitude dropped by an order of magnitude at every swap (10259 -> 1509 between two positions, verified in the app). And a committed component map would have been a checkerboard of two different peaks, not a map of anything. The fit itself was never the problem: 97% of positions already reach the noise floor, median chisq 0.991x what the TRUE generating model achieves on the same noise. Only the labelling was wrong. `relabel_scan` decides the ordering ONCE for the whole scan and applies it everywhere: find the parameter that genuinely separates the components (here sigma, 2.5 against 25.5) and sort by it. Wired into both the cold batched path and `fit_seeded` — the latter also relabels its COARSE grid before propagating, or each coarse cell hands out its own arrangement and undoes the seeding. Two things I tried first and rejected on the measurement: * Serpentine neighbour-matching — walk the grid in boustrophedon order and keep the arrangement closest to the position you just came from. It drifts: matching is chained, so one bad match (a position where the components are genuinely similar) flips the convention for everything after it. 425 of 1024 positions were permuted correctly and agreement went DOWN, 43% to 28%. * Scoring separability by how far apart the sorted slots' means are. Sorting ALWAYS separates: two components drawn from the same distribution still give a "smaller" slot and a "larger" one. The score now measures DISJOINTNESS (5th percentile of the upper slot minus the 95th of the lower), and refuses to reorder when they overlap — an invented identity is worse than an inconsistent one, because the map then looks meaningful when it is not. Relabelling changes no residual: median chisq is identical before and after (216748), and a test asserts the summed model curve is unchanged to 1e-12. Verified in the app by fit_navigate.spec.ts: fit the scan, move the navigator between two positions with different answers, and require both that the model follows the navigator AND that each named component is still the same peak.
…efits the poor
Renames "Run scan" to "Fit all Spectra" and makes it deliver what the name
promises.
**It plots the result.** One map per component PLUS chi squared, opened when
the fit finishes rather than only on Commit — the answer used to be reachable
only by scrubbing the navigator a pixel at a time. chi squared belongs beside
the components: a component map read without it can be a picture of the model
falling over rather than of the sample. The automatic window is a preview and
is REPLACED on each fit; "Keep these maps" (was "Commit component maps") makes
one that survives.
**Every position is stored, not only the converged ones.** A position left out
of the store showed whatever the last one left in the model, which reads as the
fit being stale — so moving the navigator now always shows THAT position's fit.
A poor fit is still that position's answer; it is flagged instead of withheld.
**The poor positions get refit, from their best neighbour.** A failure is
almost always a pixel whose neighbours succeeded, and a spectrum image is
smooth, so next door is nearly the answer here. Runs automatically at the end
of a fit and again from a "Refit poor (N)" button. Measured on hyperspy's
two_gaussians: positions worse than 1.5x the noise floor go 27 -> 1, total
chisq down 9.7%, at 0.1 s a pass against 2.8 s for the fit itself. Two passes
reach the fixed point, so the loop stops on "no improvement" rather than a
count, and a refit is KEPT only where it lowers chisq — which is what makes it
safe to run unattended.
"Poor" is local: chisq scales with the counts, so a bright pixel legitimately
has a bigger one and any global threshold flags a whole region or nothing. A
position with fewer than two neighbours is not judged on chisq at all — the
median of ONE number is that number, so at a scan corner the test would reduce
to "worse than the single pixel beside it", which a steep gradient satisfies on
its own.
Two bugs found while verifying this in the app:
* anyplotlib fires a `pointer_move` on a PROGRAMMATIC widget move as well as a
dragged one, so every handle written by `update_widgets` re-entered the drag
handler, which treats it as a user edit and discards the fit. Moving the
handles onto a freshly fitted model therefore threw that fit away: the maps
would not open ("run the fit before committing") and Commit vanished.
* The `update_widgets` / `draw_preview` order was wrong at all five call sites
for the same reason it was wrong inside the drag handler: the redraw's full
panel push carried the OLD handle geometry.
Verified in the app by fit_all_spectra.spec.ts: the button's name, that a
result window opens on its own with a chi-squared map among the chips, that
coverage reaches 1024/1024, that "Refit poor (14)" takes it to 0, and that
refitting replaces the maps window instead of stacking another.
…y does A HyperSpy parameter already carries `parameter.map` — a structured array over the navigation grid with values / std / is_set — and that IS "hold the parameters for the model at each position". It is what `m.multifit()` fills and what `m.store()` persists. SpyDE was keeping a dict beside it instead. `FitStore` (spyde/fitting/store.py) wraps a live model built from the spec and writes into those arrays. All SpyDE adds is the packing: the batched engine works in `ModelSpec.parameter_names()` column order, so this converts between that vector and the per-parameter maps. Nothing duplicates the storage. What the dict could not do, and this can: * **is_set.** "Not fitted" and "fitted to zero" were the same thing, so a map could not show a hole. They are now distinct, which is what lets the maps open EMPTY and fill in. * **Every position, allocated up front.** The dict grew as the user explored, so the maps could only exist once a fit had finished. * **chi-squared and std** per position, in the same place as the parameters. * **It is a HyperSpy model**, so the result can be handed back to HyperSpy. The behaviour that follows, which is the point: * "Fit components" opens with the CARET, flat, one map per component plus chi squared — verified: the chips read Gaussian | Gaussian 2 | chi squared before anything has been fitted. * Fitting one spectrum fills one pixel of it; "Fit all Spectra" fills them all. One window for the life of the caret, refreshed in place — rebuilt only when the SET of maps changes, because a window's primary map and chips are fixed at creation and refreshing would leave the first component chip-less. * "Keep these maps" still makes a separate, frozen tree. Indexing is the one hazard and it is tested on a NON-square scan: navigator indices are x-first, HyperSpy's maps are y-first like the data, and on a square scan a transposed store looks exactly right while every position holds its neighbour's answer. 24 new tests in test_fit_store.py; test_fit_wizard.py updated to the new contract (132 total). All five fit e2e specs green.
…ed another's fit
The store keyed positions by reversing the selector's indices, reasoning from
`axes_manager.navigation_shape`. The DISPLAY does no such thing: `get_local_frame`
and `_build_nav_lazy_slice` both do `data[point]` with the indices exactly as
given, so the spectrum on screen at crosshair (cx, cy) is `data[cx, cy]`. The
store's whole job is to answer "what was fitted to THAT spectrum", so it has to
index the same way. It did not.
Nothing caught it. The tutorial scan is 32x32, so the shapes matched; coverage
read 1024/1024; every recall succeeded and said so; and the DIAGONAL positions
were even correct. Every other position quietly showed its transpose's fit —
a plausible-looking curve that misses the data. My own store test asserted the
round trip, which a self-consistent-but-transposed convention passes.
Measured in the app, sweeping 25 navigator positions and comparing the drawn
model against the drawn spectrum:
median misfit 19.9% -> 2.9% of the spectrum's range
worst 47.0% -> 3.9%
"Fit spectrum" at the worst position: 47% -> 5.4% becomes 3.9% -> 3.9%
That last line is the reported symptom and now behaves as it should: at an
already-fitted position, fitting again has nothing left to do.
fit_quality.spec.ts is the test that would have caught this and is the one to
keep. Every other fit spec asserts on parameters, coverage or window counts —
all of which were perfectly happy. This one reads BOTH curves out of the same
panel state and compares them, which is what the eye does, and it sweeps
positions rather than trusting whichever one the navigator starts on (the
default position sits on the diagonal and was always fine).
The store tests now pin the convention against the DISPLAY's, not against
themselves.
"A pause and then a snap." Every pointer frame of a navigator drag posts a
`pointer_move`, and the caret sent a `fit_navigated` for each — each costing a
recall, a preview redraw and a full model re-send. They arrived faster than
they were served, so the queue drained AFTER the drag: measured 29 moves over
486 ms with the model settling 1159 ms after the last of them. The pause is the
backlog; the snap is it finally arriving.
Coalesced instead, latest-position-wins: one `fit_navigated` in flight, the
rest collapsed into a single pending flag. Skipping intermediate sends loses
nothing, because `fit_navigated` reads the CURRENT selector position when the
backend handles it — one send after the previous finishes always acts on the
newest position.
The gate opens on `fit_state` coming back, which is the backend's own
completion signal, so it self-tunes to whatever the work actually costs
instead of guessing a throttle interval. A 2-second timer releases it if a
reply is ever lost, so it cannot wedge.
settling time after the last pointer move: 1159 ms -> 113 ms
`fit_navigated` now emits a state on EVERY path. Two of them returned silently
(no components; nothing stored and adaptive off), which would have stalled the
next move until the wedge timer expired — the same pause, reintroduced by the
back door. Tested.
The old code meant to do this with `if (d.event?.type && !/up|click/.test(...))`,
but anyplotlib's payload field is `event_type`, so `d.event.type` was undefined
and the guard never fired. Removed rather than repaired: filtering moves out
entirely is what makes the model jump on release instead of following the
pointer, which is the behaviour asked for here.
fit_drag_latency.spec.ts drives a real 60 fps stream of moves and measures how
long after the last one the drawn model settles.
…s; save models You were right that this was a lifecycle problem, not two separate bugs. **Remove Background had only its backend half.** `bg_open`/`bg_close`, a `PARAMETERS` schema registered under `bg`, a controller whose `remove()` deletes the span — all present and correct. Missing were three renderer pieces, so the action was reached through the plain YAML toolbar path instead: the click fired `bg_open`, which added the span, and nothing ever unmounted to fire `bg_close`. The span stayed for the life of the window. Added the caret (`BackgroundWizard.tsx`), listed it in `WIZARD_ACTIONS`, and put `bg_state` in SpyDEContext's re-broadcast allowlist — without that last one the caret never heard where its own band was and showed 0..0 for it. Nothing in the backend changed; `useWizardLifecycle` now fires the close that the existing teardown was already waiting for, and FloatingToolbar renders one caret at a time so switching actions cleans up too. **The Fit components window had the same shape of leak** — it outlived the caret, still showing a model it had stopped tracking. It closes with the caret now. "Keep these maps" is renamed "Commit components", matching every other Commit, and is still how you get one that stays. The MODEL and the per-position store deliberately survive: they live on the tree, so reopening restores what was built and everything already fitted. **Save / load models, through HyperSpy's own store.** `m.store(name)` puts the components AND every position's parameter map into the signal's `models`, so saving the .hspy/.zspy saves the fit inside it — no SpyDE format to keep in step. `FitStore.restore` reads one back, copying the maps across so `is_set` survives: a values-only format would turn every unfitted hole in a component map into a confident zero. Tested through a real save + `hs.load`. `action_scoping.spec.ts` drives the contract in the app: open Remove Background, see the span, close the caret, see it gone — then again by switching to Fit rather than pressing the X — then the same for Fit's own window and handles. Also pins fit_quality.spec.ts's navigator position before adding components. A component is seeded from the spectrum ON SCREEN, so under load (eight apps back to back) the first paint may not have landed and the whole scan starts somewhere else — it failed at 25% misfit in a batch run and passed at 3.9% alone. That is a real property of seeding from the display, not a flake to paper over, but the test should measure fit quality rather than paint timing.
…the y-axis
Two bugs, both found by dragging it.
**The drag was never heard.** `_on_drag` read `x0`/`x1` off the EVENT.
anyplotlib's `Event` carries x/y and a handful of scalars; a range widget's
geometry is on the widget it hands back as `event.source`. So both were None,
the handler returned at its first check, and the background preview never moved
— not on pointer_move, not even on release. Exactly the mistake the Fit handles
made ("the curves don't move from the drag").
While fixing it, the preview also stopped rebuilding its line every frame:
`remove_line` + `add_line` per pointer frame is heavy AND does not repaint
during a drag, which is the same lesson `rebuild_lines` vs `refresh_lines`
records in fit_action. It updates in place now, and the `bg_state` message
waits for the release rather than going out per frame.
**The extrapolation could take the plot with it.** The background is drawn —
and subtracted — across the WHOLE axis, not just the fitted window, so a
PowerLaw whose `origin` sits on an axis that reaches zero is singular there. A
window on the far side of a peak fitted to 3e9 at the left edge: the y-range
blew away and the spectrum vanished behind it. Now seeded by the same rule as
`fit_action.seed_background` — origin 0 where that is safe (HyperSpy's
convention, and what downstream tools assume), moved a quarter-span below the
axis where it is not. 3e9 -> 2e4 on the same drag.
13 tests in test_background_drag.py, and action_scoping.spec.ts now drives a
real pointer_move on the band and requires the drawn curve to change.
NOTE, not fixed: the preview line still participates in the plot's y-range, so
a background model that suits the data poorly (a PowerLaw over the right half
of a symmetric peak) still squashes the spectrum — 2e4 against a peak of ~1e3
on the tutorial dataset. That is the model being wrong for the data rather than
a defect, but losing sight of your own spectrum to a preview is bad behaviour
and wants the data to own the scale.
…osition
`fit_from_composition` was in the backend and in the staged registry, and
nothing ever sent it: an EELS model was reachable only by placing gaussians by
hand. The Fit caret now offers it, labelled with the elements it would use
("From C, N, O"), and only when there ARE elements — the elements come from
Plot Control's Composition panel via the shared renderer state, so the button
appears the moment they are set.
On an EELS signal `create_model()` auto-populates a background plus one
`EELSCLEdge` per subshell; the edges are then tabulated, because `EELSCLEdge`
has no batched port and without that the fit falls back to HyperSpy's
one-pixel-at-a-time path. On EDS it is `add_lines()` plus a gaussian per X-ray
line. Everything after is the ordinary caret.
**That immediately broke the store, and the fix matters more than the button.**
`FitStore` held its state inside a HyperSpy model, and a tabulated edge exists
only in SpyDE — `to_model` cannot rebuild one, so the store failed to exist for
exactly the models this path produces. It surfaced as `'NoneType' object has no
attribute 'coverage'` the moment you fitted one.
The arrays are now the storage — (values, std, is_set) per position in
`parameter_names()` order, the same triple HyperSpy keeps per parameter, held
once for the model instead of once per parameter. A HyperSpy model is still
built when it CAN be, and `save_as` pushes the arrays into its maps before
storing, so a model HyperSpy can represent still travels with the dataset as
its own. When it cannot, `save_as` says so rather than inventing a private
format only SpyDE could read.
Verified end to end on the bundled synthetic EELS SI: the button appears with
the elements, builds PowerLaw + O_K + N_K + C_K with 3 edges tabulated, and the
model it built fits the spectrum to 0.4% of its range.
Also adds `bg_state` to the protocol's message union — it was relayed at
runtime but missing from the type, so tsc failed.
`EELSCLEdge` CAN be batched, including its fine structure. I claimed otherwise and shipped a private `TabulatedShape` component that fitted intensity and a shift and discarded everything else. That was wrong on both counts. What the edge actually is, from exspy's own `function`: three pieces spliced along the energy axis — a cubic B-spline over the fine-structure window, the integrated GOS cross-section, and a power-law tail — all scaled by `intensity`. The expensive part is `_integrate_GOS`, and it depends on the ELEMENT, the SUBSHELL and the MICROSCOPE GEOMETRY. Not on the pixel. It is the same curve for every spectrum in the scan. Everything left is linear or nearly so: * `intensity` scales the edge — linear, one column. * `fine_structure_coeff` are B-spline weights with fixed knots, so the edge is EXACTLY `base + sum(c_i * basis_i)`. Probing exspy once per coefficient gives those basis curves. Measured against exspy's own `function` for random coefficients: 0.0 relative error, not approximately. * `onset_energy` slides the edge; the precomputed curves are interpolated. So the fine structure is the LINEAR part — the cheapest thing in the component to batch — and I dropped it. I think I saw that `fine_structure_coeff` is a vector parameter, saw that the packed vector excludes vectors, and treated the exclusion as a constraint instead of fixing it. It is carried as one scalar per column now and reassembled on the way back to HyperSpy. **The component stays an `EELSCLEdge`.** `prepare_eels_edges` only attaches the precomputed curves; it does not change the kind, the parameters or the model's identity. So the model remains a real exspy model and stores on its own signal with `m.store()` — which the private component made impossible, and which is how this was found. Also fixes the reporting that went with it: `supports()` now resolves by TRYING to build each component rather than checking kinds against `available()`. The two differ for an edge, which is a supported kind that still needs its curves — a kind-only check called a raw edge fittable and then failed inside the engine. `unsupported()` returns the reason per component, and the composition path surfaces it. 11 parity tests against exspy (base shape, linearity in intensity, random coefficients, onset direction, analytic gradients vs autodiff, the HyperSpy round trip). Verified in the app: PowerLaw + O_K/N_K/C_K at 532/401/284 eV, fitting the spectrum to 0.4% of its range. One bug worth recording: `_edge_spec` first REPLACED the reserved `spyde` key rather than merging it, which dropped `fine_structure_active` — so the rebuilt edge had fine structure off, kept a filled cross-section where the probe left a spline-shaped hole, and disagreed with exspy by the full height of the edge across the whole fine-structure window. Deletes spyde/spectroscopy/tabulate.py and its tests; `TabulatedShape` is gone from components, quantify and `available()`.
CSSFrancis
changed the base branch from
feat/0.3.0-wave2-spectroscopy
to
feat/0.3.0
July 28, 2026 13:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #55, #56, #58. Part of tracker #48.
The first UI on top of
spyde.fitting. Stacked on #84 (needsModelSpec+ the engine).Shape
Two tabs, matching the Orientation caret: Model builds the model component by component, Run fits it and commits the maps. Layout is horizontal by preference — a component's parameters sit side by side, so the caret grows wider rather than taller.
Components are added from a
+ Componentpopup rather than an always-open palette.#56 — the picker shows shapes, not names
The backend samples every offerable component at defaults over the current signal axis and sends a normalised polyline, drawn as an inline sparkline. "Gaussian" and "Lorentzian" aren't distinguishable by name to someone who hasn't met them.
Seeding matters here: a default Gaussian sits at 0 with σ=1, which is off-screen on a 200–800 eV axis — every peak shape would otherwise preview as an identical flat line.
#58 — component maps
Commit ships one integrated-area map per component through
commit_result_tree, which already gives the strain-style toggle (click one, ⌘-click to tile). No new display code.Two real bugs only running the app could find
Both handler suites pass either way — this is the case CLAUDE.md is about.
A click on a caret control was silently lost. Capture-phase listeners showed
mousedown -> the button's span, mouseup -> mdi-area: the mousedown bubbled to the window, re-rendered the subwindow subtree, and replaced the control between down and up — so React never saw a click. The add simply didn't happen, with no error anywhere. Fixed by keeping mousedown inside the caret.dispatchEventworked throughout, which is precisely what makes this look like a test problem rather than a UI one. It isn't: a user clicking that button gets nothing.emitwas imported asfrom ipc import emit, which binds the original at import time. The test fixture patchesipc.emit, so every message assertion would have silently observed nothing. Imported as a module now.Verification
fit_wizard.spec.tsdrives the real app end to end — toolbar button, palette sparklines, model building, a parameter edit echoed back from the backend, the fit, teardown/reopen, and Commit opening a window with a chip per component. Screenshots at every stage, and I looked at them.Worth your call
Closing the caret disposes the fit. The wizard owns the result, so close/reopen discards it and Commit is no longer offered. Consistent with the other wizards, but a fit is more expensive than most wizard state — happy to persist it on the tree instead if you'd prefer.
Still open in Wave 1's UI: #57 (drag components directly on the plot — the anyplotlib interactive-fitting port).