UI Updates on the Wellplate Controller

[beniroquai]

This pull request introduces several improvements and cleanups across the codebase, most notably adding comprehensive documentation for Opentrons-style labware integration in ImSwitch, enhancing the frontend's Axon tab with a new Overview Scan feature, and refining the overview registration workflow. Additionally, it removes obsolete or development-only files and makes small adjustments for better cross-platform support.

Documentation and Labware Integration:

• Added a detailed docs/labware_opentrons.md guide describing how Opentrons-compatible labware is integrated, discovered, and used within ImSwitch, including backend structure, HTTP endpoints, frontend integration, and OME-NGFF metadata output.

Frontend Features and Improvements:

• Added the OverviewScanTab component to the Axon tab's UI, updating AxonTabComponent.js to include the new tab and its contents. [1] [2] [3]
• Improved the overview registration wizard (OverviewRegistrationWizard.js) so the MJPEG stream URL is built whenever the wizard is open, regardless of camera availability, and enhanced metadata persistence by storing the stage position at snapshot time for later autonomous scanning. [1] [2] [3]

Configuration and Dependency Cleanups:

• Removed the .readthedocs.yaml file, indicating a possible shift away from Read the Docs for documentation builds.
• Added a Windows-specific Python launch configuration to .vscode/launch.json for improved cross-platform development support.
• Updated package-lock.json to add a new yaml dependency under the tailwindcss node_modules tree, likely as a transitive dependency.

Maintenance and Cleanup:

• Removed the frontend/scripts/create_test_animations.sh script, which was used for generating placeholder GIFs for acceptance testing, indicating these assets or their creation are no longer needed.
• Deleted the development/test server implementation in frontend/python/server.py, further cleaning up unused or obsolete files.

[beniroquai]

@gokugiant the remaining issues based on yesterdays' and last Friday's session are the following:

WP-3 · Referential Z-Offset / Multi-Objective Focus

Scope: Backend-heavy, minor frontend
Complexity: Medium · ~2–3 hours
Branch suggestion: feature/z-offset-auto-apply

Analysis

The backend already implements Z-offset logic. In ObjectiveController.py, moveToObjectiveThread() (line ~218) calculates z_delta = target_z - current_z and applies a relative Z move. The values come from self._zPositions[slot] which are loaded from config and can be saved via saveCurrentZPosition().

What's broken: The saveCurrentZPosition endpoint (line ~295) works correctly — it reads the live Z from the positioner and saves it. But the frontend wizard (WizardStep4.js) never calls it because the Z display doesn't update (fixed in WP-2.5).

Tasks

3.1 Verify the Z-offset flow end-to-end: After WP-2.5 fixes the Z display, test: calibrate Z for slot 0, switch to slot 1, focus manually, save Z1, switch back to slot 0 — stage should auto-move Z.

3.2 Add per-objective exposure/gain to the config. Currently ObjectiveManager.py stores pixelsizes, NAs, magnifications, objectiveNames per slot but NOT exposure/gain. Add:

• Backend: add exposures and gains lists to ObjectiveManager.__init__ (alongside the existing lists), load from ObjectiveInfo, add to setObjectiveParameters/getObjectiveParameters/_saveObjectiveConfigToFile
• Frontend: add exposure/gain fields in the ObjectiveController component per slot

3.3 Add +/− Z fine-focus buttons to the frontend ObjectiveController. These should call the existing apiPositionerControllerMovePositioner with axis: "Z" and small step values (e.g. ±10, ±100 µm).

WP-8 · Hardware Connection Status & Menu Organization

Scope: Frontend + minor Backend
Complexity: Small–Medium · ~2–3 hours
Branch suggestion: feature/connection-status-improvements

Task 8.1 — ESP32 connection heartbeat

Backend: The server already has /getAvailableControllers (in ImSwitchServer.py line ~117). Add a /ping or /health endpoint that actually tests ESP32 responsiveness:

@api_router.get("/ping")
def ping():
    """Quick health check including hardware connectivity."""
    try:
        stage = master.positionersManager["ESP32Stage"]
        pos = stage.getPosition()  # actual hardware call
        return {"status": "ok", "hardware": "connected"}
    except:
        return {"status": "ok", "hardware": "disconnected"}

Frontend: In the connection status indicator (likely in frontend/src/components/navigation/TopBar.jsx or ConnectionSettings.jsx), add periodic polling:

useEffect(() => {
  const interval = setInterval(async () => {
    try {
      const resp = await fetch(`${hostIP}:${hostPort}/imswitch/api/ping`);
      const data = await resp.json();
      setHardwareConnected(data.hardware === "connected");
    } catch { setHardwareConnected(false); }
  }, 5000);
  return () => clearInterval(interval);
}, []);

Task 8.2 — Hardware menu consolidation

File: frontend/src/components/navigation/NavigationDrawer.jsx

The sidebar uses an app registry pattern (APP_REGISTRY from constants/appRegistry.js). Hardware-related widgets should be grouped under a "Hardware" or "System" category. Check appRegistry.js and add a new category or reassign scattered hardware entries.

---

I worked on the others in our document

[Copilot]

Pull request overview

This PR modernizes ImSwitch’s wellplate/overview workflow by introducing an Opentrons-compatible labware layer (with validation + selection patterns), extending acquisition metadata output for plate-aware OME-NGFF sidecars, and updating both backend pixel-calibration handling and frontend Axon/WellSelector UX to support these workflows.

Changes:

• Added a new imswitch.imcontrol.model.labware module (schema validation, loaders, selection patterns, definitions) and accompanying unit tests.
• Extended experiment/OME-Zarr writing to emit OME-NGFF plate/well metadata (plate_metadata.json sidecar + optional per-well attrs).
• Updated multiple controllers/managers and frontend components to centralize pixel calibration (PixelCalibration as mandatory), add labware selection state, and improve overview/stream controls.

Reviewed changes

Copilot reviewed 109 out of 144 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
uv.lock	Adds jsonschema, opencv-contrib-python, updates NanoImagingPack lock entry.
setup.py	Bumps NanoImagingPack pin to 2.1.5.
scripts/convert_experiment_tiffs.py	Ashlar mode now tries to infer pixel size from protocol JSON; switches default entrypoint to main().
requirements.txt	Removed legacy requirements file.
requirements-dev.txt	Removed legacy dev requirements file.
requirements-arm64.txt	Removed legacy ARM64 requirements file.
pyproject.toml	Adds jsonschema, adds opencv-contrib-python, bumps NanoImagingPack pin to 2.1.5.
imswitch/imcontrol/view/ImConMainView.py	Forces PixelCalibration widget to be present in enabled dock keys.
imswitch/imcontrol/model/SetupInfo.py	Updates PixelCalibrationInfo doc/shape, adds overviewCameraName and overviewRegistration, refines affine calibration keying.
imswitch/imcontrol/model/managers/ExperimentManager.py	Adds overviewCameraName config plumbing.
imswitch/imcontrol/model/managers/detectors/VirtualCameraManager.py	Removes per-manager flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/detectors/TucsenCamManager.py	Removes per-manager pixel size/flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/detectors/HikCamManager.py	Removes per-manager pixel size/flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/detectors/GXPIPYManager.py	Removes per-manager pixel size/flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/ArkitektManager.py	Adds device-code hook for auth flow.
imswitch/imcontrol/model/labware/selector.py	New: well selection pattern resolution utilities.
imswitch/imcontrol/model/labware/schema_v2.json	New: vendored (reduced) Opentrons labware schema v2 subset.
imswitch/imcontrol/model/labware/models.py	New: pydantic models for Opentrons v2 + ImSwitch in-memory labware.
imswitch/imcontrol/model/labware/manager.py	New: discovery/caching/offset API for labware definitions.
imswitch/imcontrol/model/labware/loader.py	New: validates/parses/converts Opentrons JSON mm→µm.
imswitch/imcontrol/model/labware/definitions/openuc2/slide_4x_histosample_heidstar/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/ropod_2slides_uc2/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/ibidi_8well_chambered_coverslip/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/corning_6_wellplate_16.8ml_flat/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/corning_12_wellplate_6.9ml_flat/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/init.py	New: labware public API exports.
imswitch/imcontrol/model/io/ome_writers/plate_metadata.py	New: OME-NGFF plate metadata sidecar writer.
imswitch/imcontrol/model/io/ome_writers/ome_writer.py	Adds optional well_metadata to emit OME-NGFF well attrs + ImSwitch extension block.
imswitch/imcontrol/model/io/ome_writers/init.py	Exports plate metadata helpers.
imswitch/imcontrol/model/interfaces/thorlabs_tsi_sdk/dll/thorlabs_tsi_logger.cfg	Removed SDK logger config file.
imswitch/imcontrol/model/interfaces/thorlabs_tsi_sdk/dll/Copy 64-bit native libraries here.txt	Removed placeholder text file.
imswitch/imcontrol/controller/MasterController.py	Always instantiates PixelCalibrationManager.
imswitch/imcontrol/controller/ImConMainController.py	Ensures PixelCalibrationController is created last and registered.
imswitch/imcontrol/controller/controllers/wellplate_layouts.py	Removed legacy hard-coded layout definitions.
imswitch/imcontrol/controller/controllers/SettingsController.py	Adds per-detector stream params storage and optional detectorName parameter.
imswitch/imcontrol/controller/controllers/pixelcalibration/_archive/init.py	Added archive package marker.
imswitch/imcontrol/controller/controllers/ObjectiveController.py	Pixel size now prefers PixelCalibrationController with legacy fallback.
imswitch/imcontrol/controller/controllers/experiment_controller/models.py	Adds labware/well metadata fields + z_speed to parameters.
imswitch/imcontrol/controller/controllers/experiment_controller/experiment_performance_mode.py	Writes best-effort plate metadata sidecar (performance mode).
imswitch/imcontrol/controller/controllers/experiment_controller/experiment_normal_mode.py	Adds per-well metadata into OMEWriter + writes plate metadata sidecar.
imswitch/imcontrol/_test/unit/test_labware/test_selector_patterns.py	New: selector tests.
imswitch/imcontrol/_test/unit/test_labware/test_plate_metadata.py	New: plate sidecar tests.
imswitch/imcontrol/_test/unit/test_labware/test_loader_validation.py	New: loader validation tests.
imswitch/imcontrol/_test/unit/test_labware/test_loader_unit_conversion.py	New: mm→µm conversion tests.
imswitch/imcontrol/_test/unit/test_labware/test_loader_roundtrip.py	New: shipped definitions load/roundtrip tests.
imswitch/imcontrol/_test/unit/test_labware/test_generators.py	New: generator geometry sanity tests.
imswitch/imcontrol/_test/unit/test_labware/init.py	New: test package marker.
imswitch/imcontrol/_test/unit/pixelcalibration/_archive/test_overview_calibrator.py	Adds archived pixelcalibration tests.
frontend/src/state/slices/WellSelectorSlice.js	Adds labware selection state + MOVE_CAMERA speed controls.
frontend/src/state/slices/OverviewRegistrationSlice.js	Persists overlay prefs, adds autonomous scan + editable config state.
frontend/src/state/slices/ObjectiveSlice.js	Adds arrays for per-slot objective metadata.
frontend/src/state/slices/ExperimentSlice.js	Adds z_speed, point grouping metadata, and batch point append/replace helpers.
frontend/src/middleware/fetchObjectiveControllerGetStatus.js	Populates new objective metadata arrays.
frontend/src/components/wizard-steps/WizardStep6.js	Tweaks wizard styling.
frontend/src/components/wizard-steps/WizardStep5.js	Tweaks wizard styling.
frontend/src/components/wizard-steps/WizardStep4.js	Adds 1s polling for Z updates.
frontend/src/components/StreamControlOverlay.js	Refreshes objective status after stream setting changes.
frontend/src/components/ObjectiveController.js	Adds editable objective metadata UI; alters objective-switch guard behavior.
frontend/src/components/FRAMESettingsController.js	Removes Track Motion tab.
frontend/src/components/FRAMESettings/TestHomingTab.js	Reuses LiveView stream instead of PixelCalibrationController MJPEG stream.
frontend/src/components/FRAMESettings/SetLasersTab.js	Reuses LiveView stream instead of PixelCalibrationController MJPEG stream.
frontend/src/components/FRAMESettings/ObjectiveControllerTab.js	Reuses LiveView stream instead of PixelCalibrationController MJPEG stream.
frontend/src/components/FRAMESettings/ManualPixelCalibrationTab.js	Adds detector selection + calibration read/write/edit; fixes subsampling scaling path.
frontend/src/components/DetectorParameters.js	Adds polling in auto-exposure mode; improves numeric parsing; adds warning alert.
frontend/src/backendapi/apiUpdateOverviewRegistrationConfig.js	New: persist overview registration config edits.
frontend/src/backendapi/apiRunAutonomousOverviewScan.js	New: triggers autonomous overview scan endpoint.
frontend/src/backendapi/apiPixelCalibrationControllerSetCalibrationData.js	New: direct calibration write endpoint helper.
frontend/src/backendapi/apiPixelCalibrationControllerOverviewStream.js	Repoints overview stream control to LiveViewController.
frontend/src/backendapi/apiPixelCalibrationControllerManualPixelSizeCalibration.js	Adds detectorName + previewSubsamplingFactor parameters.
frontend/src/backendapi/apiPixelCalibrationControllerGetPendingCalibration.js	New: pending calibration polling helper.
frontend/src/backendapi/apiPixelCalibrationControllerGetCalibrationData.js	New: persisted calibration read helper.
frontend/src/backendapi/apiPixelCalibrationControllerGetAvailableDetectors.js	New: detector list helper for pixel calibration.
frontend/src/backendapi/apiPixelCalibrationControllerDiscardPendingCalibration.js	New: discard pending calibration helper.
frontend/src/backendapi/apiPixelCalibrationControllerCalibrateStageAffine.js	Adds detectorName/objectiveId query param handling.
frontend/src/backendapi/apiPixelCalibrationControllerApplyPendingCalibration.js	New: apply pending calibration helper.
frontend/src/backendapi/apiObjectiveControllerSetObjectiveParameters.js	New: set objective metadata endpoint helper.
frontend/src/backendapi/apiObjectiveControllerMoveToObjective.js	Default skipZ=false.
frontend/src/backendapi/apiGetOverviewRegistrationConfigData.js	New: loads persisted overview-registration config.
frontend/src/backendapi/apiGetOverviewOverlayImage.js	New: loads a stored slot overlay image.
frontend/src/backendapi/apiExperimentControllerSelectWellsByPattern.js	New: resolves selection patterns without mutating experiment state.
frontend/src/backendapi/apiExperimentControllerGetWellplateLayouts.js	Removed legacy layouts list helper.
frontend/src/backendapi/apiExperimentControllerGetWellplateLayout.js	Removed legacy single-layout helper.
frontend/src/backendapi/apiExperimentControllerGetLabwareList.js	New: fetch labware summaries.
frontend/src/backendapi/apiExperimentControllerGetLabwareDefinition.js	New: fetch full labware definition.
frontend/src/backendapi/apiExperimentControllerGenerateCustomWellplate.js	Removed legacy custom layout generator helper.
frontend/src/backendapi/apiExperimentControllerApplyWellSelectionToExperiment.js	New: returns point dicts for experiment pointList population.
frontend/src/axon/WellSelectorUtils.js	Adds point-in-polygon helper.
frontend/src/axon/WellSelectorComponent.js	Adds labware selection panel, freehand convert UI, and layout-switch confirmation.
frontend/src/axon/WellSelectorCanvas.js	Adds FREEHAND mode + freehand rendering/scan-position generation; MOVE_CAMERA XY speed usage.
frontend/src/axon/OverviewRegistrationWizard.js	Starts/stops MJPEG via LiveViewController and persists snapshot stage position.
frontend/src/axon/LiveViewComponent.js	Improves adaptive pixel-size computation for subsampled streams.
frontend/src/axon/ExperimentComponent.js	Adds “Use Current Camera Settings” and auto-loads exposure/gain defaults.
frontend/src/axon/experiment-designer/TilingDimension.js	Syncs overlap to areaSelectOverlap; adds Z speed control.
frontend/src/axon/experiment-designer/FocusMapDimension.js	Improves button enable/labels/tooltips.
frontend/src/axon/AxonTabComponent.js	Adds Overview Scan tab.
frontend/scripts/create_test_animations.sh	Removed acceptance-test GIF generator script.
frontend/python/server.py	Removed dev/test server implementation.
frontend/package-lock.json	Adds transitive yaml under tailwindcss tree.
docs/labware_opentrons.md	New: detailed Opentrons labware integration documentation.
.vscode/launch.json	Adds Windows Python launch config.
.readthedocs.yaml	Removes Read the Docs build config.

Files not reviewed (1)

• frontend/package-lock.json: Language not supported

Comments suppressed due to low confidence (1)

frontend/src/axon/WellSelectorComponent.js:345

• style={{ backgroundColor: "primary.main" }} uses a theme token string in a plain inline style, so the browser receives an invalid CSS color and the highlight won’t apply. Use MUI sx={{ bgcolor: 'primary.main' }} (or style={{ backgroundColor: theme.palette.primary.main }} via useTheme).

          >
            {/* current layout */}
            <MenuItem
              style={{ backgroundColor: "primary.main" }}
              value={experimentState.wellLayout.name}
            >
              {experimentState.wellLayout.name}
            </MenuItem>

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gokugiant]

Task 8.2 — Hardware menu consolidation

Done in frontend Version 1.6.6