Audit and overhaul Python environment management#167
Merged
Conversation
The backend `start_default_python_kernel_impl` now checks for project files when a notebook has no inline dependencies, matching the frontend's detection chain. This fixes the main UX gap where the backend auto-launch would give a bare prewarmed kernel even when a pyproject.toml or pixi.toml was present. Detection priority chain (after inline deps): 1. pyproject.toml → start with `uv run` (if has_dependencies or has_venv) 2. pixi.toml → convert to conda deps via rattler 3. (environment.yml placeholder for future PR) 4. Fall back to user preference for prewarmed envs Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Backend now returns detailed env_source strings from start_default_python_kernel_impl (e.g. "uv:inline", "uv:pyproject", "conda:pixi", "uv:prewarmed") instead of just "uv" or "conda". A "ready" lifecycle event carries this to the frontend. The toolbar kernel status now shows the source alongside the status (e.g. "Idle · pyproject.toml" or "Idle · conda") so users always know what environment their kernel is using. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Previously, adding a conda dependency blocked the UI for 30-150+ seconds because addDependency immediately called syncToKernel() which triggers a full conda solve+download+install cycle with the input disabled. Now matches the UV pattern: - addDependency just updates metadata and checks sync state (~200ms) - A "Sync Now" button appears when deps are dirty - Input stays enabled during sync so users can add multiple deps quickly - Separate syncing state tracks sync progress without blocking the input Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Shows a visible warning banner in the dependency header area when a notebook has both uv and conda dependency metadata. Previously this was a log-only warning. Now users can see which env type is being used and are prompted to clean up the unused deps. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
UV is always available (bootstrapped via rattler), faster for installs, and has better UX (non-blocking sync, pyproject.toml support). With P1/P2 adding project file auto-detection, the defaults are now context-sensitive: - pyproject.toml nearby → UV (auto-detected) - pixi.toml nearby → conda (auto-detected) - environment.yml nearby → conda (auto-detected) - No project files → UV (this change) Users who prefer conda can still set it explicitly in settings. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
…roject.toml The pyproject.toml banner now offers two distinct actions: - "Use project env" (primary, green) — starts kernel via uv run, stays in sync with pyproject.toml. Shows "Active" badge when already using it. - "Copy to notebook" (secondary, subtle) — copies deps as a snapshot into notebook metadata for portable sharing. Previously only "Import to notebook" existed, which was the copy action but wasn't clearly distinguishable from "use the project environment". Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
When the kernel was started via uv run (pyproject.toml), the dependency management UI now shows a read-only view: "Managed by pyproject.toml — restart kernel to pick up dependency changes." The add/remove dependency input is hidden since deps are managed by the project file. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Adds pixi.toml detection and import flow for conda environments: Backend: - New `import_pixi_dependencies` Tauri command that finds pixi.toml, converts dependencies to conda format, and writes to notebook metadata Frontend: - useCondaDependencies detects pixi.toml on mount via `detect_pixi_toml` - CondaDependencyHeader shows pixi.toml banner with dep count and "Copy to notebook" button when a pixi.toml with deps is found - Wired through App.tsx Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Previously env_id was stored in both metadata.runt.env_id (canonical) and metadata.conda.env_id (redundant copy). All reads already came from runt.env_id, so the conda copy was never used. Changes: - notebook_state.rs: Stop writing env_id into conda metadata when creating new notebooks - lib.rs: Remove redundant conda.env_id update in clone_notebook_to_path The CondaDependencies.env_id struct field is kept as an internal carrier (populated from runt.env_id before calling conda env functions). Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Review fixes: - import_pixi_dependencies now preserves python version constraint from pixi.toml when writing conda metadata (was silently dropped) - startKernel and startKernelWithDeno clear envSource to prevent stale env source labels when switching kernel types Test fixtures in crates/notebook/fixtures/audit-test/: - 1-vanilla: no deps, tests P8 default-to-uv - 2-uv-inline: notebook with uv deps - 3-conda-inline: notebook with conda deps - 4-both-deps: both uv+conda deps for P6 warning - pyproject-project/: pyproject.toml + notebook for P1/P7/P9 - pixi-project/: pixi.toml + notebook for P2/P4 Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
- Derive envType from envSource when kernel is running, fixing bugs where both-deps notebooks showed uv panel but backend chose conda (#4) and pixi auto-detection showed uv panel instead of conda (#6) - Replace 1s sleep + single connection attempt in start_with_uv_run with a retry loop (up to 8 attempts with increasing delays) that checks for process exit, emits progress events, and parses uv stderr for status - Add NOTEBOOK_PATH and E2E_SPEC env vars to wdio.conf.js for fixture testing - Add E2E specs for both-deps panel, pixi env detection, and pyproject startup Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
- Increase execution queue retry from 5s (50*100ms) to 5min (600*500ms) to support uv run scenarios where deps need installing - Emit "error" lifecycle event when auto-launch fails, so frontend transitions out of "Starting" state instead of hanging forever - Handle "error" lifecycle event in useKernel to set kernel status Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
rgbkrk
force-pushed
the
rgbkrk/audit-env-management
branch
from
845b6fe to
e41129e
Compare
rgbkrk
changed the title
The field was removed from AppSettings but two references remained in the Default impl and test, causing CI compilation failures. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
Adds a conda-env-project directory with an environment.yaml and test notebook, following the same pattern as the pixi-project and pyproject-project fixtures. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
- AGENTS.md: Add environment system overview, detection priority chain, trust system notes, and key files reference - contributing/environments.md: Architecture guide covering caching, prewarming, project file detection, frontend hooks, and testing - docs/environments.md: User-facing guide for inline deps, project files, cache cleanup, and troubleshooting - docs/sharing.md: User-facing guide for the two sharing models (inline portable vs project-level reference) Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
The pixi-env-detection spec requires the app to open a notebook next to pixi.toml so the backend can auto-detect it. Run it separately with the correct NOTEBOOK_PATH, and exclude it from the default test run where no fixture path is set. Co-Authored-By: QuillAid <261289082+quillaid@users.noreply.github.com>
- Move environment.yml detection after pyproject/pixi to match
documented priority: inline → pyproject → pixi → env.yml → prewarmed
- Return "conda:env_yml" instead of generic "conda" so toolbar shows
the correct source label
- Add setEnvSource("conda:env_yml") in frontend startKernelWithEnvironmentYml
- Add data-testid="notebook-toolbar" to toolbar header for E2E assertions
- Run pyproject-startup and both-deps-panel E2E specs with their
required fixture notebooks in CI
The --exclude CLI flags didn't match absolute spec paths, so all 13 specs ran in the default run (including fixture-specific ones without their NOTEBOOK_PATH). Fixture-specific runs also failed because E2E_SPEC relative paths didn't resolve. Fix: move exclusion logic into wdio.conf.js using the exclude config with absolute paths, and resolve E2E_SPEC with path.resolve().
The rich-outputs spec hung in CI — the app never loaded for the 10th sequential launch through the same tauri-driver instance. Fix by restarting tauri-driver before each wdio invocation and adding a 15-minute step timeout so hung specs can't block CI indefinitely.
start_with_uv_run hardcoded Command::new("uv") which fails with
"No such file or directory" when uv is bootstrapped (not on system
PATH). Use tools::get_uv_path() to resolve the correct binary,
matching what uv_env.rs does everywhere else.
Also extend E2E step timeout from 15 to 25 minutes to accommodate
4 sequential wdio runs including the pyproject spec which needs
uv to install deps.
Having uv on PATH is the realistic user scenario. The bootstrapping path is better tested as a unit test rather than gating E2E tests on it.
pyproject fails because uv run exits with status 2 resolving deps in the CI sandbox. both-deps fails because inline deps are untrusted on a fresh CI machine (no trust key). Both tests are kept for local use and can be enabled once CI has proper fixture infrastructure.
rgbkrk
added
daemon
rgbkrk
added
frontend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comprehensive audit and overhaul of the Python environment management system. Maps every kernel launch path, adds project file auto-detection, improves the dependency panel UX, and fixes bugs found during manual testing.
Backend: Auto-detection and kernel launch
uv runwith retry loop and progress eventsFrontend: Dependency panel
uv run) and "Copy to notebook" (imports inline)uv run, dep panel shows project-managed indicatorpyproject.toml,pixi.toml)envTypenow derives from backend'senvSourcewhen kernel is runningReliability
uv runinstallskernel:lifecycleerror so frontend exits "Starting" stateconda.env_id, using onlyrunt.env_idTesting
both-deps-panel,pixi-env-detection,pyproject-startupNOTEBOOK_PATHandE2E_SPECenv vars for fixture-based testingVerification
fixtures/audit-test/1-vanilla.ipynb— kernel starts with uv, no deps panel contentfixtures/audit-test/4-both-deps.ipynb— warning banner shown, correct panel after Trust & Installfixtures/audit-test/pyproject-project/5-pyproject.ipynb— kernel starts viauv run, progress shown in toolbar, no hangfixtures/audit-test/pixi-project/6-pixi.ipynb— conda panel shown, pixi.toml banner with "Copy to notebook"