Persistent manifest path/context when in a pixi shell

[synapticarbors]

Problem description

After activating a pixi shell, pixi should maintain the context of that shell even if you move to another directory. @baszalmstra described the desired behavior nicely in #1021 (reply in thread):

...if you are in a pixi shell you are always in the context of the activated project regardless of where you navigate on your disk. If the project discovery locates a different manifest than the activated one it will show a warning that it is not using the one it discovered. This means using pixi run whilest in an activated shell will also always refer to the original manifest as well.

Activating a shell within in a shell will not be allowed regardless of where you are on disk. You should exit the initial shell first.

This way you can never mix projects unless you exit the shell but it does allow sort of global activation without too much effort. I think this kinda makes sense because the shell is already sort of in the context of the original project. I think this approach could remove a lot of the ambiguity that we are afraid of.

Locking the context of the subshell would allow a user to run commands like pixi list, pixi run, etc from outside of the project directory without having to specify the --manifest-path flag.

It is also desirable that the context should include the specific environment if a non-default env was selected when starting the shell (e.g. pixi shell -e fooenv). This is related to #893.

[dhirschfeld]

I think this would enable all the use-cases I care about (which would allow me to switch to pixi as my primary packaging tool).

[dhirschfeld]

It would be nice if it were possible to show (part of) the manifest-path and environment in the prompt of the shell so the user was more aware of which project the shell was associated with.

Not necessary, but a nice-to-have IMHO.

[synapticarbors]

Currently, my shell (macos running zsh with omz) prompt shows me the (<project name>) if in an activated pixi shell using the default environment and (<project name>:<env name>) if using a non-default env.

[dhirschfeld]

I don't have any pixi info in my prompt but I'm using starship.rs. Maybe I need some extra config? Or maybe it's just not supported (I'll have to look into that...)

[chawyehsu]

I don't have any pixi info in my prompt but I'm using starship.rs.

I'm also using starship, project env name will show after entering the env via pixi shell

DESKTOP in pixi on  win-arm64 is 📦 v0.16.1 via 🦀 v1.75.0 via 🅒 pixi

notice 🅒 pixi. It should be supported out-of-box

[dhirschfeld]

project env name will show after entering the env via pixi shell

I confirm that works for me! 🚀

Makes sense if you think of opening a shell as being like activating an environment (which it would be if this proposal were implemented!)

[tdejager]

The only question I have is what should happen if you want to run tasks from another 'pixi.toml' that's is not the original activated one :)?

[synapticarbors]

The only question I have is what should happen if you want to run tasks from another 'pixi.toml' that's is not the original activated one :)?

I think in that case, you would just have to explicitly use the --manifest-path flag to point pixi at the correct project. That would be the default behavior if you were not inside that other project currently. If you are in that project's directory and don't override with the flag, then Bas suggested "if the project discovery locates a different manifest than the activated one it will show a warning that it is not using the one it discovered".

[abkfenris]

Could something like having pixi read from environment variables for manifest path (PIXI_MANIFEST_PATH?) and other args when they aren't explicitly set work?

Then pixi shell could set those environment variables in the shell to allow persistence, and other commands would pick them up as default, but would allow overriding with explicit use of --manifest-path to point to another project as well.

I quickly tested this by mucking with shell, info, and list.

info for the local environment

pixi on  main is 📦 v0.17.1 via 🦀 v1.76.0 took 55s
❯
cargo run info
    Finished dev [unoptimized + debuginfo] target(s) in 0.39s
     Running `target/debug/pixi info`
      Pixi version: 0.17.1
          Platform: osx-arm64
  Virtual packages: __unix=0=0
                  : __osx=14.4=0
                  : __archspec=1=arm64
         Cache dir: /Users/akerney/Library/Caches/rattler/cache
      Auth storage: /Users/akerney/.rattler/credentials.json

Project
------------
     Manifest file: /Users/akerney/Geek/pixi/pixi.toml
  Config locations:
      Last updated: 22-03-2024 13:46:21

Environments
------------
       Environment: default
          Features: default
          Channels: conda-forge
  Dependency count: 11
      Dependencies: pre-commit, rust, openssl, pkg-config, git, pytest, jsonschema, pydantic, pyyaml, cffconvert, tbump
  Target platforms: linux-64, osx-64, osx-arm64, win-64
             Tasks: bump, test-schema, lint, install, build, test, generate-schema, test-all

       Environment: docs
          Features: docs, default
          Channels: conda-forge
  Dependency count: 17
      Dependencies: mkdocs, mkdocs-material, mdx_truly_sane_lists, pillow, cairosvg, mike, pre-commit, rust, openssl, pkg-config, git, pytest, jsonschema, pydantic, pyyaml, cffconvert, tbump
  Target platforms: osx-arm64, osx-64, linux-64, win-64
             Tasks: deploy-dev, build-docs, build, bump, generate-schema, install, test-schema, test-all, test, deploy-latest, docs, lint

shell to set a persistent environment

pixi on  main [!] is 📦 v0.17.1 via 🦀 v1.76.0
❯
cargo run shell --manifest-path /Users/akerney/GMRI/NERACOOS/NERACOOS_ERDDAP_K8S/pixi.toml
   Compiling pixi v0.17.1 (/Users/akerney/Geek/pixi)
    Finished dev [unoptimized + debuginfo] target(s) in 7.21s
     Running `target/debug/pixi shell --manifest-path /Users/akerney/GMRI/NERACOOS/NERACOOS_ERDDAP_K8S/pixi.toml`
 . "/var/folders/n6/hl07fcc55ys7mbybt01gb8r80000gn/T/pixi_env_M3e.sh"
pixi on  main [!] is 📦 v0.17.1 via 🦀 v1.76.0
❯  . "/var/folders/n6/hl07fcc55ys7mbybt01gb8r80000gn/T/pixi_env_M3e.sh"

info to see the persistent environment

(sea-eagle)
pixi on  main [!] is 📦 v0.17.1 via 🦀 v1.76.0 via 🅒 sea-eagle
❯ pixi info
      Pixi version: 0.17.1
          Platform: osx-arm64
  Virtual packages: __unix=0=0
                  : __osx=14.4=0
                  : __archspec=1=arm64
         Cache dir: /Users/akerney/Library/Caches/rattler/cache
      Auth storage: /Users/akerney/.rattler/credentials.json

Project
------------
           Version: 0.1.0
     Manifest file: /Users/akerney/GMRI/NERACOOS/NERACOOS_ERDDAP_K8S/pixi.toml
  Config locations:
      Last updated: 19-03-2024 11:02:21

Environments
------------
       Environment: default
          Features: default
          Channels: conda-forge
  Dependency count: 27
      Dependencies: python, cftime, dagster-postgres, dagster-k8s, dagster, dask, fsspec, ioos_qc, pandas, pydantic, pystac, python-frontmatter, yaml, requests, sentry-sdk, shapely, xarray, netcdf4, pint, ipykernel, matplotlib, erddapy, uvicorn, xpublish-edr, fastapi, xpublish, xpublish-opendap
  Target platforms: osx-arm64
             Tasks: install-kernel

info with an explicit --manifest-path to override the environment variable

(sea-eagle)
pixi on  main [!] is 📦 v0.17.1 via 🦀 v1.76.0 via 🅒 sea-eagle
❯ pixi info --manifest-path .
      Pixi version: 0.17.1
          Platform: osx-arm64
  Virtual packages: __unix=0=0
                  : __osx=14.4=0
                  : __archspec=1=arm64
         Cache dir: /Users/akerney/Library/Caches/rattler/cache
      Auth storage: /Users/akerney/.rattler/credentials.json

(sea-eagle)
pixi on  main [!] is 📦 v0.17.1 via 🦀 v1.76.0 via 🅒 sea-eagle
❯ pixi info --manifest-path ./pixi.toml
      Pixi version: 0.17.1
          Platform: osx-arm64
  Virtual packages: __unix=0=0
                  : __osx=14.4=0
                  : __archspec=1=arm64
         Cache dir: /Users/akerney/Library/Caches/rattler/cache
      Auth storage: /Users/akerney/.rattler/credentials.json

Project
------------
     Manifest file: /Users/akerney/Geek/pixi/pixi.toml
  Config locations:
      Last updated: 22-03-2024 13:46:21

Environments
------------
       Environment: default
          Features: default
          Channels: conda-forge
  Dependency count: 11
      Dependencies: pre-commit, rust, openssl, pkg-config, git, pytest, jsonschema, pydantic, pyyaml, cffconvert, tbump
  Target platforms: win-64, linux-64, osx-arm64, osx-64
             Tasks: lint, generate-schema, test-schema, test-all, bump, install, test, build

       Environment: docs
          Features: docs, default
          Channels: conda-forge
  Dependency count: 17
      Dependencies: mkdocs, mkdocs-material, mdx_truly_sane_lists, pillow, cairosvg, mike, pre-commit, rust, openssl, pkg-config, git, pytest, jsonschema, pydantic, pyyaml, cffconvert, tbump
  Target platforms: linux-64, osx-64, win-64, osx-arm64
             Tasks: build-docs, deploy-latest, test-schema, test, docs, generate-schema, build, deploy-dev, lint, test-all, bump, install

[ruben-arts]

He @abkfenris, this is indeed the simplest version of it, to get some warning however i would like to add some more logic to the discovery code. PR's are welcome!

[abkfenris]

Cool! I'll give it a shot.

Would tracing::warn!() be the right level of communication if an environment variable is set, and that's the manifest being used? Or is there something from miette?

It also looks like there is already a PIXI_PROJECT_MANIFEST environment variable getting set. Should I use/update that?

[ruben-arts]

Nice! @abkfenris

Yeah give it an initial try with tracing::warn.

You can see what pixi is setting in an environment using pixi shell-hook. We should depend on those variables to avoid two states. If you start with PIXI_PROJECT_MANIFEST we can see get a feel for it by testing your branch.

[abkfenris]

Ok, I've made a draft PR #1055. It's working for subsequent commands, at least until the manifest is changed, and then the environment variable is getting reset after the change and the context is lost.