bug(python,venv): Global python bin being prepended when activating venv virtual environment within fish shell (works correctly in bash)

[inigohidalgo]

Describe the bug

Within fish, when activating a virtual environment created using venv, I am not able to actually use the python within the virtual environment. Instead, the available python executable stays as .local/share/mise/installs/python/3.10/bin/python. This is because the global python3.10 bin is being prepended to $PATH:

echo $PATH
>>> /home/jovyan/.local/share/mise/installs/python/3.10/bin /home/jovyan/tmp/venv-test/test1/bin

If I delete ~/.config/mise/config.toml which contains:

[tools]
python = "3.10"

and then activate my virtual environment, I get the expected behavior.

Everything works correctly from bash, even with the global python version configured.

mise doctor output

I am attaching mise doctor for both bash and fish, in case it helps to somehow bisect the issue. I also had debug and trace enabled, let me know if you'd rather I remove those from the output.

fish

[DEBUG] (1) mise::cli: [src/cli/mod.rs:211] ARGS: /home/jovyan/.local/bin/mise doctor
[TRACE] (1) mise::config: [src/config/mod.rs:68] Settings: Settings {
    all_compile: false,
    always_keep_download: false,
    always_keep_install: false,
    asdf_compat: false,
    color: true,
    disable_default_shorthands: false,
    disable_tools: {},
    experimental: false,
    jobs: 4,
    legacy_version_file: true,
    legacy_version_file_disable_tools: {},
    node_compile: false,
    not_found_auto_install: true,
    paranoid: false,
    plugin_autoupdate_last_check_duration: "7d",
    python_compile: false,
    python_venv_auto_create: false,
    raw: false,
    shorthands_file: None,
    task_output: None,
    trusted_config_paths: {},
    quiet: false,
    verbose: true,
    yes: false,
    ci: false,
    cd: None,
    debug: true,
    env_file: None,
    trace: true,
    log_level: "trace",
}
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:70] parsing: ~/.config/mise/config.toml
[TRACE] (1) mise::file: [src/file.rs:83] cat ~/.config/mise/config.toml
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:74] [tools]
python = "3.10"

[DEBUG] (1) mise::config: [src/config/mod.rs:97] Config {
    Config Files: [
        "~/.config/mise/config.toml",
    ],
    Installed Plugins: [],
}
mise version:
  2024.1.12 linux-x64 (3750934 2024-01-07)

build:
  Target: x86_64-unknown-linux-gnu
  Features: DEFAULT, NATIVE_TLS, OPENSSL
  Built: Sun, 7 Jan 2024 12:44:08 +0000
  Rust Version: rustc 1.75.0 (82e1608df 2023-12-21)
  Profile: release

[DEBUG] (1) mise::cmd: [src/cmd.rs:87] $ fish --version
shell:
  fish
  fish, version 3.6.1

mise data directory:
  /home/jovyan/.local/share/mise

mise environment variables:
  MISE_DEBUG=1
  MISE_TRACE=1
  MISE_SHELL=fish

mise settings file:
  ~/.config/mise/settings.toml

settings:
  {
    "all_compile": false,
    "always_keep_download": false,
    "always_keep_install": false,
    "asdf_compat": false,
    "color": true,
    "disable_default_shorthands": false,
    "disable_tools": [],
    "experimental": false,
    "jobs": 4,
    "legacy_version_file": true,
    "legacy_version_file_disable_tools": [],
    "node_compile": false,
    "not_found_auto_install": true,
    "paranoid": false,
    "plugin_autoupdate_last_check_duration": "7d",
    "python_compile": false,
    "python_venv_auto_create": false,
    "raw": false,
    "shorthands_file": null,
    "task_output": null,
    "trusted_config_paths": [],
    "quiet": false,
    "verbose": true,
    "yes": false,
    "ci": false,
    "cd": null,
    "debug": true,
    "env_file": null,
    "trace": true,
    "log_level": "trace"
  }

config files:
  /home/jovyan/.config/mise/config.toml

plugins:
  bun      (core)
  deno     (core)
  go       (core)
  java     (core)
  node     (core)
  python   (core)
  ruby     (core)

[DEBUG] (1) mise::toolset::builder: [src/toolset/builder.rs:52] Toolset: python@3.10
toolset:
  python@3.10

[TRACE] (1) mise::file: [src/file.rs:83] cat ~/.cache/mise/latest-version
No problems found
[DEBUG] (1) mise::cli: [src/cli/mod.rs:211] ARGS: /home/jovyan/.local/bin/mise hook-env -s fish
[TRACE] (1) mise::config: [src/config/mod.rs:68] Settings: Settings {
    all_compile: false,
    always_keep_download: false,
    always_keep_install: false,
    asdf_compat: false,
    color: true,
    disable_default_shorthands: false,
    disable_tools: {},
    experimental: false,
    jobs: 4,
    legacy_version_file: true,
    legacy_version_file_disable_tools: {},
    node_compile: false,
    not_found_auto_install: true,
    paranoid: false,
    plugin_autoupdate_last_check_duration: "7d",
    python_compile: false,
    python_venv_auto_create: false,
    raw: false,
    shorthands_file: None,
    task_output: None,
    trusted_config_paths: {},
    quiet: false,
    verbose: true,
    yes: false,
    ci: false,
    cd: None,
    debug: true,
    env_file: None,
    trace: true,
    log_level: "trace",
}
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:70] parsing: ~/.config/mise/config.toml
[TRACE] (1) mise::file: [src/file.rs:83] cat ~/.config/mise/config.toml
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:74] [tools]
python = "3.10"

[DEBUG] (1) mise::config: [src/config/mod.rs:97] Config {
    Config Files: [
        "~/.config/mise/config.toml",
    ],
    Installed Plugins: [],
}
[TRACE] (1) mise::hook_env: [src/hook_env.rs:67] config file modified: "/home/jovyan/.local/share/mise"
[DEBUG] (1) mise::toolset::builder: [src/toolset/builder.rs:52] Toolset: python@3.10

bash

[DEBUG] (1) mise::cli: [src/cli/mod.rs:211] ARGS: /home/jovyan/.local/bin/mise doctor
[TRACE] (1) mise::config: [src/config/mod.rs:68] Settings: Settings {
    all_compile: false,
    always_keep_download: false,
    always_keep_install: false,
    asdf_compat: false,
    color: true,
    disable_default_shorthands: false,
    disable_tools: {},
    experimental: false,
    jobs: 4,
    legacy_version_file: true,
    legacy_version_file_disable_tools: {},
    node_compile: false,
    not_found_auto_install: true,
    paranoid: false,
    plugin_autoupdate_last_check_duration: "7d",
    python_compile: false,
    python_venv_auto_create: false,
    raw: false,
    shorthands_file: None,
    task_output: None,
    trusted_config_paths: {},
    quiet: false,
    verbose: true,
    yes: false,
    ci: false,
    cd: None,
    debug: true,
    env_file: None,
    trace: true,
    log_level: "trace",
}
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:70] parsing: ~/.config/mise/config.toml
[TRACE] (1) mise::file: [src/file.rs:83] cat ~/.config/mise/config.toml
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:74] [tools]
python = "3.10"

[DEBUG] (1) mise::config: [src/config/mod.rs:97] Config {
    Config Files: [
        "~/.config/mise/config.toml",
    ],
    Installed Plugins: [],
}
mise version:
  2024.1.12 linux-x64 (3750934 2024-01-07)

build:
  Target: x86_64-unknown-linux-gnu
  Features: DEFAULT, NATIVE_TLS, OPENSSL
  Built: Sun, 7 Jan 2024 12:44:08 +0000
  Rust Version: rustc 1.75.0 (82e1608df 2023-12-21)
  Profile: release

[DEBUG] (1) mise::cmd: [src/cmd.rs:87] $ /bin/bash --version
shell:
  /bin/bash
  GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
  Copyright (C) 2020 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

  This is free software; you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.

mise data directory:
  /home/jovyan/.local/share/mise

mise environment variables:
  MISE_TRACE=1
  MISE_DEBUG=1
  MISE_SHELL=bash

mise settings file:
  ~/.config/mise/settings.toml

settings:
  {
    "all_compile": false,
    "always_keep_download": false,
    "always_keep_install": false,
    "asdf_compat": false,
    "color": true,
    "disable_default_shorthands": false,
    "disable_tools": [],
    "experimental": false,
    "jobs": 4,
    "legacy_version_file": true,
    "legacy_version_file_disable_tools": [],
    "node_compile": false,
    "not_found_auto_install": true,
    "paranoid": false,
    "plugin_autoupdate_last_check_duration": "7d",
    "python_compile": false,
    "python_venv_auto_create": false,
    "raw": false,
    "shorthands_file": null,
    "task_output": null,
    "trusted_config_paths": [],
    "quiet": false,
    "verbose": true,
    "yes": false,
    "ci": false,
    "cd": null,
    "debug": true,
    "env_file": null,
    "trace": true,
    "log_level": "trace"
  }

config files:
  /home/jovyan/.config/mise/config.toml

plugins:
  bun      (core)
  deno     (core)
  go       (core)
  java     (core)
  node     (core)
  python   (core)
  ruby     (core)

[DEBUG] (1) mise::toolset::builder: [src/toolset/builder.rs:52] Toolset: python@3.10
toolset:
  python@3.10

[TRACE] (1) mise::file: [src/file.rs:83] cat ~/.cache/mise/latest-version
No problems found
[DEBUG] (1) mise::cli: [src/cli/mod.rs:211] ARGS: /home/jovyan/.local/bin/mise hook-env -s bash
[TRACE] (1) mise::config: [src/config/mod.rs:68] Settings: Settings {
    all_compile: false,
    always_keep_download: false,
    always_keep_install: false,
    asdf_compat: false,
    color: true,
    disable_default_shorthands: false,
    disable_tools: {},
    experimental: false,
    jobs: 4,
    legacy_version_file: true,
    legacy_version_file_disable_tools: {},
    node_compile: false,
    not_found_auto_install: true,
    paranoid: false,
    plugin_autoupdate_last_check_duration: "7d",
    python_compile: false,
    python_venv_auto_create: false,
    raw: false,
    shorthands_file: None,
    task_output: None,
    trusted_config_paths: {},
    quiet: false,
    verbose: true,
    yes: false,
    ci: false,
    cd: None,
    debug: true,
    env_file: None,
    trace: true,
    log_level: "trace",
}
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:70] parsing: ~/.config/mise/config.toml
[TRACE] (1) mise::file: [src/file.rs:83] cat ~/.config/mise/config.toml
[TRACE] (1) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:74] [tools]
python = "3.10"

[DEBUG] (1) mise::config: [src/config/mod.rs:97] Config {
    Config Files: [
        "~/.config/mise/config.toml",
    ],
    Installed Plugins: [],
}
[TRACE] (1) mise::hook_env: [src/hook_env.rs:72] config files unmodified
[TRACE] (1) mise::hook_env: [src/hook_env.rs:41] early-exit

Additional context
A confounding factor might be the fact that I installed fish using homebrew on linux, so my fish executable is in /home/jovyan/bin/homebrew/bin/fish, and I don't actually have it set as my system shell since I don't have root permissions on my system.

[inigohidalgo]

I just saw #863 but I'm not sure if it's directly related since on my end it only breaks on fish, not bash

[inigohidalgo]

Using shims as you mentioned in some comment solves my issue, but I guess it would be preferrable to use mise activate if possible, right?

[jdx]

activate/shims is a tough question that has a lot of subtle differences. Personally I use neither, but I'm not a typical user. I did a writeup on this topic here: asdf-vm/asdf#1565 (comment)

[inigohidalgo]

Thanks for the detailed link. My thoughts relating to shims vs hooks mostly align with yours, in pyenv it always felt a bit weird to me that "which" will always point to the same file, as well as the fact that the shims are always in the PATH.

For my case it's not a huge deal since I only work with python and will usually have a specific desired version for each subdirectory I'm working in, so I'm happy to keep using shims, particularly since mise is so dang fast it makes it a nonissue.

I'll keep the issue open but if you think it's a duplicate of the general PATH issue feel free to close. I am also happy to help you test any changes you might make relating to this issue so feel free to ping me :)

Thanks for the awesome tool, and congrats on the new name, I like it :) (though I do have to say, runtime executor had a certain flair to it haha)

[jdx]

yeah I think that a lot of it might come down to whether or not you care if which python works or not. Personally I care quite a lot about that but others may not.

[alikefia]

When I had that on tasks, I have noticed the PATH variable is not filled the same way (task and interactive shell)
You can echo $PATH on fish and bash and see if mise behaves differently

[jdx]

this is working for me:

mise  main ❯ mise activate fish | source
mise  main ❯ which python
/Users/jdx/.cache/venv/bin/python

I assume it's been fixed since posting but lmk if not

[jdx]

you may want to try setting MISE_EXPERIMENTAL=1 because of #863