Environment which sets PATH may break

[jaredmontoya]

everything works as expected when I run bash-env-json on
/etc/profiles/per-user/nix/etc/profile.d/hm-session-vars.sh

but it fails when I run it on
/nix/store/qalsdzij4y0xgrawdn5nndjvp0mgqm1d-set-environment
with
bash-env-json /nix/store/qalsdzij4y0xgrawdn5nndjvp0mgqm1d-set-environment
or
cat /nix/store/qalsdzij4y0xgrawdn5nndjvp0mgqm1d-set-environment | bash-env-json

I tried both, the flake package and my package: NixOS/nixpkgs#358140

both failed like this:
flake package:

/nix/store/wz0bq4l97lkrwpvshvwh1mn6l9jk0vwp-bash-env-json/bin/.bash-env-json-wrapped: line 58: jq: command not found
/nix/store/wz0bq4l97lkrwpvshvwh1mn6l9jk0vwp-bash-env-json/bin/.bash-env-json-wrapped: line 58: jq: command not found
/nix/store/wz0bq4l97lkrwpvshvwh1mn6l9jk0vwp-bash-env-json/bin/.bash-env-json-wrapped: line 58: jq: command not found
/nix/store/wz0bq4l97lkrwpvshvwh1mn6l9jk0vwp-bash-env-json/bin/.bash-env-json-wrapped: line 270: jq: command not found

nixpkgs package:

/nix/store/f5zrcmscsj7c37czksl0ls6jjbl67ad1-bash-env-json-0.9.1/bin/.bash-env-json-wrapped: line 57: jq: command not found
/nix/store/f5zrcmscsj7c37czksl0ls6jjbl67ad1-bash-env-json-0.9.1/bin/.bash-env-json-wrapped: line 57: jq: command not found
/nix/store/f5zrcmscsj7c37czksl0ls6jjbl67ad1-bash-env-json-0.9.1/bin/.bash-env-json-wrapped: line 57: jq: command not found
/nix/store/f5zrcmscsj7c37czksl0ls6jjbl67ad1-bash-env-json-0.9.1/bin/.bash-env-json-wrapped: line 269: jq: command not found

/nix/store/qalsdzij4y0xgrawdn5nndjvp0mgqm1d-set-environment is responsible for environment.sessionVariables attrset in NixOS configuration.
Even without using bash-env on the file, environment variables from it are available in nushell when it is used as a login shell(so it is not because the parent user shell provided the environment), but I still think the fact that bash-env can't parse it might be an issue if it turns out that there is another script that needs to be sourced but bash-env doesn't work on it.

contents of /nix/store/qalsdzij4y0xgrawdn5nndjvp0mgqm1d-set-environment:

# DO NOT EDIT -- this file has been generated automatically.

# Prevent this file from being sourced by child shells.
export __NIXOS_SET_ENVIRONMENT_DONE=1

export CUPS_DATADIR="/nix/store/jq7rbn8yh1bnkxqqjaq3vhj9cskn5p7b-cups-progs/share/cups"
export GIO_EXTRA_MODULES="/nix/store/m7my9dk3j26x50z9mw5d8csgkvx0najj-gvfs-1.56.1/lib/gio/modules:/nix/store/sm3dq2zlf5p8shf4xir4hdpl9fgwy2yv-dconf-0.40.0-lib/lib/gio/modules"
export GTK_A11Y="none"
export GTK_PATH="$HOME/.nix-profile/lib/gtk-2.0:$HOME/.nix-profile/lib/gtk-3.0:$HOME/.nix-profile/lib/gtk-4.0:${XDG_STATE_HOME}/nix/profile/lib/gtk-2.0:${XDG_STATE_HOME}/nix/profile/lib/gtk-3.0:${XDG_STATE_HOME}/nix/profile/lib/gtk-4.0:$HOME/.local/state/nix/profile/lib/gtk-2.0:$HOME/.local/state/nix/profile/lib/gtk-3.0:$HOME/.local/state/nix/profile/lib/gtk-4.0:/etc/profiles/per-user/$USER/lib/gtk-2.0:/etc/profiles/per-user/$USER/lib/gtk-3.0:/etc/profiles/per-user/$USER/lib/gtk-4.0:/nix/var/nix/profiles/default/lib/gtk-2.0:/nix/var/nix/profiles/default/lib/gtk-3.0:/nix/var/nix/profiles/default/lib/gtk-4.0:/run/current-system/sw/lib/gtk-2.0:/run/current-system/sw/lib/gtk-3.0:/run/current-system/sw/lib/gtk-4.0"
export INFOPATH="$HOME/.nix-profile/info:$HOME/.nix-profile/share/info:${XDG_STATE_HOME}/nix/profile/info:${XDG_STATE_HOME}/nix/profile/share/info:$HOME/.local/state/nix/profile/info:$HOME/.local/state/nix/profile/share/info:/etc/profiles/per-user/$USER/info:/etc/profiles/per-user/$USER/share/info:/nix/var/nix/profiles/default/info:/nix/var/nix/profiles/default/share/info:/run/current-system/sw/info:/run/current-system/sw/share/info"
export LESSKEYIN_SYSTEM="/nix/store/3jyyi36yacqxkhrzk5ynkwc6nkw2jsq3-lessconfig"
export LESSOPEN="|/nix/store/75fsi0w4qjxc97ggznrc67mr3d88xhjn-lesspipe-2.11/bin/lesspipe.sh %s"
export LIBEXEC_PATH="$HOME/.nix-profile/libexec:${XDG_STATE_HOME}/nix/profile/libexec:$HOME/.local/state/nix/profile/libexec:/etc/profiles/per-user/$USER/libexec:/nix/var/nix/profiles/default/libexec:/run/current-system/sw/libexec"
export LOCALE_ARCHIVE="/run/current-system/sw/lib/locale/locale-archive"
export NAUTILUS_4_EXTENSION_DIR="/nix/store/h3h33h3fwcxm34x8nzrj036i77dg2k36-nautilus-python-4.0.1/lib/nautilus/extensions-4"
export NIXPKGS_CONFIG="/etc/nix/nixpkgs-config.nix"
export NIX_LD="/run/current-system/sw/share/nix-ld/lib/ld.so"
export NIX_LD_LIBRARY_PATH="/run/current-system/sw/share/nix-ld/lib"
export NIX_PATH="nixpkgs=/nix/store/4r8s42c9mwfvgdlxv0izb1cmlzrsb5nz-source"
export NIX_XDG_DESKTOP_PORTAL_DIR="/run/current-system/sw/share/xdg-desktop-portal/portals"
export NO_AT_BRIDGE="1"
export PAGER="less"
export PATH="$HOME/.nix-profile/bin:${XDG_STATE_HOME}/nix/profile/bin:$HOME/.local/state/nix/profile/bin:/etc/profiles/per-user/$USER/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin"
export QML2_IMPORT_PATH="$HOME/.nix-profile/lib/qt-5.15.15/qml:$HOME/.nix-profile/lib/qt-6/qml:${XDG_STATE_HOME}/nix/profile/lib/qt-5.15.15/qml:${XDG_STATE_HOME}/nix/profile/lib/qt-6/qml:$HOME/.local/state/nix/profile/lib/qt-5.15.15/qml:$HOME/.local/state/nix/profile/lib/qt-6/qml:/etc/profiles/per-user/$USER/lib/qt-5.15.15/qml:/etc/profiles/per-user/$USER/lib/qt-6/qml:/nix/var/nix/profiles/default/lib/qt-5.15.15/qml:/nix/var/nix/profiles/default/lib/qt-6/qml:/run/current-system/sw/lib/qt-5.15.15/qml:/run/current-system/sw/lib/qt-6/qml"
export QTWEBKIT_PLUGIN_PATH="$HOME/.nix-profile/lib/mozilla/plugins/:${XDG_STATE_HOME}/nix/profile/lib/mozilla/plugins/:$HOME/.local/state/nix/profile/lib/mozilla/plugins/:/etc/profiles/per-user/$USER/lib/mozilla/plugins/:/nix/var/nix/profiles/default/lib/mozilla/plugins/:/run/current-system/sw/lib/mozilla/plugins/"
export QT_PLUGIN_PATH="$HOME/.nix-profile/lib/qt-5.15.15/plugins:$HOME/.nix-profile/lib/qt-6/plugins:${XDG_STATE_HOME}/nix/profile/lib/qt-5.15.15/plugins:${XDG_STATE_HOME}/nix/profile/lib/qt-6/plugins:$HOME/.local/state/nix/profile/lib/qt-5.15.15/plugins:$HOME/.local/state/nix/profile/lib/qt-6/plugins:/etc/profiles/per-user/$USER/lib/qt-5.15.15/plugins:/etc/profiles/per-user/$USER/lib/qt-6/plugins:/nix/var/nix/profiles/default/lib/qt-5.15.15/plugins:/nix/var/nix/profiles/default/lib/qt-6/plugins:/run/current-system/sw/lib/qt-5.15.15/plugins:/run/current-system/sw/lib/qt-6/plugins"
export SPEECHD_CMD="/nix/store/wj4hbc70w1maj6ks92x3z4sxpv6ibnih-speech-dispatcher-0.11.5/bin/speech-dispatcher"
export SSH_ASKPASS=""
export TERMINFO_DIRS="$HOME/.nix-profile/share/terminfo:${XDG_STATE_HOME}/nix/profile/share/terminfo:$HOME/.local/state/nix/profile/share/terminfo:/etc/profiles/per-user/$USER/share/terminfo:/nix/var/nix/profiles/default/share/terminfo:/run/current-system/sw/share/terminfo"
export TZDIR="/etc/zoneinfo"
export XCURSOR_PATH="$HOME/.icons:$HOME/.local/share/icons:$HOME/.nix-profile/share/icons:$HOME/.nix-profile/share/pixmaps:${XDG_STATE_HOME}/nix/profile/share/icons:${XDG_STATE_HOME}/nix/profile/share/pixmaps:$HOME/.local/state/nix/profile/share/icons:$HOME/.local/state/nix/profile/share/pixmaps:/etc/profiles/per-user/$USER/share/icons:/etc/profiles/per-user/$USER/share/pixmaps:/nix/var/nix/profiles/default/share/icons:/nix/var/nix/profiles/default/share/pixmaps:/run/current-system/sw/share/icons:/run/current-system/sw/share/pixmaps"
export XCURSOR_SIZE="24"
export XDG_CONFIG_DIRS="/etc/xdg:$HOME/.nix-profile/etc/xdg:${XDG_STATE_HOME}/nix/profile/etc/xdg:$HOME/.local/state/nix/profile/etc/xdg:/etc/profiles/per-user/$USER/etc/xdg:/nix/var/nix/profiles/default/etc/xdg:/run/current-system/sw/etc/xdg"
export XDG_DATA_DIRS="/nix/store/7cmcy3jrzapj4hrxbi47yld6maf1l550-desktops/share:$HOME/.nix-profile/share:${XDG_STATE_HOME}/nix/profile/share:$HOME/.local/state/nix/profile/share:/etc/profiles/per-user/$USER/share:/nix/var/nix/profiles/default/share:/run/current-system/sw/share"

# Wrappers override other bin directories.
export PATH="/run/wrappers/bin:$PATH"


export NIX_USER_PROFILE_DIR="/nix/var/nix/profiles/per-user/$USER"
export NIX_PROFILES="/run/current-system/sw /nix/var/nix/profiles/default /etc/profiles/per-user/$USER $HOME/.local/state/nix/profile ${XDG_STATE_HOME}/nix/profile $HOME/.nix-profile"


# reset TERM with new TERMINFO available (if any)
export TERM=$TERM

[tesujimath]

This is strange.

The Nix Flake already has jq as a dependency, and wraps bash-env-json so that jq is on the path.

aya> cat (which bash-env-json)
#! /nix/store/izpf49b74i15pcr9708s3xdwyqs4jxwl-bash-5.2p32/bin/bash -e
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/x8jzsy0y1zk30mcvav2rh6lrw1gbzzy3-jq-1.7.1-bin/bin'':'/':'}
PATH='/nix/store/x8jzsy0y1zk30mcvav2rh6lrw1gbzzy3-jq-1.7.1-bin/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/gjlh1zvckhz0qv795lnzgw2zciklbzj2-gnused-4.9/bin'':'/':'}
PATH='/nix/store/gjlh1zvckhz0qv795lnzgw2zciklbzj2-gnused-4.9/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/0kg70swgpg45ipcz3pr2siidq9fn6d77-coreutils-9.5/bin'':'/':'}
PATH='/nix/store/0kg70swgpg45ipcz3pr2siidq9fn6d77-coreutils-9.5/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
exec -a "$0" "/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped"  "$@"

Have you collected garbage so your Nix store no longer has the jq package installed?

aya> ls -l /nix/store/x8jzsy0y1zk30mcvav2rh6lrw1gbzzy3-jq-1.7.1-bin/bin
total 36
-r-xr-xr-x 1 root root 35128 Jan  1  1970 jq

[tesujimath]

See https://github.com/tesujimath/bash-env-json/blob/main/flake.nix#L27

[jaredmontoya]

Well, of course it is strange.

#! /nix/store/mm0pa3z7kk6jh1i9rkxqxjqmd8h1qpxf-bash-5.2p37/bin/bash -e
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/wj603ds3b3gdwsrlx4nzcg4v3ba2298b-jq-1.7.1-bin/bin'':'/':'}
PATH='/nix/store/wj603ds3b3gdwsrlx4nzcg4v3ba2298b-jq-1.7.1-bin/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/13wxcn0j2wk911ahfkbvdq241rkprsv8-gnused-4.9/bin'':'/':'}
PATH='/nix/store/13wxcn0j2wk911ahfkbvdq241rkprsv8-gnused-4.9/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
PATH=${PATH:+':'$PATH':'}
PATH=${PATH/':''/nix/store/sf6y4arqcm100rnnl3dhpg732i774zp6-coreutils-9.5/bin'':'/':'}
PATH='/nix/store/sf6y4arqcm100rnnl3dhpg732i774zp6-coreutils-9.5/bin'$PATH
PATH=${PATH#':'}
PATH=${PATH%':'}
export PATH
exec -a "$0" "/nix/store/53c32nsxkj4gk0p306yzd6hzdnmm93w2-bash-env-json-0.9.1/bin/.bash-env-json-wrapped"  "$@" 

Because I manually went to /nix/store/wj603ds3b3gdwsrlx4nzcg4v3ba2298b-jq-1.7.1-bin/bin that should provide the jq binary and it is there, even tested the binary with --help.

I don't know if you tried to reproduce the issue by running bash-env-json on a file with the contents I provided but you should be able to.

If it really missed jq in path then it wouldn't work on other files too right? Because I assume that jq is used every time when bash-env-json is used.

Instead it just fails on this particular file with the jq is missing error and when used on other files it works perfectly. And I am sure jq doesn't go on vacation and refuse to process this file in prticular.

Weird I know, that's why I tried your flake package too, to make sure that it wasn't a problem with my package.

If you aren't able to reproduce the issue then I will try to run it on different NixOS machines.

I suspect that because those file contents manipulate PATH it might actually affect the PATH of bash-env-json and by the time it's evaluation is finished jq is no longer in PATH.

Edit:
Turns out I was completely correct.
this line:

export PATH="$HOME/.nix-profile/bin:${XDG_STATE_HOME}/nix/profile/bin:$HOME/.local/state/nix/profile/bin:/etc/profiles/per-user/$USER/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin"

was the culprit, commenting it out makes everything work.

[tesujimath]

Ah, I understand what's wrong, and I know how to fix it.

That file sets PATH to be something very minimal which doesn't have jq on it, so once it has been sourced by bash-env-json it can no longer find things on the path it expected to have.

The solution will be to not wrap the script with path setting, but instead access jq and the other dependencies by full paths.

I can fix this shortly. I used to do such a thing previously before I introduced the wrapping thing.

Thanks for reporting!

[tesujimath]

Minimal example:

aya> echo 'PATH=/oops' | bash-env-json
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 31: env: command not found
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 45: sed: command not found
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 237: mktemp: command not found
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 238: touch: command not found
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 240: : No such file or directory
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 241: rm: command not found
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 243: 6: Bad file descriptor
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 244: 6: Bad file descriptor
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 267: 6: Bad file descriptor
/nix/store/r7s9dyiir0m0ksa7l8kl9hjkxmwf2yj1-bash-env-json/bin/.bash-env-json-wrapped: line 270: 5: Bad file descriptor
Exception: bash-env-json exited with 1

[tesujimath]

OK @jaredmontoya this is fixed.

aya> echo 'PATH=/oops' | bash-env-json
{
  "env": {
    "PATH": "/oops"
  },
  "shellvars": {},
  "meta": {
    "version": "0.9.2a"
  }
}

I just want to add some tests for the Nix packaging before I merge that PR, but this should unblock you. 😄

[tesujimath]

Quicker than expected with the tests, is merged and tagged 0.9.2.

Enjoy!