startup files and XONSH_INTERACTIVE

[bestlem]

I am trying to use xonsh as my shell.

Interactive use works well and as expected.

But xonsh does not seem to read any startup files if it is launched non interactively. This differs from POSIX shells and fish where the .profile .zshenv and other config files are read in all cases.

I want to be able to control my configuration when I run say a compile process from my editor or from cron and then test on $XONSH_INTERACTIVE to control what is done.

How do I read startup files when run such that $XONSH_INTERACTIVE == False

I know there there is a --rc flag but not all tools all extra parameters to be passed in other than the command to be run.

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

[gforsyth]

Hi @bestlem -- there's a -i / --interactive flag to force running in interactive mode. Does that cover your request?

[bestlem]

No sorry I want to run xonsh in non-interactive mode but load the startup files e.g. ~/.config/rc.xsh - I can then test in the file to see if interactive or not

[gforsyth]

Got it. We removed static config files a while back, but sounds like we might want to bring those back.

[bestlem]

I think we are still at cross purposes here or else we are using different names for startup files. The files I am talking about are still there and used by xonsh, looking at old documentation https://xon.sh/xonshconfig.html (well that's another bug)

In addition to the run control file, xonsh allows you to have a static config file. This JSON-formatted file lives at $XONSH_CONFIG_DIR/config.json

I am talking about the run control file.

The initialisation files e.g .xonshrc ~/.config/rc.xsh i.e. the files listed in $XONSHRC are still picked up if you don't provide a command or file on the command line. xonsh will then read the file and set $XONSH_INTERACTIVE from other shells I would expect these to be independent actions

Although looking at history this decision was made long ago for 0.6.0 The changes start at 1f464fb

I disagree with the comment - but my view does need to be checked. I tried zsh -c "echo $PATH" and fish -c "echo $PATH" and they loaded their config files when I put an echo "In shell" in their config files.

Interactive is not the same as as reading startup files they are independant.

Comment is
Don't load xonshrc is xonsh is running a script

As per discussions with @asmeurer -- if xonsh is being used to launch a script,
e.g.

xonsh run_my_cool_deploy.xsh

then it should not load the xonshrc file since this should be reserved for
interactive functionality and we don't want side effects in the script.

Basically, if you need a xontrib to be loaded for your script, load it in your
script

[bestlem]

Looking at other issues especially #3176 Interactive and possibly login shells are not used much in xonsh and have some rough edges.

[anki-code]

Also similar issue #3579

[bestlem]

No this has nothing to do with the question in #3579 That refers to reading sh scripts from /etc/profile.d The questions states that existing environment variables are not read - which are in .sh files. The second comment does suggest that xonsh has something similar but that does not solve the actual problem. So I think that should be 2 separate items (I suspect that this is an example of why I think xonsh startup files are confusing as various concerns are not been separated)

This is simply that the standard initial file is ONLY read when in interactive mode.

The best description of startup files I see is zsh http://zsh.sourceforge.net/Doc/Release/Files.html#Files However even given that clearly in the official documentation Apple and Debian have put the environment settings the /etc/zprofile file (they just copied without thinking the POSIX setup)

xonsh does not need separate files and then tset $XONSH_INTERACTIVE and $XONSH_LOGIN to choose what code to use. This is also how fish does startup files. https://fishshell.com/docs/current/index.html#initialization

[anki-code]

@bestlem could you give the examples of the commands with expected behavior aligned with other shells?

In example:

xonsh --interactive --flags some.xsh  # This command should read the RC file than run xsh
xonsh some.xsh # this command should just run the xsh file
# etc

[anki-code]

@bestlem Also please take a look here #3374

[bestlem]

@anki-code you are correct that is how xonsh runs now but other shells run the other way round, always loading the startup files in a well defined order.

e.g. zsh http://zsh.sourceforge.net/Intro/intro_3.html (bash and other POSIX shells are similar woth different files) I am just showing the user's files in their HOME directory, there are defined places for system wide files

zshenv command.sh # runs ~/.zshenv and then the shell
Example
~/.zshenv is
echo "In .zshenv"
~/t.zsh is
echo "In t.zsh"

zsh t.zsh outputs
In zshenv
in t.zsh

That is the default is ALWAYS load the default startup file.

The other issue #3374 about multiple files is not an issue in zsh as there is only one file on the command line, that is what zsh executes after the startup file (usually just the .zshenv ones ) everything else after the command file is a parameter to that file

fish https://fishshell.com/docs/current/#initialization-files is more similar to xonsh in that there is only one user startup file (default is ~/.config/fish/config.fish) and you use the status command to see if the shell is interactive or login . So your startup files test that and only do interactive (e.g. GUI) stuff if the interactive flag is set.

fish is actually more complex as it does not just read a file - but also can be set up to read all scripts from ~/.config/fish/conf.d This is also needed as function as and aliases are usually setup to be in separate files.

A similar setup to zsh using echo will give
~/.config/fish/config.fish
echo "In fish config"
t.fish
echo "In t.fish"
Run t.fish
$ fish t.fish
In fish config
In t.fish

In summary other shells have defined startup files and always loads some of them. There is no flag to load a different startup file
POSIX shells load more files depending if it is a login or interactive shells. They also haev OS defined global files e.g. in /etc for Unix
fish loads a defined directory and a file for the user and recommends there are not system wide files for startup.

Given that xonsh only has the idea of one startup file I would look at the fish model but only load the one file. not a directory as fish needs to scan directories for its functions and aliases whilst xonsh uses normal python mechanisms for this.

[anki-code]

Thanks for the details @bestlem ! Let's try to understand what xonsh is doing and what you want to change.

# Let's use docker to test in isolated environment
docker run --rm -it python:3.9-slim /bin/bash -c "pip install xonsh; /bin/bash"

# Prepare RC files and script:

echo 'print("Hello from /etc/xonshrc")' > /etc/xonshrc
echo 'print("Hello from ~/.xonshrc")' > ~/.xonshrc
mkdir -p ~/.config/xonsh
export XDG_CONFIG_HOME='/root/.config'
echo 'print("Hello from ~/.config/xonsh/rc.xsh")' > ~/.config/xonsh/rc.xsh

echo 'print("Hello from ~/custom_rc")' > ~/custom_rc
echo 'print("Hello from script.xsh")' > ~/script.xsh

# Tests:

xonsh ~/script.xsh
# Hello from script.xsh

xonsh -i ~/script.xsh
# Hello from /etc/xonshrc
# Hello from ~/.config/xonsh/rc.xsh
# Hello from ~/.xonshrc
# Hello from script.xsh

xonsh --no-rc ~/script.xsh 
# Hello from script.xsh

xonsh --no-rc -i ~/script.xsh 
# Hello from script.xsh

xonsh --rc ~/custom_rc -- ~/script.xsh 
# Hello from script.xsh

xonsh -i --rc ~/custom_rc -- ~/script.xsh 
# Hello from ~/custom_rc
# Hello from script.xsh

xonsh -i --rc ~/custom_rc --no-rc -- ~/script.xsh 
# Hello from script.xsh

Please copy this example (starting from "Test") and add the changes you suggest. For example:

xonsh ~/script.xsh
# PROPOSE: Hello from ~/.xonshrc
# Hello from script.xsh

To begin and end code blocks on Github we use three back quotes. Example:

```python
code here
```

[bestlem]

I think you have understood - the last example should have the proposed line.

I am on an Apple Silicon Mac Mini so I can't run Docker yet. This also means that I have no reason to see if a shell is a login shell or not.

From the others - we need to see what $XONSH_INTERACTIVE and $XONSH_LOGIN are. I think that these are set correctly but from other issues might not be available early enough in the script.

The /etc/xonshrc - I have no view on - fish says don't do it. POSIX does use it. I am a programmer not a system administrator so only really manage one login so haven't thought about the pros or cons of having it.

Re ~/.xonshrc and ~/.config/xonsh/rc.xsh - I am using the latter and I have no view on what happens if you have both. It is probably an error if you have both but I don't know what xonsh does now. I also note that you can supply several --rc files - again I have no view on this. I think other shells just read in one file or directory so there is no example to follow for multiple startup files. I think other shells don't allow config files on the command line, they just allow one file on the command line and that is equivalent to script.xsh, so again no example to look at.

I will ignore ~/.xonshrc in the tests for simplicity

# Tests:

xonsh ~/script.xsh
# PROPOSE: Hello from /etc/xonshrc
# PROPOSE: Hello from  ~/.config/xonsh/rc.xsh
# Hello from script.xsh

xonsh -i ~/script.xsh
# Hello from /etc/xonshrc
# Hello from ~/.config/xonsh/rc.xsh
# Hello from ~/.xonshrc
# Hello from script.xsh

xonsh --no-rc ~/script.xsh 
# Hello from script.xsh

xonsh --no-rc -i ~/script.xsh 
# Hello from script.xsh

xonsh --rc ~/custom_rc -- ~/script.xsh 
# PROPOSE : Hello from ~/custom_rc 
# Hello from script.xsh

xonsh -i --rc ~/custom_rc -- ~/script.xsh 
# Hello from ~/custom_rc
# Hello from script.xsh


xonsh -i --rc ~/custom_rc --no-rc -- ~/script.xsh 
# PROPOSE - This is an error you can't have --rc and --no-rc in the same command 

# or if xonsh does accept this command line (but what does --no-rc mean here?
# PROPOSE : Hello from ~/custom_rc 
# Hello from script.xsh

For a fuller test - set the config files to be more like

if $XONSH_INTERACTIVE:
	print("Hello interactive custom_rc")
if $XONSH_LOGIN:
	print("Hello login custom_rc")
print("Hello custom_rc always")

Then

xonsh -i --rc ~/custom_rc -- ~/script.xsh 
# PROPOSED : Hello interactive custom_rc
# Hello custom_rc always
# Hello from script.xsh

xonsh --rc ~/custom_rc -- ~/script.xsh 
# Hello custom_rc always
# Hello from script.xsh

Also just running the xonsh shell as your shell should give

xonsh --rc ~/custom_rc 
# Hello interactive custom_rc
# PROPOSED : If you run from the command line as here there should be no login print which the current xonsh does.
# Hello custom_rc always
$ ....

[bestlem]

Looking at premain() I note the test for --norc is combined with --rc so --no_rc means that --rc is not checked for.
I think shell_kwargs["login"] is set correctly by the if args.login or sys.argv[0].startswith("-"): test but when args.mode is set to XonshMode.interactive it forces login to be true in all cases which is wrong.

[bestlem]

The rationale for the current startup looks like commits by gforsyth on 28 Sep 2017

These were presumably to fix an issue that arose then but I can't see a discussion other than the comment on 1f464fb

As per discussions with @asmeurer -- if xonsh is being used to launch a script,
e.g.

xonsh run_my_cool_deploy.xsh

then it should not load the xonshrc file since this should be reserved for
interactive functionality and we don't want side effects in the script.

Basically, if you need a xontrib to be loaded for your script, load it in your
script

[Eximius]

Not sure if this is the best issue to add to.

However, the fact that xonsh has no way of doing non-interactive shell config breaks many things. Such as PATH setup for non-interactive sessions.

In bash, many setups load interactive resources only after checking [[ $- != *i* ]] && return for example.

Was it expected that there should always be a parent interactive xonsh that sets up the env?

My current problem is I cannot use xonsh as my default shell on a remote machine, because it breaks tools that use an ssh connection to connect to remote services (here libvirtd). Creating the ssh connection necessarily runs the shell. The tools expect a sh/bash like shell. Admittedly they have a bug in their invocation, but that bug does not break in sh/bash. https://gitlab.com/libvirt/libvirt/-/issues/273

Adding to this issue, because if I could use such rc files, I could fixup this bug locally with a function.

[bestlem]

What does [[ $- != *i* ]] && return do?

I think that sort of hieroglyphics is a reason I use xonsh :)

[Eximius]

$- is a special variable holding flags the shell was started with. i is the interactive flag. That != checks for the interactive flag not set. This returns (stops sourcing) of non-interactively sourced scripts.

[anki-code]

I want to leave here some nice representation of how it works in bash/sh/zsh from twitter post:

[anki-code]

After reading this thread and #5491, #5488, #5496, #5099.

@bestlem sorry but you introduced statements that are wrong and because I feel pressure I want to break it all one by one:

• "All shells load home based RC files forever". NO. Bash, Zsh - the most world spreaded and default shells - do NOT do this. They avoid loading ~/.bashrc and ~/.zshrc in not-interactive mode (doc).

• "Zsh loads RC file in non-interactive mode". NO. Zsh loads ~/.zshenv it's NOT the same as ~/.zshrc. Please do NOT say that "zsh loads RC in non-interactive mode" it's misleading statement (doc).

• "Fish shell loads everything and feel good". NO. We CAN'T say that fish is doing the same and feel good because in fish shell the loading system is COMPLEX (1, 2) with many use cases and for example completion configs has separate place (doc). In xonsh shell we have RCs to solve all needs (at this time) and it's NOT similar NOT with fish and NOT with bash/zsh. By current RC behavior xonsh is closer to bash than to fish but the border is so blurry and we need to talk about use cases first.

• "Interactive mode and RC files are from parallel worlds and we need to just load all RC forever". NO. There is NO practice in the world that average user must use checking for interactive in RC files. There are NO significant amount of tools that have checking interactive flag in theirs installation instructions. Overwhelming majority of tools (search) have echo >> ~/.rc in installation instructions because historically home based RC is a buffer. By breaking this you will not breaking the world you will breaking how things work without sane proposal of alternative.

• "It's easy, just load all RC files". NO. We have 100+ xontribs, xonfig web, inexperienced users, experienced users from sh-world, guru from Nix-like systems with well structured XDG, etc, etc, and after switching RC to non-interactive all of these folks wondering "Why I see errors from RC and why my scripts became slower?". To mitigate this I've created Refactoring: review interactive functionality #5495.

Closing eyes on all of this I'm personally ok with #5099 (except #5491) and I want to help you with your use case but I can't see the use case (OS, list of commands, problem).

[bestlem]

I was use rc files as generic name for all startup files xonsh and fish do not have the split in files that zsh and bash has - so I was using rc files in fish/xonsh terms not the split between zshrc and zshenv

zsh loads .zshenv in a non interactive shells (In practice I think I am only after the settings that go in here)

I don't think xonsh is not nearer zsh than fish - I think this is where we look at things from differently - xonsh has one rc file zsh has several I see as the major difference.

Out of the tools that I use fzf broot, iterm all in fish test for --is-interactiove in fish. I am starting to use nix and the setup there goes through and extra layer and that layer has curated many of the other tools to put them inside is interactive guards. I agree that not all do

For zsh and bash the examples do >>~/.zshrc >>.bashrc so will not be loaded in non-interactive shells so zsh/bash does not have this issue. So the standard install only loads them for interactive use.

zsh/bash does not need the explicit test if it is interactive or not - it does that by putting interactive commands in a different file than things that run non-interactive and interactive.

The use case is the equivalent of .zshenv into non interactive shells ie loading in environment variables most obviously $PATH - this is made more obvious on macOS as there is not necessarily a login shell that set $PATH to be inherited into the script. Also if run from a service/cron - but that case is more complex and if you are doing that you are able to work out to use --no-rc or the default or be very explicit.

re "Interactive mode and RC files are from parallel worlds and we need to just load all RC forever"
I have never said that. We need to load SOME of the rc in non-interactive running.

In xonsh we do not have the split in files that zsh and bash does, xonsh is like fish in that they only way for the shell to work out whether to run stuff from the config if it is interactive or not is to make a test on $XONSH_INTERACTIVE

the loading of rc files in xonsh has three states.

1. None covered by --no-rc
2. Explicit name of file covered by --rc (and as you discovered with xxh fish does not have that so gets complex to do what is needed and I would consider that a bug in fish.)
3. Loading of default rc - only currently possible if $XONSH_INTERACTIVE is true but I think it needs to be able to load if it is false.

Is another case login shells? - I don't know the startup of X11 etc but aren't they non interactive and if so where do they get environment from. I suspect in practice no-one is using xonsh at that level as it is rather sh/bash dependent.

So my end point is

1. zsh and bash - all examples work as the instructions are to install ONLY for interactive - xonsh does not have that so easily, by design of xonsh you have to make a test if interactive or not,
2. If you don't load xonsh rc files then you need a way to load the default non interactive setup into xonsh list zsh does with .zshenv.

[anki-code]

Now I mostly agree, and it looks all of this solved by #5099. Right?
And because of I see no arguments why we can't merge #5491 I prefer to say that this topic is solved, isn't it?

[bestlem]

@anki-code
I think I see what you are trying to do with the not loading of ~/.xonshrc is to replicate the bash/zsh of having separate files for interactive and non-interactive.

But you are not being explicit on thi - or perhaps given your emphasis on comments about loading ALL rc you might be.

The issue is ~/.xonshrc does not contain JUST interactive stuff as bashrc and zshrc do. If you want that split you need to have two separate files. Currently xonshrc = zshenv+zprofile+zshrc but xonshrc ≠ zshrc which is what I think you are trying to implement. (given the bash diagram I won't try to work out what it is in bash)

Another issue is that bash and zsh include both or none of ~/.zshrc and /etc/zshrc - I think you are trying to include /etc/xonsh/* but not ~/.xonshrc and I think that might be inconsistent.

[anki-code]

Now I understand our misunderstanding I think.

At this time (here, now) I'm ok with #5099 and I have no plans to change this or doing split you described. I'm going to get popcorn and see what users say when they see unexpected loading of RC. It's not a sarcasm (or a bit) I'm really interesting.

What I want in #5491 is to just reduce the uncontrolled unexpected unintended stream of errors that will be produced by "user error" because of current expectation in the world that home based ~/.rc is mostly for interactive mode. I expect that if user see that he want to move env variables (e.g. PATH) to upstream RC the user will be use other places (this reflected in the new rc doc). That's all.

So it looks like we have no direct contradictions now.

[bestlem]

I apologise for wasting lot of time here. Looking at what I wrote in 2021 that does agree with @anki-code. But I think not what was implemented then did not match my example.

I made a mistake in not fully testing and reading what I said in my examples.

I assumed that xonsh would read only one of only ~/.config/xonsh/rc.sh and ~/.xonshrc as I thought only one would be loaded. So missed exactly what was going on. As I never have both I have never seen that. So in the latest comments I was considering ~/.config/xonsh/rc.sh only and @anki-code is looking at both.

I am also confused - in that is all the patch does is actually implement what was in the suggestions that ~/.xonshrc is not loaded when running a script?

However the new documentation does make it a bit clearer in that .xonshrc and ~/.config/xonsh/rc.xsh are treated differently and are both loaded.

It will be surprising to some that both to some that both are loaded but that has been the case for some time.

We have ended up with a mixture of how bash/zsh and fish deal with it and it is a unique way, but so is xonsh and I don't think anyone has got it fully correct. (Whatever it is better than bash anyway)

[anki-code]

Yes, the key is here:

# Defaults
$XONSHRC
# ('/etc/xonsh/xonshrc', '/USERHOME/.config/xonsh/rc.xsh', '/USERHOME/.xonshrc')
$XONSHRC_DIR
# ('/etc/xonsh/rc.d', '/USERHOME/.config/xonsh/rc.d')

All of these will be executed both interactive or non-interactive modes (#5099).
After #5491 '/USERHOME/.xonshrc' will work only in interactive mode.
That's all.

My last question: does current behavior solve your case?

[bestlem]

For testing I think I have done fuller checks here.
I am ignoring /etc files an login shells.

This is on xonsh 0.16.0 running on zsh and a brand new VM of macOS - just MacPorts and Xode command line tools installed.

script.xsh is

echo 'In script'

All the config files - ~/.xonshrc, ~/.config/xonsh/xsh.rc and ~/custom_rc are

file = "file name"
if $XONSH_INTERACTIVE:
	print(f"interactive {file}")
else:
    print(f"non interactive {file}")
if $XONSH_LOGIN:
	print(f"Hello login {file}")
print(f"Hello {file} always")

I think we need 6 tests of the flags a)interactive or not - against b)3 rc states (however see the end of this comment I assumed that command line flag order does not matter but it does)

% xonsh  script.xsh
non interactive .config/xonsh/rc.xsh
Hello .config/xonsh/rc.xsh always
non interactive .xonshrc
Hello .xonshrc always
In script

% xonsh  -i script.xsh
interactive .config/xonsh/rc.xsh
Hello .config/xonsh/rc.xsh always
interactive .xonshrc
Hello .xonshrc always
In script

% xonsh  --no-rc script.xsh
In script

% xonsh  --no-rc -i script.xsh
In script

% xonsh  --rc custom_rc  script.xsh
interactive custom_rc
Hello login custom_rc
Hello custom_rc always
In script
bestlem@bestlems-Virtual-Machine ~ @ exit

% xonsh  --rc custom_rc  -i script.xsh
interactive custom_rc
Hello custom_rc always
In script

I think @anki-code wants to just change the first one to remove ~/.xonshrc and change no other results.
so it should be

% xonsh  script.xsh
non interactive .config/xonsh/rc.xsh
Hello .config/xonsh/rc.xsh always
In script

However whilst we are here. Note the 5th one xonsh --rc custom_rc script.xsh It says it is interactive which it should not and then It ran the script but then dropped into a xonsh shell - That is not what I expected. The version with -i came back to the zsh. However altering the flag order to xonsh -i --rc custom_rc script.xsh gives exactly the same as the non i version.

[bestlem]

Current behaviour solves my case - albeit accidentally - I don't use ~/.xonshrc

What does confuse me now is your comment

I'm going to get popcorn and see what users say when they see unexpected loading of RC. It's not a sarcasm (or a bit) I'm really interesting.

Doesn't your change mean that users would now see a unexacting NON loading of RC, the change does not load any new rc just less than currently.

[anki-code]

Nope. Sorry, it was personal joke about popcorn. I just know persons who are using xonsh home based rc.d/ and have lot of code that is for interactive. They will know that this code executing in non-interactive mode some day :)

Current behaviour solves my case

Perfect. Thank you for attention to this case and improvements!

[anki-code]

Note the 5th one xonsh --rc custom_rc script.xsh It says it is interactive

Wow, it looks like a bug, I can reproduce. Thanks!

[anki-code]

Ah, it's not a bug. @bestlem

cd /tmp
echo 'echo 1' > 1.xsh

xonsh --rc 1.xsh 1.xsh  # run interactive with two rc files 
# 1
# 1
# prompt

xonsh --rc 1.xsh -- 1.xsh  # run script with one rc file 
# 1
# 1

[anki-code]

Closed by #5099 #5491

[bestlem]

OK you need -- to run a script after --rc. Every thing after --rc until a -- is treated as a config file.

Makes sense , however not documented I think
xonsh --help shows -rc RC [RC ...] but not the -- to end it. The document I think just shows examples of one RC.
xonsh --help also implies the arguments must be in the order shown.