Avoiding writing user config that is defined in the global config scope

[pelson]

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

The global config (at /etc/spyder, /usr/local/etc/spyder, $CONDA_PREFIX/etc/spyder, and hopefully soon at {sys.prefix}/etc/spyder (#15426)) is honoured when a user starts spyder for the first time. After that first use of spyder though the user gets a fully resolved config file written to their user config space (~/.config/spyder) and the global config is effectively ignored. The global config does change the defaults, but the defaults aren't used since the user config has every-single config item in it explicitly.

Further to this, the user config file becomes huge and unwieldy:

$ cat ~/.config/spyder-py3-dev/config/spyder.ini | wc -l
847
$ cat ~/.config/spyder-py3-dev/config/transient.ini | wc -l
98

If possible, it would be great if only the non-default config were saved to the user config file. This would result in two benefits, and one drawback:

• 👍 Global config has a chance to influence the "default" user configuration even after the first run
• 👍 The user config becomes much smaller, and easier to read/tweak
• 👎 The user config could change under the user's feet if the global config changes

Once this in place though, I fear that the next issue with global vs user config will reveal itself, namely: You cannot extend the global config in your user config. For example, suppose I wanted to provide a list of interpreters that are available to the spyder user, I could configure globally spyder with:

[main_interpreter]
custom_interpreters_list = ['/path/to/bin/python', '/another/path/to/bin/python']

This should (is) be picked up when constructing the user config, it is then extended to look at the user's possible interpreters (I haven't found where this is done in the codebase, but it seems all of my conda environments are picked up somehow).

After that, I might like to extend the list in the global config, but unfortunately it is too late! The user config now has a long list of items, which may or may not have the items in the global config. Ideally there would be some mechanism to declare that a user config item extends the default, rather than replacing it.

What steps reproduce the problem?

1. Setup a global config with some main_interpreter/custom_interpreters_list
2. Clear the user config
3. Start spyder
4. Observe that a bunch of interpreters are listed in the config panel, including those from the global config
5. Close spyder
6. Add another interpreter to the global config
7. Start spyder
8. Observe that the newly added interpreter is not available in the config panel

Versions

• Spyder version: master (ac79ebb)
• Python 3.7.9
• Qt 5.12.10
• PyQt5 5.12.3
• OS: Linux 5.4.0-70-generic

[ccordoba12]

This is a very interesting proposal and something we're open to discuss and review PRs for, but not for Spyder 5 but 6 (to be released next year). Our plan for 5 is to stabilize it as much as possible and this would touch a key piece of infrastructure.

Ideally there would be some mechanism to declare that a user config item extends the default, rather than replacing it.

Agreed, and we don't have something like that at the moment.

[pelson]

This is a very interesting proposal and something we're open to discuss and review PRs for, but not for Spyder 5 but 6 (to be released next year). Our plan for 5 is to stabilize it as much as possible and this would touch a key piece of infrastructure.

This is very reasonable. The first question to ask in terms of the implementation:

Is it reasonable that the user config should no longer contain the whole set of possible config items, but only those which are different from the default?

In this case "default" includes the global config. One implication of that is that you can't take a user config, copy it to another machine and expect that it will work in exactly the same way (as it would depend on the global config). This is not the case today.

If that is acceptable, it should be possible to even go from a user config today (v5) and strip out the values which have no effect. This is something that I could happily test out on a number of users to get some hands-on experience of how it works in practice (and find some of the rough edges wrt to config inheritance).

[ccordoba12]

This is very reasonable

Thanks for understanding @pelson!

Is it reasonable that the user config should no longer contain the whole set of possible config items, but only those which are different from the default?

Yes, I think it is. What we need to do is to provide a graphical, easy way to export settings so users can put them under version control.

If that is acceptable, it should be possible to even go from a user config today (v5) and strip out the values which have no effect.

Yep, I think that shouldn't be too problematic.

This is something that I could happily test out on a number of users

Could you also give us a hand with the implementation? I mean, we could probably start working on this at the end of the year.