Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QEP 82: Unified Application Profiles #82

Open
NathanW2 opened this issue Nov 12, 2016 · 58 comments
Open

QEP 82: Unified Application Profiles #82

NathanW2 opened this issue Nov 12, 2016 · 58 comments

Comments

@NathanW2
Copy link
Member

@NathanW2 NathanW2 commented Nov 12, 2016

QGIS Enhancement 82: Unified Application Configuration

Date 2016/11/12
Author Nathan Woodrow @NathanW2
Contact woodrow.nathan@gmail.com
maintainer @NathanW2
Version QGIS 3.0

Summary

This QEP is to add and expand Unified Application Configuration support into core QGIS. The main idea is to bundle everything that is normally in .qgis2 and settings (registry on
windows, config files on OS X/*nix, etc) into a single folder in the users platform based settings folder. This would allow the creation of saved config folder, you can think of them like profiles in Chrome/Firebreak, etc

The config folder would be a single folder with all .qgis2 and settings files which currently the result of doing the following in QGIS 2:

qgis --configpath folder\qgis

As saved configs contain isolated settings and plugins they can be great for different workflows, demos, users of the same machine, or testing settings, etc.

Proposal includes removing the use of current .qgis2 and settings paths (registry on Windows) in QGIS 3.x in favour of new profiles folder setup.

I think this method of storing settings and plugins, etc, could remove a lot of current confusion in how QGIS stores settings, as well as increase the flexibility in enterprise style deployments of QGIS.

Backups of user profiles also because very easy. A simple copy the profile folder to make a backup.

Proposed Solution

This proposal is to add new option --profile, as well as UI related to creating and switching saved configs.

All configs would live in a single top level folder in the platform dependent location e.g appdata on windows, etc, A link the about screen would also have the option to locate the current profile folder in the OS file manager.

An example of profile layout:

%APPDATA%
 - \QGIS
   - \configs
      - .default
      - \config1
      - \config2
      - \config3

By default, QGIS would run using the default config which is found in .default, e.g profile1

Part of this QEP is also loading machine-wide settings that are installed in the QGIS install folder. These should be loaded and then joined with the users saved config settings. Settings will follow the same format as the a users folder setup however there is only one.

Affected Files.

main() of QGIS would be altered to load configs from the settings folder.

A new class will also be introduced called QgsSettings which hides the implementation of config loading away from the code base. QgsSettings will be used in place of QSettings though out the core code base.

QgsSettings will have an API similar to QSettings in order to be a drop in replacement. The current implementation is to just be a wrapper for QSettings however in the future this could hold other logic for finding settings.

Performance Implications

None.

Further Considerations/Improvements

Cloud config

By hiding any implementation details inside QgsSettings we can also add loading config from a database and webserver in the future.

Webserver could expose endpoints to download/upload zip files of the user's config folder into a hosted environment. This is out of the scope of this QEP, however, this can be used as a building block for that function in future.

Backups

Built in backup process to allow rollbacks of settings. As the settings file is simply text files each change and is backed up and compressed for later rollback.

Backwards Compatibility

If QGIS 3.0 will inherit the settings of QGIS 2.x would migrate any existing settings and plugins into the new profile folder layout.

Issue Tracking ID(s)

(optional)

@NathanW2 NathanW2 added this to the 3.0 milestone Nov 12, 2016
@NathanW2 NathanW2 self-assigned this Nov 12, 2016
@NathanW2 NathanW2 changed the title User Profiles and Unified Settings. Application Profiles and Unified Settings. Nov 12, 2016
@pcav
Copy link
Member

@pcav pcav commented Nov 12, 2016

Agreed. Very good improvement. Thanks Nathan.

@nyalldawson
Copy link

@nyalldawson nyalldawson commented Nov 24, 2016

All profiles would live in a single top level folder, maybe in ~, %HOME% to
make it easy for the admins and users to find if needed. A link the about
screen would also have the option to locate the current profile folder.

I think we should respect the standard folder locations here. So Appdata\Roaming (or local?) for windows, and ~/.config/qgis for linux. Not sure what the OSX standard is.

We've been abusing people's home folders with the .qgis2 folder for a long time, it'd be good to start following the standard now that we've got the chance.

See also http://hub.qgis.org/issues/11926

At the same time we could also should split up config and cache folders, so users aren't unnecessarily backing up temporary files when they are saving their qgis settings.

@nyalldawson
Copy link

@nyalldawson nyalldawson commented Nov 24, 2016

Just so I understand this fully - this means no more registry use on windows?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 24, 2016

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 24, 2016

If we provide a way to jump to the folder from inside QGIS it doesn't really matter the location it's saved. So happy to match what the standard is for each platform.

@nyalldawson
Copy link

@nyalldawson nyalldawson commented Nov 25, 2016

When would someone need to jump to this folder? For backups or something?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 25, 2016

@pcav
Copy link
Member

@pcav pcav commented Nov 25, 2016

+1 for getting rid of win registry
+1 for a standard location
unsure about designing our way to manage the config folder: what other major sw do? Can't we leave this to the OS file manager?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 25, 2016

@ninsbl
Copy link

@ninsbl ninsbl commented Nov 25, 2016

Yes, indeed very nice to get rid of registry use, which several others already wished for on the mailinglists!

Not sure if that can be within the scope of this QEP, but in an (our) organizational setting, we have some settings (e.g. connections to OWS) which are frequently used by almost all users. These were put into a default setting, which is the copied into the user-homer. Users then can adjust the settings and add more OWS and so on. However, OWS-url get updated now and then and the question arises how to update the URL in all user settings without overwriting changes/adjustments users made to their QGIS.ini.

What I would like to have would be the flexibility, that users can change the settings as they like, but that "central" settings can be updated by an administrator (and not every user has to do that himself). A kind of versioning of the QGIS.ini. Splitting up the Settings into different sections seems to be helpful in that direction.

But maybe that is something for a Python-Macro...

@haubourg
Copy link

@haubourg haubourg commented Nov 25, 2016

Hi, this is a very welcome change!
+1 for standard location by default, + still a configpath option to move the profile elsewhere.

BTW, if we throw some light on profiles, I remember that some ressources can't be centralized easily by adding extra paths to qgis config.
We currently can add additionnal paths to SVG folders, plugins folders, but we miss that for symbols (markers, color ramps aka palettes.), and processing models (and also themes, and project templates). Ressource sharing is great for individual users but not adapted for centralized systems. Currently, admins have to deploy complex ini file management strategies for those parts. Adding to extra paths options in QGIS would make life easier for many of us :). Sorry if I'm off topic.

@rduivenvoorde
Copy link
Contributor

@rduivenvoorde rduivenvoorde commented Nov 25, 2016

Yes, I also like this idea's very much!!

Note that it is (at least on commandline in linux) already very easy using --configpath (which then takes BOTH settings and config). This also makes it easy to start with a clean slate or use different configs. If I remember correct @wonder-sk and @blazek did this for the configuration-work for schools.

I have QGIS config's for several client-projects within the client-project folder and start QGIS from command line: qgis --configpath /home/richard/zuidt/16/client_project/qgisfolder
In that way I also keep client-specific plugins/data/proxysettings etc etc together.

Firefox has the option to start a 'profile manager' using the -P option:

selection_173

Off course for windows/mac it is probably better to have a second small binary for that?

Do you think we still would need 'application'-settings? Or a concept in which you can 'save' options either in 'current' config, or in the 'global'-config? I'm thinking about for example: the theme to use (I would never use the 'Vista'-theme anymore on windows for example), or the place where you put all your icons? We could have a concept in which QGIS first loads application settings and then overrides some of those via the config settings?

Another idea: I find the starting up of QGIS sometimes slow. So IF we put a 'config'-chooser in the startup process, some core part of QGIS can be started very quick (in parallel when the 'choose config' dialog is shown)... and then only starts what is nessecary. At least making the impression that we start up faster would be nice I think.

Another idea: IF we have this minimal config choose already, would it be an option (in case of a debug build) to stream the debug output to that window too? In that case it would make it easier for windows/mac users to see what is happening when something goes wrong. I always do debug builds, and when something goes wrong it is the first place I look into....

@elpaso
Copy link

@elpaso elpaso commented Nov 25, 2016

+1 from me, I cannot see any negative implications, and I see only clear advantages.

@ninsbl
Copy link

@ninsbl ninsbl commented Nov 25, 2016

The --configpath option is available on Windows too.

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 25, 2016

Hmm how did I not see the --configpath option already. Good to know. I will update the QEP.

Most of this is still valid as I guess it's more about making this the default setup and I would still add a --profile option which can just take a name of a profile to load.

--configpath can be used for non profile based setups e.g save settings on a network drive or something.

@rduivenvoorde
Copy link
Contributor

@rduivenvoorde rduivenvoorde commented Nov 25, 2016

@NathanW2 mmm, please do not overload the user with concepts: we already had 'config' and 'settings' and now we have 'profile' too?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 25, 2016

@rduivenvoorde profile is a higher level concept a named config if you will, most users will never load QGIS on the command line.

If it's better maybe we change all the naming from profile to config and use that? I'm not fixed on the profile naming so happy to roll with just calling it config.

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Nov 25, 2016

@rduivenvoorde A config loader is a good idea, I will add that.

On the slow starting, I suspect this is due to the large start method in code and also in the plugins. A different topic that we will need to dig into.

@rduivenvoorde
Copy link
Contributor

@rduivenvoorde rduivenvoorde commented Nov 25, 2016

@NathanW2 yeah, I'm ok with whatever naming, it's just that I do not want to overload people with (our) abstract ideas as what they want is just "have all my QGIS configurations" in one place.
Do you also think there should be two levels of configurations (global and profile)?

@pcav
Copy link
Member

@pcav pcav commented Nov 25, 2016

Agreed, having different user profiles and the possibility of switching is a long standing idea, it would be great to see it implemented.
Thanks.

@Frederikssund
Copy link

@Frederikssund Frederikssund commented Nov 25, 2016

Hi Nathan - A very welcome change!!

Just a couple of additions:

It would nice if it was possible to use some "variable substitution" mechanism inside the setup files.

Use Case: When I have to prepare a rollout of Qgis to a "enterprise size" number of users (250), I'll do the following:

  1. Install a normal Qgis on a "clean" computer.
  2. Change the startup to use --configpath qualifierer, which points to a non-existing directory inside the Qgis program directory
  3. Prepare my installation by adding plugins, changing setup options and so on. This will create the abovementioned directory with setup information for Qgis
  4. At last distribute the new Qgis installation essentially by copying the Qgis program directory to the end users computer.
  5. The first time startup of the new Qgis with a slightly modified qgis startup file triggers a copy of the user directory prepared by me to a final location on the users computers.
    And now comes the problem.... The qgis.ini setup file contains a boatload af location references that points to the original location of the user directory. This is taken care of by running a sed based "search and replace" function that changes the original location references to the final location of the QGis userdirectory in the qgis.ini file. This "search and replace" sed command is very shaky (and butt-ugly to look at). It would be much simpler to use some kind a variable mechanism inside the ini file and simply change the value of the variable (ex. the location of the Qgis program directory and the qgis user-directory) when it's necessary.

In case the above is total gibberish, I have a detailed description of the process at https://github.com/Frederikssund/Alternativ-QGIS-installation

Suggestion no. 2: Put the microsoft support dll's ( msvcp*.dll and msvcr*.dll) in the qgis bin directory. That would make the Qgis program file directory "self contained" with no external file dependencies

Suggestion no 3: Add a user accesible "profile switcher", so it's possible to change profile dynamically during a session (A restart af Qgis it acceptable ;-)

Suggestion no 4: Loading the plugins sequentially (and not using threads) slows Qgis a lot during startup.
A nice easy method to use threads during plugin initialization would help a lot !

Regards Bo Victor

@ninsbl
Copy link

@ninsbl ninsbl commented Nov 25, 2016

Use Case: When I have to prepare a rollout of Qgis to a "enterprise size" number of users
Pretty similar here compared to what @Frederikssund does and experiences.

A "profile switcher" would be great, because this is currently quite cumbersome on Windows esp. if one does not have admin rights...

For relative path / variable support there is also: https://hub.qgis.org/issues/12623

However, I am afraid discussion is drifting away from the original intention of the QEP...

@SrNetoChan
Copy link
Member

@SrNetoChan SrNetoChan commented Nov 25, 2016

Very good idea. Thanks for that.

The only negative implication that I can think of is multiplying the same plugin(s) in several different profiles. But not sure if that is really an issue.

@pcav
Copy link
Member

@pcav pcav commented Nov 25, 2016

@SrNetoChan I do not think the extra Mb wasted for multiple copies of the plugin are a problem. In fact, this would allow for testing new version, having different sets installed, etc.

@Frederikssund
Copy link

@Frederikssund Frederikssund commented Nov 25, 2016

The "profile switcher" idea is explored by Boundless: "http://boundlessgeo.github.io/qgis-plugins-documentation/profiles/plugin.html". However it would be nice if its baked into Qgis proper instead of being a plugin.

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 23, 2017

Hi @NathanW2 and @elpaso,

@NathanW2 wrote:

Right. I think we just load that one from the install folder maybe.

Seems like a logical place for the default ini file is in QgsApplication::pkgDataPath()? Generally, this would require admin user permissions to change after install (relative to install location and type, of course), which I think is appropriate for such a file.

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 23, 2017

Followup to my last post. The idea behind having the default ini part of the QGIS install is that such a file should IMO be relative to the install package. If it is in a standard Qt-known location, then it will be shared across QGIS installs, which is sometimes nice to have, but often causes more problems than it solves, especially if it gets stomped by a subsequent (or even an older) install. It is not uncommon for users to have multiple QGIS installs and isolation should be considered here.

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 24, 2017

@dakcarto right so we agree it should go into the install folder, so you don't share between installs as that would be a pain to manage.

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 24, 2017

I will update this QEP tonight with more notes on the final design

@elpaso
Copy link

@elpaso elpaso commented Jan 24, 2017

Here is a first C++ quick and dirty prototype bound to a single setting as a proof of concept.
planetfederal/QGIS@042abdf
Note: I developed this as a proof of concept before our yesterday's conversation, I'm posting the link just in case we can re-use something.

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 27, 2017

@elpaso that looks good to me. mind if I just yank that into my code?

@elpaso
Copy link

@elpaso elpaso commented Jan 27, 2017

@NathanW2 not at all, I can port the python tests to C++ if you like, or write the bindings and use Python for the tests too. Currently missing from the implementation are the array read/write methods though.

@NathanW2 NathanW2 changed the title QEP 50: Application Profiles and Unified Settings. QEP 50: Unified Application Configuration Jan 29, 2017
@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 29, 2017

@dakcarto @elpaso how do you feel about the machine settings using the same format as what user profiles do? e.g on windows it might look like this:

image

image

This makes it easy to "deploy" machine settings by just building it up as a local config and then putting it in the install folder?

Does that make sense?

We might have do some more advanced merging logic for some of the other things but I think it should work ok.

@NathanW2 NathanW2 changed the title QEP 50: Unified Application Configuration QEP 82: Unified Application Configuration Jan 29, 2017
@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 29, 2017

@rduivenvoorde I have update the QEP to now call it "configs" and not profiles to avoid any confusion with computer profiles and logins.

@elpaso
Copy link

@elpaso elpaso commented Jan 30, 2017

@NathanW2 looks good to me. We need an easy way to create the default (machine) settings and I cannot think of an easiest one than copying from and existing configuration.

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 30, 2017

@NathanW2 wrote:

how do you feel about the machine settings using the same format as what user profiles do?

That seems fine to me, especially if inheritance (in @elpaso's prototype) is to work properly.

Maybe there could be checkboxes added to the Options -> Advanced -> Advanced Settings Editor, then one could just select what to export to a custom settings file in INI format (i.e. the QSettings-style INI that is, plus anything needed for custom configs in this QEP). Might be a simple way to generate the default (install-relative) settings file.

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 30, 2017

Or, do you mean the install-relative defaults config would actually be a directory, instead of a single config file?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 30, 2017

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 30, 2017

Ok. So if one were to just want to set defaults, add [QGIS]/QGIS3 to the QgsApplication::pkgDataPath()/<some standard name for defaults>/ directory? Or, some variation on that?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 30, 2017

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 30, 2017

Seems like the install-relative defaults wouldn't necessarily be something you would want copied over to a user config directory, if the referenced directory is also the template for such a user config.

Or, maybe you do, as a starting point for editing, since, in user space, those settings are just user overrides, not defaults?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Jan 30, 2017

@dakcarto
Copy link
Member

@dakcarto dakcarto commented Jan 30, 2017

Ah, ok, good point. Yeah, seems like a good idea to me.

@elpaso
Copy link

@elpaso elpaso commented Feb 3, 2017

@NathanW2 @dakcarto I've updated the dev branch with a full implementation with bindings and tests for the QgsSettings class: https://github.com/boundlessgeo/QGIS/tree/qgssettings-prototype
https://github.com/boundlessgeo/QGIS/blob/qgssettings-prototype/src/core/qgssettings.h

doxigen comments are still missing and I'd probably add some more shortcuts.

@NathanW2 NathanW2 changed the title QEP 82: Unified Application Configuration QEP 82: Unified Application Profiles Mar 16, 2017
@haubourg
Copy link

@haubourg haubourg commented Aug 31, 2017

I just had a look of current master's status concerning profiles management.
--configpath has no more effect and is replaced by the profiles-path and profile-name.

Shouldn't we raise a message if a qgis session is launched with deprecated options ?

@NathanW2
Copy link
Member Author

@NathanW2 NathanW2 commented Sep 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.