Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement the new profiling system as described in Issue wrye-bash#250
------------------------------- S t a g e d P r o f i l e s ------------------------------- Staged Profiles is a new profiling system for Wrye Bash. It does not replace the original profile system which is now referred to as Classic Profiles. This does not fix, enhance, or change the way Classic Profiles work. As these approaches are not compatible with each other, you must choose one or the other. Staged Profiles are an advanced feature that has it's own copy of the game Data folder, game Installers folder, and load order history (BashLoadOrder.dat). It is based on the design described in Issue wrye-bash#250. This feature is disabled by default. To enable staged profiles, set the Bash.ini General Setting "useStagedProfiles=True". Staged Profiles created from other profiles initially share all mods and settings via hard linked files. Once created, installing or deleting mods in any profile does not affect other profiles! However, CHANGING a hard linked file WILL also change it in all profiles in the staged set. With Staged Profiles, there is no official game Data folder. The Data folder in each profile is just as legitimate as any other Data folder. You can now have a clean Data folder AND a Data folder filed with mods, at the same time. Only one is active at any given moment. Staged Profiles makes it easy and safe to create profiles that are as separate and independent as you wish with minimal increase in disk storage. Modding best practice is to either not modify your setup once you start a playthrough or remember the save file so you can go back if something goes wrong. With Staged Profiles rather than creating a special save, just create a new experimental profile based on your current profile and install mods into it. If something goes wrong, delete the experimental profile and go back to the earlier profile and try again. Since deleting a mod in a profile does not affect other profiles, you can delete mods from your new profile that don't relate to your current character build. This means you can have a "vanilla" profile that has a clean Data folder; never touched, always clean. You can also simulate profiles for programs that don't natively support them. For example, you can have a separately configured Mator Smash for each profile. This works for all games that Bash supports. This is discussed later. A technical description of what happens under the hood when you create, use, or delete a profile is detailed at the bottom of this file. Tutorial to convert your classic profiles ========================================== This is a tutorial on how to use Staged Profiles. It assumes you already know how to use WB. It's always a good idea to backup you Data, Mods and Saves directories before you begin. If you have a second computer, you may want to do these steps there just in case there are any surprises. Do not use WB's builtin in save and restore settings for this purpose. Do NOT enable Staged Profiles yet. The following are first use steps only. First, switch to the classic profile that you want to use as the basis for your first Staged Profile. Now enable Staged Profiles via the Bash.ini file. Depending on the number of mods installed, creating a new Staged Profile may take a few minutes, just wait. If you have more than just the Default profile, switch to each profile to convert it to a Staged Profile. WB will restart every time you switch to a Staged profile. Create a new profile named "Vanilla" that we will configure to contain NO mods. Now switch to the "Vanilla" profile. Bash will restart and when it comes back you will be in the "Vanilla" profile. Now that you're in the "Vanilla" profile, uninstall and delete ALL mods! There should now be NO mods listed in the installer tab and only the base game files in the mods tab. Open the game Data folder. If there are any stray files, those are files that were not being tracked by WB. Delete those untracked files to get a clean Data folder. If you are a purist, you can delete and re-install the game, then create your "Vanilla" profile. Now you can start the game and enjoy a vanilla experience. This profile holds your clean game Data folder. Notice the Staged profile looks no different than the originating profile. Also notice you have not consumed any additional disk space. Switch back to the other profile. Notice, everything looks as it did before. All your mods are present. Now switch to the "Vanilla" profile. Notice, NO mods. You're back to a nice clean Data folder. Whenever you create a new staged profile, it gets all the mods that are currently active/installed. So if you want a copy of an existing profile, switch to it, then go to Edit Profiles and create the new profile. What's next? ============ With the Vanilla profile active, I recommend you create a profile named "Vanilla+Essentials" which will be the source for future profiles. Add any mods you consider essential to your gameplay in general, like weather and environment mods. Customize any ini settings that you don't plan to change in other profiles. Why? This is explained later. The more mods in this essential profile, the less disk space you will use in future staged profiles. Create all new profiles from this "Vanilla+Essentials" profile and customize them to fit your character. When you want to create a new character, switch back to "Vanilla+Essentials" to jump start the creation of a character's profile. With this approach you have a common mod setup as a base for your new profiles. Are mods in Staged Profiles independent of each other? ====================================================== Yes, completely independent for any mod you ADD or DELETE to the profile. No, not if you CHANGE mods/files that are shared (hard linked), with other profiles, i.e. have a common ancestor profile. It's VERY IMPORTANT to understand the affect of CHANGING a file that Here's an example of this shared file effect. Let's say you have two profiles that have a common ancestor profile that contains the Skyrim Uncapper mod. If you change an uncapper ini settings in either profile it is changed in both profiles. If that's not what you want, you need to UNLINK (stop sharing) the Skyrim Uncapper ini file in the profile you want to make independent. To UNLINK a file and make changing it independent of ALL other profiles, you need to ADD/COPY it to your profile, not change it. You can do one of the following. 1. If the mod is a archive, change it to a project. This will ADD new files which will unlink them from other profiles. Remember to change the setting in the project and delete the archive. 2. Duplicate the mod and uninstall the original mod. 3. Uninstall, delete and re-install the mod. 4. Don't include the mod in the "Vanilla+Essentials" profile. ADD it to any profiles created while "Vanilla+Essentials" is active. Now that Uncapper is UNLINKED, you can change it and any changes will only be reflected in that profile. Note: Each time you unlink a file, it will consume disk space equivalent to the size of the file. CHANGING a linked file in one profile and having it change in all other linked profiles may be helpful. For example, if skyrim.esp is a linked file in the Vanilla profile, when you clean and save it in xEdit, you just cleaned it in all profiles derived from Vanilla! Nice. Icon overlay ============= WB gives no visual feedback if a file is hard linked. To get a visual indicator (and in my opinion this is imperative), you need to install HardLinkShellExt. Once installed all hard linked files displays a special icon. This is critical feedback when working with your profile. If the file is shared it will have an upward slanting red arrow in it's icon. You can download HardLinkShellExt here: http://schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html If the hard link overlay icon is not displaying, you have too many icon overlays installed. Windows has a limit. See this link for more information. https://www.ghacks.net/2016/07/24/fix-sync-icons-not-showing-explorer/ Simulating profiles in other applications ========================================= If an application is self-contained, i.e. doesn't write to the Windows registry and only uses files within it's folder, it can be independently associated with a WB profile. This allows each profile to have a separate configuration for a program that doesn't natively support them. Mator Smash is a good example. COPY or install the Mator Smash folder into the games Mod folder. It is now associated with the active profile. If you want another independent copy of Mator Smash, switch to the profile you want to add it to. There shouldn't be Smash there yet. COPY or install the Smash folder into the games Mod folder. The disadvantage is any changes like new tags you define are only associated with that profile. You'll need to manually combine them by looking at the other profile in the Saves folder. You can create a shortcut to this location and add it to WB's status bar and it'll always point to the correct/isolated version as you switch profiles. Switch back to classic profiles =============================== You can switch back to classic profiles at anytime, however this requires deleting all your enhanced profiles. Steps to return to classic profiles. - Switch to the profile that contains the mods you want to use as the foundation of the classic profile. - Delete all other profiles except the Default profile. While not actually necessary, you may cause a mess if you re-enable Staged Profiles. - Turn off staged profiles in bash.ini with setting "bUseStagedProfiles=False" - Restart Wrye Bash Future Staged Profile Enhancements ================================== (1) Associate Bash settings (BashSettings.dat) with a profile so each profile has its own settings. Today, Bash settings are common to all Staged Profiles. Use the same technique as is used for BashLoadOrder.dat. (2) Remove Backup Settings functionality. Other than the bash.ini file, I think all data stored in the settings backup archive are now associated with a Staged Profile. If you associate the bash.ini file with the profile the the Backup Settings functionality is pointless. If you want a backup of a profile, create a new Staged Profile with "(backup)" appended to the profile name. (3) Remove the Classic Profile system. Classic Profiles are only needed to support games whose Saves and Game directories are on different drives. Use junctioned point folders to cross the drives. This may not work as mentioned in the "rejected approach" in the Technical Description below. (4) The BashProfiles.dat file must remain common since it contains paths to all profiles and is needed when switching profiles. Technical Description ===================== This design is slight of hand, a shuffling of the cards. WB isn't aware of Staged Profiles. It continues to think it has only one game Data folder, one Installers folder and one load order file. WB creates hard linked files of every file in the game Data and Installers folders and stores them in the profile's Saves folder. Unlike NMM, which creates and deletes hard linked files in the game Data folder when switching profiles, WB swaps the game Data folder itself with a preconfigured/staged Data folder containing hard linked and normal files. The profile swapping process... When switching to a profile, WB first DEACTIVATEs the existing profile by MOVING the Game/Data and Game Mod folders to the active profile's Saves folder then ACTIVATEs the requested profile by MOVING its folders from the Saves folder to the Game/Data and Game Mod folders. It also swaps the BashLoadOrder.dat files so each profile has its own load order history. A new profiles starts with no load order history. Installers.dat, Table.dat, and Converters.dat are unique to each profile. BashConfig.dat is common to all Staged Profiles. A design priority for this feature is to never break existing mod configurations. Always preserve the integrity of the WB system, i.e. it must always start up and work. If the swapping process fails, even just occasionally, this will have a permanent negative impact on WB's reputation and adoption. Therefore, (1) Staged Profiles do NOT use BashInstallers.dat to determine which files to be hard linked. It hard links ALL files in the game Data folder without regard to what's in the BashInstallers.dat file. Why? Potential untracked files must be preserved in the new profile to preserve the integrity of the profile. (2) No destructive actions, i.e. do not delete any files or folders. When the user removes a staged profile its folder is moved to a new folder, "Trash (deleted profiles)". A unique suffix is appended to the profile's folder name to guarantee there are no collisions in this Trash folder. Occasionally the user should "take out the trash". Classic profiles continue to delete profiles from the Saves directory. The final step in switching to a new profile is to restart WB. Ideally, WB would call refresh methods to handle this but restarting was the easiest way to guarantee that all caches, buffers, etc were initialized with the new Data and Installer folders. Other games like The Dark Mod (http://www.thedarkmod.com) uses this same approach when switch from mission to mission. Staged profiles use ln.exe (http://schinagl.priv.at/nt/ln/ln.html) version 2.8.7.2 to hard link all files within a folder. This "batch approach" is faster than having python walk through the tree and hard linking files individually as NMM does. Hard linking only works on Windows systems where the Game and Saves folders are on the same drive. With this release, it does not work on Linux. An alternate but rejected approach was to not move folders around between Game and Saves. Instead create a junction in Game/Data that points to the appropriate profile in Saves. When the user switches profiles, delete the junctions and recreate them but now pointing to the switched-to profile's Saves folder. This works fast and flawlessly. If the user has a file or folder open in a moving folder, the user can continue working undisturbed since he is actually working in the original profiles Saves folder. This is ideal since no folders are moved so less likely to encounter an error. However when you click on the Installers tab, Bash blows. Obviously there is some code in the refresh logic that doesn't like junctions. I was concerned this was a can of worms so I didn't pursue it.
- Loading branch information
All this does not really belong to the gui side of things but that's too early to tackle