Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reintegration of PluginManager. #6664

Merged
merged 3 commits into from
Sep 4, 2019

Conversation

maraf
Copy link
Member

@maraf maraf commented May 24, 2019

This PR reintegrates PluginManager to the GitExtensions installer and portable zip.

To make PM self updateable I'm moving it to user plugins folder.

Here I'm facing a problem with WIX.

error LGHT0204: ICE64: The directory LocalAppDataOrganizationDir is in the user profile but is not listed in the RemoveFile table.

All folders in user profile must be uninstalled, so I have defined their removal. But on of the folders is generated by <HeatDirectory /> build task and has generated id (Setup/Product.wxs#L75). Which may change unexpectedly..

The reason why PM WIX component is generated is that its file structure may change in the future, so I tried to make the installer universal.

Does anyone of you guys know how to make <HeatDirectory /> generate uninstall rules?
The only solution that came to my mind is replacement of HeatDirectory with powershell script.

@ghost ghost assigned maraf May 24, 2019
Setup/Product.wxs Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented May 24, 2019

Codecov Report

Merging #6664 into release/3.2 will decrease coverage by 0.01%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           release/3.2    #6664      +/-   ##
===============================================
- Coverage        47.34%   47.32%   -0.02%     
===============================================
  Files              745      745              
  Lines            54524    54524              
  Branches          7162     7162              
===============================================
- Hits             25815    25806       -9     
- Misses           27298    27299       +1     
- Partials          1411     1419       +8
Flag Coverage Δ
#production 36.49% <ø> (-0.03%) ⬇️
#tests 97.6% <ø> (ø) ⬆️

Setup/Prepare-Release.ps1 Outdated Show resolved Hide resolved
@mast-eu mast-eu added this to In progress in Move plugins via automation May 24, 2019
@mast-eu mast-eu mentioned this pull request May 24, 2019
37 tasks
@RussKie
Copy link
Member

RussKie commented Jun 24, 2019

@Johno-ACSLive any thoughts?

@Johno-ACSLive
Copy link
Contributor

Johno-ACSLive commented Jun 25, 2019

@Johno-ACSLive any thoughts?

I've had a quick look at this PR, roadmap item #6542 etc. If I understand this correctly, you wish to use a plugin manager for GE and the plugin manager will pull plugins using NuGet.

This PR is to remove plugins from the installer / portable and only include the plugin manager. There's also changes between system install for all users vs per user installation of plugins.

I saw this piece:

I think we should move in order:

  • first set up gitextensions/GitExtensions.PluginManager
  • review the code (at least I for myself never really looked into it)
  • build a release from the reviewed code under gitextensions/GitExtensions.PluginManager
  • include it in the installer

I agree in principle with the above approach.

From a distribution mechanism, the installer / portable just needs to know where the plugins go and if they are per user or per machine based once we know the details of how the plugin manager will work etc.

What is the decision around plugins, will they be installed for all users, per user or configurable for the installer?

For portable, it doesn't make sense to have these options as it should be on a per instance (folder copy) basis. Portables are not expected to be copied to sensitive locations and so should have full access to it's local structure.

For the installer this could be different based on the per machine / per user support, but it sounds like we simply need to create the appropriate directory based on a per machine or per user install. Uninstall just needs to know about the root Plugins folder for clean-up, we don't care what is below as we can tell the installer to recursively delete the Plugins directory.

Do we want similar capability to VS where users can "Roam" their plugins? I know this question is out of scope of this PR and should be covered under the roadmap. If the Plugin Manager does some smart things, then the installer and portable will need to know how to handle these things.

I may have gone into too much detail here 😄 not sure if this helps at all?

@mast-eu
Copy link
Member

mast-eu commented Jun 25, 2019

If I understand this correctly, you wish to use a plugin manager for GE and the plugin manager will pull plugins using NuGet.

Yes, you captured it nicely.

This PR is to remove plugins from the installer / portable and only include the plugin manager.

Not exactly. This PR will add the PluginManager, but not yet remove the plugins. They will be removed later, after they have been moved - one by one - into own repos and made available as NuGet packages.

What is the decision around plugins, will they be installed for all users, per user or configurable for the installer?

Do we want similar capability to VS where users can "Roam" their plugins?

There is a related discussion in #6415 / #6551. The conclusion was to go for plugins on a per user basis and without roaming.

Uninstall just needs to know about the root Plugins folder for clean-up, we don't care what is below as we can tell the installer to recursively delete the Plugins directory.

This is mostly correct. Deleting the plugins directory is sufficient to remove the plugin itself, but not it's settings.
The caveat is that all current plugins are saving their settings to GE's default settings file (C:\Users\<UserName>\AppData\Roaming\GitExtensions\GitExtensions\GitExtensions.settings for the installed version and <PortableRootDir>\GitExtensions.settings for the portable version). See the related discussion in gitextensions/gitextensions.extensibility#4.
Therefore, if we simply delete the plugin directory, these plugin-specific settings would remain in GitExtensions.settings. This would not create any immediate trouble (they will be simply "dead weight"), but it would clutter the settings file over time if we don't take care of removing them.

@gerhardol
Copy link
Member

The caveat is that all current plugins are saving their settings to GE's default settings file

As well as possibly in settings per repo

@Johno-ACSLive
Copy link
Contributor

OK understand now, I agree that the plugins themselves shouldn't roam (i.e. the binaries should be stored in %LOCALAPPDATA%. I think that roaming which plugin a user has enabled and potentially it's settings could be stored in %APPDATA%.

What this means is that the plugin manager can recognise that a plugin has been downloaded and installed on device A and plugin manager can then see that has occured and download the associated plugin on device B (irrespective of GE / Plugin version, it just cares about the plugin being installed). This assumes the version of GE supports the plugin being migrated to PM otherwise it will have no effect.

For individual plugins, it would depend on how breaking changes implement their configuration handling, perhaps initially %LOCALAPPDATA% is used and as the plugins mature their configuration management they can support %APPDATA% for roaming.

From an installer perspective, cleaning up configuration from main GE settings and having a config file per plugin would depend on the migration strategy to the PM. If we move a plugin from Legacy to PM, something needs to migrate the data.

So lets say we upgrade GE to a version where plugin A is removed from GE and is now available from PM. The installer will have the payload removed (the binaries will no longer be included and during upgrade, the installer will remove the previous version which will remove the exisiting binaries) but something in GE / PM will need to detect if the user was using the Plugin, install the package via PM and migrate the settings from GE main config to plugin config.

Is this expected to be the installer or GE / PM / Plugin? I believe that GE / PM / Plugin should be responsible to perform this action as I am assuming that a plugin via PM will be self contained (it will contain base config of the plugin). I have looked at the TODO's and there are still some decisions to be made.

However, what we can do in the installer is launch an executable (or script) with a flag to indicate a migration needs to be performed. The installer UI can indicate it is migrating plugins while the custom action is occuring. The executable or script (especially a PS script) will also need to be signed.

Alternatively, the first load of GE can run through a Plugin migration wizard and we could detect what plugins we think a user is using and have them select which ones to preserve.

Again, depends on the migration strategy for the plugins.

In the case where GE is being uninstalled and not upgraded, we would remove all directories associated with GE. So if we create two user profile locations e.g. %LOCALAPPDATA%\gitextensions and %APPDATA%\gitextensions then uninstall just deletes those to ensure a clean removal. For upgrades these directories will of course be preserved (the installer just needs to ensure this is the case).

Is the PM ready to have its binaries compiled / signed with GE ready for inclusion in the installer? At the moment I see a PS script to download the binaries to sub folder of the Plugins directory during installation. Ideally we would bundle the PM binaries in the installer.

@mast-eu
Copy link
Member

mast-eu commented Jul 3, 2019

Actually I am thinking of a much simpler approach: leaving it entirely to users what plugins they want to install. Detecting which plugins a user might have used in the past would be very tricky and error-prone. For example, I often look at the results from AppVeyor (mostly the green and red dots in the revision graph) and occasionally open the Statistics. Both are plugins and for both I didn't even change the default settings. I cannot think of a way to reliably detect this kind of usage.

Also if you use multiple devices, I would require you install the plugins multiple times. At least for the first release, since I think the majority of our users does not require roaming of plugins (or of the metadata of installed plugins). It would be nice to have, but it is not a mandatory feature.

Is the PM ready to have its binaries compiled / signed with GE ready for inclusion in the installer?

Yes, almost. The PM code can be found in gitextensions/gitextensions.pluginmanager and the artifact can be downloaded from AppVeyor. It is not yet signed (see gitextensions/gitextensions.pluginmanager#17).

At the moment I see a PS script to download the binaries to sub folder of the Plugins directory during installation. Ideally we would bundle the PM binaries in the installer.

🤔 The script is downloading the PM while assembling the installer, so that it is already bundled with the MSI. There shouldn't be a separate download during installation on the target PC.

@RussKie
Copy link
Member

RussKie commented Jul 7, 2019

The 3.2 release is imminent, so if you want to include the PM in the distribution - it is a good time to bring it back now.

@maraf
Copy link
Member Author

maraf commented Jul 7, 2019

In my option only GitExtensions.Extensibility nuget package is a must have from our todo list.
All other things are "nice to have" for broader adoption.

@RussKie
Copy link
Member

RussKie commented Jul 8, 2019 via email

Copy link
Member

@mast-eu mast-eu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review is only for a plain simple typo.
Fixing it doesn't cause any functional change and doesn't resolve the open points of this PR.

Setup/PluginManager.wxs Outdated Show resolved Hide resolved
Setup/Product.wxs Outdated Show resolved Hide resolved
Setup/Setup.wixproj Outdated Show resolved Hide resolved
Setup/Setup.wixproj Outdated Show resolved Hide resolved
Setup/Setup.wixproj Outdated Show resolved Hide resolved
@mast-eu
Copy link
Member

mast-eu commented Aug 6, 2019

All folders in user profile must be uninstalled, so I have defined their removal. But on of the folders is generated by build task and has generated id (Setup/Product.wxs#L75). Which may change unexpectedly..

I deleted the file Setup/PluginManager.wxs locally on my system and recreated it from scratch by running Setup\BuildInstallers.cmd.
It is regenerated identically to your commited version, with the same ID in

<Directory Id="dir7351E76F113B364403061A39D2447F8D" Name="PackageManager">

Thus I don't think this ID can change unexpectedly, as long as the path and filenames do not change.

The reason why PM WIX component is generated is that its file structure may change in the future, so I tried to make the installer universal.

Sure, the file structure of the PM artifact might change in the future, but in this case I think it would be acceptable that the installer needs to be updated, too. Furthermore, I don't think that changes will be frequent.
In other words, I doubt that we need an universal installer. It would be nice to have, but it is not mandatory. Let's remove the HeatDirectory task if it is creating trouble.
IMHO, it is acceptable for the installer to work only with the precise locations and filenames $(var.PluginManagerSourceDir)\GitExtensions.PluginManager.dll and $(var.PluginManagerSourceDir)\PackageManager\PackageManager.UI.exe.

@maraf
Copy link
Member Author

maraf commented Aug 7, 2019

That makes things a lost simpler, the heat stuff was very painful for me. I'm going to update the PR.

@maraf
Copy link
Member Author

maraf commented Aug 10, 2019

The PM download URL still points to the original repository, as the official repository doesn't any release yet.

@mast-eu
Copy link
Member

mast-eu commented Aug 13, 2019

The PM download URL still points to the original repository, as the official repository doesn't any release yet.

I just created pre-release v1.0.0-RC1 in the official repo.
Please update the URL to https://github.com/gitextensions/gitextensions.pluginmanager/releases.

@mast-eu mast-eu self-requested a review August 13, 2019 21:50
@maraf maraf requested a review from RussKie August 19, 2019 17:48
@maraf
Copy link
Member Author

maraf commented Aug 19, 2019

I have updated the URL to the gitextensions/gitextensions.pluginmanager repository.

@mast-eu
Copy link
Member

mast-eu commented Aug 19, 2019

Any idea why the CI build is failing? I didn't look into details yet...

Could you please rebase on current master branch and fix the remaining typo?

@maraf
Copy link
Member Author

maraf commented Aug 19, 2019

@mast-eu Oh, sorry. I completely missed that. Do it now..

@RussKie
Copy link
Member

RussKie commented Aug 21, 2019

Release 3.2 is technically ready, I'm happy to wait a little longer, if you think this PR is almost ready.

@RussKie

This comment has been minimized.

@mast-eu
Copy link
Member

mast-eu commented Aug 26, 2019

I fixed the build (by simply restarting it 😁 )

Seems like the hiccup was on AppVeyors servers, not in our code: https://ci.appveyor.com/project/gitextensions/gitextensions/builds/26923655#L286

@maraf
Copy link
Member Author

maraf commented Aug 26, 2019

@mast-eu I still don't see the installer.

@mast-eu
Copy link
Member

mast-eu commented Aug 26, 2019

MSI artifacts are not available on GitExtensions' AppVeyor account, because installers are deliberately not build on pull requests. Therefore, I built it on my personal account.

https://ci.appveyor.com/project/mast-eu/gitextensions/builds/26958591/artifacts

@RussKie
Copy link
Member

RussKie commented Aug 26, 2019 via email

@mast-eu

This comment has been minimized.

@RussKie RussKie force-pushed the release/3.2 branch 2 times, most recently from 4df1705 to 4999140 Compare September 1, 2019 19:22
@mast-eu

This comment has been minimized.

@RussKie

This comment has been minimized.

@RussKie
Copy link
Member

RussKie commented Sep 2, 2019

The 3.2 branch appears to be in a good shape.
Please finalise the PR and we'll do a release.

@mast-eu
Copy link
Member

mast-eu commented Sep 3, 2019

The change I pushed corrects the path where PluginManager is copied to. It was C:\<UserDir>\AppData\Local\GitExtensions\GitExtensions\UserPlugins\*, but should be C:\<UserDir>\AppData\Local\GitExtensions\UserPlugins\GitExtensions.PluginManager\*. See gitextensions/gitextensions.pluginmanager#29.
The portable build has also been corrected.

@maraf: I changed your original implementation a bit. Most importantly, I removed /Setup/PluginManager.wxs and put everything in /Setup/Product.wxs. Please have a look.

Move plugins automation moved this from In progress to Reviewer approved Sep 3, 2019
Copy link
Member

@mast-eu mast-eu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the MSI and the portable build. PluginManager now works correctly with both.
MSI artifacts are not directly available on GitExtensions' AppVeyor account, because installers are deliberately not build on pull requests. Therefore, I built it on my personal account (same SHA, no code changes): https://ci.appveyor.com/project/mast-eu/gitextensions/builds/27160008/artifacts

Maybe worth noting that the PM, although technically a plugin, is not de-selectible during installation:
image

IMHO that's ok, since it should be the only mandatory "plugin".

@RussKie
Copy link
Member

RussKie commented Sep 4, 2019 via email

@RussKie
Copy link
Member

RussKie commented Sep 4, 2019 via email

@RussKie RussKie marked this pull request as ready for review September 4, 2019 20:48
@RussKie RussKie merged commit 4341346 into gitextensions:release/3.2 Sep 4, 2019
Move plugins automation moved this from Reviewer approved to Done Sep 4, 2019
@maraf maraf deleted the PluginManagerReintegration branch September 5, 2019 05:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Move plugins
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

5 participants