Squirrel.Windows - Reboot Proposal

[shiftkey]

I wanted to provide an update to this situation that will hopefully address some concerns.

First off, many thanks to @anaisbetts for her hard work over the years on creating and maintaining Squirrel.Windows. As issues like #1466 and #1469 show, there’s a lot of love for the project and a lot of interest in keeping this going.

For those who don’t know me, I work at GitHub and have been around since the early days of Squirrel.Windows. I’m not an active contributor, but I’m a happy user of it due to its adoption in the Electron ecosystem and want to see it continue.

I had a chat with @robmen yesterday about how we might be able to help keep things going, and we agreed upon some short-term goals. These goals are rather modest, but they feel achievable over the coming weeks and months:

• Review the outstanding open issues and get the issue tracker into a state that reflects the current situation
• Hold a regular fortnightly meeting to discuss the project with contributors and users
• Investigate whether there is some low-hanging fruit that can be worked on
• Get the project to a place where we could potentially ship an update

We’re only two people with plenty of other things on our plate currently, so we’re looking for others to help with this work to build a more vibrant community around the project.

If this sounds like something you’d like to get involved with, please comment in this thread so we can gauge the interest.

If you have any questions, feel free to comment here - or you can reach out privately to shiftkey@github.com if you don’t feel comfortable asking here.

[patrickklaeren]

I was saddened to see Squirrel.Windows be deprecated - just last month we managed to migrate most of our work apps to use Squirrel after having started using it late 2018. We've had a good experience. Upon the announcement me and a friend embarked on building a "spiritual-successor", if you will, that caters for our needs, both in "enterprise" and for personal applications.

I'm glad to see Squirrel.Windows still has life left and would love to contribute where possible if you elaborate what help you're after.

[shiftkey]

I'm glad to see Squirrel.Windows still has life left and would love to contribute where possible if you elaborate what help you're after.

The first thing that comes to mind is taking stock of the issue tracker. If anyone is able to spare some time, I'd love help with reviewing the open issues and seeing what we can close out.

Things to look out for in open issues, and my suggested action:

• Has the issue been resolved?
  • Helpful: Comment in the issue that you think this can be closed and mention me
• Is the issue not clear? Is something missing that you think would help out?
  • Helpful: Ask a question that would help with understanding the problem and ensure that you @-mention the submitter so they're aware of it
• Is this an old problem or a problem with an old version of the library?
  • Helpful: Ask the submitter if they are still able to reproduce the issue with the latest version of the library.
• Is the issue asking a general question something the maintainers cannot really help with?
  • Helpful: Ask the submitter if they are fine to close out the issue, and ensure that you @-mention the submitter so they're aware of it

I can come through and help with sorting/closing issues, but having many eyes scanning over the open issues here would be a great help to get things moving...

[anaisbetts]

I'm glad to see that some folx found Squirrel so useful that they want to revive it :) I wanted to take a moment and share A few Guiding Principles that have served Squirrel.Windows well over the years.

• Be as Dumb As Possible - since the beginning, Squirrel.Windows has always tried to do literally the simplest, least clever thing that could possibly work. Simple things are great, because it means that people can build their own more complicated / niche things on top of it that only make sense for their scenario. Let Developers who Use Squirrel be the clever ones.
  
  Another extremely good reason to avoid complexity is, reliably installing and updating software on Windows is already walking a tightrope on a windy day while people throw tomatoes at you, complexity will end up creating more failed updates and users getting left behind. Which leads me to....

• Be extremely Conservative - Squirrel.Windows routinely turns down features, moreso than other projects that I've maintained, because breaking the updater is Catastrophic. Slack broke the Mac updater shortly before moving to the Electron-based version of the app, and it took them years to remediate this mistake, it sucks really bad. Anything that touches the core path of updating the app should start off in your head with -5 votes. It sucks to turn people down, especially when they've already put in some work for their idea, but complexity adds up, and every feature you add will make it harder to feel confident that you haven't broken the updater.

• Stick to The Principles and Say No - If you buy Fix the folder layout so we can have more than one project under src #2 above, you're already sold on this, but for this project in particular, it is so important to Say No. People want Squirrel.Windows to be a lot of different things, and Squirrel.Windows has stayed a reliable product because it is willing to tell people, "We just don't solve that problem. Sorry." Ideas like making Squirrel an NT Service to do updating would definitely mean it would apply to more scenarios, but it would also make Squirrel much much more complicated and prone to leaving users stranded.

• Focus on the User Experience - Another reason to Say No is because Squirrel.Windows has drawn a very strong Line In The Sand around user experience. The goal of an installer is, from the second the user double-clicks Setup.exe, we want to get the App in front of them, as fast as possible, and do nothing else. When the user uninstalls? Our goal is to remove ourselves, as fast as possible - that's it!
  
  And in terms of updating, our goal is to be like Chrome - you open it and it's the latest version. No prompts, no UAC dialogs, no Wizards or Progress Bars. Every time we show an update progress bar, we say to the user, "What you're doing isn't important. Look at Me instead". Any feature that tries to do this should be met with "That's Not What We Do Here"

Hope that helps explain some of the Zen of Squirrel.Windows. Excited to see what folx come up with in the future!

[peters]

Thanks for all the hard work you put into this project @anaisbetts 🎉

[frederikolsen88]

Yes, thank you for your work, @anaisbetts, and the best of luck on your journey!

And thank you for taking control, @shiftkey. What a relief 😄

[TonyValenti]

Hi All,
I just saw that aparently squirrel is deprecated? If so, was there an announcement?

I'm very interested in keeping the project alive and well, especially with .NET Core 3.0 coming out soon, however, a lot of the way that code was written makes me feel fairly uncomfortable. I'm more on board with a rewrite over maintenance. Thoughts?

[shiftkey]

@TonyValenti

especially with .NET Core 3.0 coming out soon,

This is an interesting potential audience, and I'd love to hear more thoughts about this and how Squirrel.Windows might affect them.

however, a lot of the way that code was written makes me feel fairly uncomfortable.

Can you elaborate on this? I'd love to understand these concerns and how we can mitigate them.

I'm more on board with a rewrite over maintenance. Thoughts?

There is a lot of existing usage of Squirrel.Windows in the wild currently, and I'm very reluctant to contemplate rewriting things (and introducing breaking changes and potentially breaking these users) without more explicit goals and a plan to go about it.

[TonyValenti]

@shiftkey -

1. With .NET core 3.0, desktop apps are now a supported scenario and I think that Squirrel.Windows makes a ton of sense for it. If you deploy a desktop app, you're going to want a way to update it. Also, I believe that the self-contained nature of .NET core is great for Squirrel.

2. Here's a really good example. Take a look at :
   https://github.com/Squirrel/Squirrel.Windows/blob/master/src/Squirrel/Utility.cs

There is a ton of code in there that isn't documented and should really be broken out into multiple classes. Additionally, you'll see things like Tuple returns and methods like this which make me shudder:

  public static Task ForEachAsync<T>(this IEnumerable<T> source, Func<T, Task> body, int degreeOfParallelism = 4)
        {
            return Task.WhenAll(
                from partition in Partitioner.Create(source).GetPartitions(degreeOfParallelism)
                select Task.Run(async () => {
                    using (partition)
                        while (partition.MoveNext())
                            await body(partition.Current);
                }));
        }

When I look at the code in general, it works but it needs some good love to start looking maintainable.

[robmen]

Yeah, we can definitely improve Squirrel.Windows but we don't need to start with a "rewrite". Small, iterative steps that bring along everyone to a better and better experience will work out well. A big bang rewrite would require significant backwards compatiblity verification and not allow us to see improvement along the way.

[aaronstine]

Hey guys, my company (Identifi) recently adopted squirrel and wouldn't mind investing in its future. Our current product plan keeps our devs busy for the next few months. But after that, we could free up some folks to contribute.

[anaisbetts]

@TonyValenti Squirrel.Windows is an Ineffable Paragon of Software Engineering Elegance and I will hear nothing to the contrary! 😅😅

[stefanolson]

I think the nice thing about squrirrel is how well it has worked. We've got a couple of thousand users using it on a daily basis and really we have quite minimal numbers of problems. Certainly better than anything else out there so well worth continuing with I think. I'm still running version 1.4.4. I did try and upgrade at one point but it started signing all the DLLs and taking ages to do the build, so I gave up and went back to 1.4.4. But we are now having a couple of problems with 1.4.4 so I will be downloading the code and submitting some changes if I can figure out the problem we are having.

[MeikTranel]

I would be more than willing to help migrate the project files to multitargeted projects where we can move Squirrel to .NET Core while keeping support for full framework. I must say... theres some complex stuff living in the project files. Maybe the current maintainers should make an effort to improve architecture documentation - the use cases are fairly well documented from what it looks like, but i don't see much information regarding how all the pieces fit together.
This is not a dig on the current maintainers... it's just me being interested to contribute but being a little overwhelmed with the architecture... and i assume we can all agree that more community participation is a factor that could really help this kind of project.

[Keboo]

I too would be willing to help out on keeping this project alive and moving forward.

[shiftkey]

Just wanted to give a shout out to @Thieum who has been investigating old issues and confirming things which can be closed out as being resolved! Thanks for your help!

[robmen]

@MeikTranel I am slowly working through the build process, simplifying as I can but primarily working towards a fully automated build and release process.

[TonyValenti]

@MeikTranel @robmen -
I'm working through a fairly substantial upgrade/cleanup of the squirrel.windows DLL.

[MeikTranel]

@MeikTranel I am slowly working through the build process, simplifying as I can but primarily working towards a fully automated build and release process.

That'll definitely help me. Seeing how things move through the build ending up at the consumer is exactly the information i need regarding project file cleanup/migration.

I'm in a super tight spot in my education right now with finals coming up next week, but i'm hoping to join the rest of the pack starting next thursday. There's a number of open source things i've got going, but currently on ice because of this reason.

@TonyValenti That's great, but i honestly feel like the more critical pillar of the codebase is the C++ stuff. There's some self-titled "janky" unzip code here, some "stub-setup" there... Maybe we should focus first on trimming things out which should not be Squirrel's primary domain. Without knowing to much of the architecture i'd say unzipping should be something we let .NET or any library do. It just feels to me that it shouldn't be my job to look at decompression code.

[TonyValenti]

@MeikTranel
I think that's a good idea. I'm targeting the current .NET side of things because it aligns with a near-term need for our business.

Regarding the C++ side of things, I think it would be worth exploring to see if C++ can be removed from the stack and entirely replaced with .NET Core 3.0.

[robmen]

@TonyValenti arguably, Squirrel would benefit if everything was written in native code. That would remove the need for multi-megabyte external dependencies (.NET Framework is fortunately normally pre-installed so .NET Framework is usually not a distribution burden). .NET Core would add tens of megabytes and would dwarf many of the applications distributed via Squirrel.

Now installing .NET Core 3.0 runtime as a dependency (like .NET Framework today) will be an important feature in the not too distant future. But making Squirrel depend on it will have to take into account the whole of the Squirrel ecosystem.

[TT39]

I would be willing to help out. Squirrel is really useful software.

[MeikTranel]

@robmen i agree 100%. We should use .NET Framework where it can be useful. .NET Standard for the referenced libraries and .NETCore/.NETFramework (to be able to use dotnet cli for development and consumption) for the build time components. As for Native C++ i don't see a reason why we should forcibly remove it from the stack. As long as we don't try to solve problems that have been solved already by the dependencies we already pull in the only real reasons to remove it would be consistency of the stack and the potential for xplat support; both of which are really weak arguments at the moment i reckon.

[anaisbetts]

As for Native C++ i don't see a reason why we should forcibly remove it from the stack

You can't remove the C++ code because you can't guarantee that the version of .NET that you want is installed on the machine. The C++ code exists to install the .NET Framework.

.NET Core would add tens of megabytes and would dwarf many of the applications distributed via Squirrel.

This is actually far worse, it's hundreds of megabytes right now. Mono's linker is much better at the moment, you can get Hello World down to 4MB, but you're still not beating the C++ bootstrapper.

[Thieum]

.NET Core would add tens of megabytes and would dwarf many of the applications distributed via Squirrel.

Would CoreRT, when ready, be an option to adress that issue?

[anaisbetts]

@Thieum afaik at the moment, CoreRT is similar or slightly bigger than the Mono Linker

[anaisbetts]

Squirrel is supported on .NET Core via the non-C# API, check out this Gist for an example of how it works:

https://gist.github.com/anaisbetts/9015f33a95c523a133bb624e3c77213d

[RudolfJan]

@anaisbetts Thank you for sharing this . I assume the general idea is the create a .Net Framework console wrapper application to handle squirrel updates? I'll see if I can get it working, should be possible ..

[JayBeavers]

While I get the dynamics here and the importance of never breaking the installer, both VSCode and GitHub Desktop are blocked from supporting Arm64 native builds by #1616 which is a minor dependency update from 2017. Any chance someone with maintainer privs could just push through the PR for this low risk update from a dependency bugfix that's three years mature?

[peters]

Is anyone interested in an trying out our xplat update solution? It's been in production at our company since April 2019. It has a lot of features such as: built-in support for delta updates, release channels (test, staging, production) and automatic deployment using GitHub Actions. It uses Avalonia for xplat gui (offline and web installer support). Checkout https://github.com/fintermobilityas/snapx. The documentation is a bit scarce but a fully working demo can be found at https://github.com/fintermobilityas/snapx.demoapp. If you want to chat you can join our gitter community: https://gitter.im/fintermobilityas-snapx/community. NB! This is not a commercial product and will forever remain free. It's not a business area for us as a we are fin-tech company that wants to give back to the open source community :)

[bfourie]

Hi @anaisbetts , @robmen
Our compagny want to use Squirrel.Windows to deploy and update our Software but with the following announcement we don't understand if this project is maintenaining.
I Hope you can enlighten us on that subject,
Regards.

[Sunbreak]

@anaisbetts is investigating Flutter. We'll see

[shiftkey]

I'm going to have to step down from any sort of maintainership work here for personal and professional reasons.

When I pitched this proposal I imagined having the bandwidth to support this important piece of community infrastructure, but since then my work responsibilities have taken me far from the Electron community, and my free time for anything open-source related is now limited. I've also been trying to look after my health these past few months, which is why I've been avoiding this hard discussion as well as other neglected projects on my plate.

Best of luck with everything!

[IanKemp]

Since ClickOnce is back with .NET 5.0, what is the future of this project?

[robmen]

@IanKemp Nothing changes. Squirrel addresses the weaknesses in ClickOnce, not the fact that it continues to exist.

[caesay]

I'm now using this to deploy one of my net5 projects, so I've forked and updated everything to netstandard2/net5 and fixed a bunch of issues. I wouldn't consider it ready for general consumption but it's a good start. I also merged a couple PR's from here related to dotnet support. I will continue to improve it as my needs evolve, and I'm happy to review and accept PR's or discuss the shortcomings with anyone interested.

https://github.com/clowd/Clowd.Squirrel

At the time of writing, these are the changes that have been made, and there have been some feature regressions noted also.

Dependency / Framework Changes:

• Nuget submodule removed and inlined. Updated to netstandard2.0
• DeltaCompressionDotNet dependency removed and inlined. Updated to netstandard2.0
• Removed ILRepack (replaced with PublishSingleFile)
• All other nuget dependencies have been removed or updated to latest versions
• Squirrel updated to netstandard2.0
• Update/SyncReleases updated to net5.0 (w/PublishSingleFile)

Code Changes:

• Removed lots of code from NugetCore that was not used to cut down on dependencies.
• Removed all usages of Assembly.Location as this does not work with PublishSingleFile
• CreateShortcutForThisExe allows configurable shortcut locations
• Automatically get Uninstall icon from SquirrelAware executable instead of downloading from url
• Update.exe icon can be optionally provided by dropping a 'ins-update.ico' file into project directory. Useful as rcedit currently can not replace icons in a PublishSingleFile executable so this must be done during Squirrel build.
• SyncReleases supports downloading releases from 'web', 'github' or 'b2' (backblaze). It also now supports uploading new releases, and can be used to fully-automate a release deployment (eg. download last release, releasify build, upload new release)
• Other misc bug fixes
• build script has been replaced entirely, although produces a similar output that could be used with electron in theory
• Setup.exe now bootstraps .net 5.0 desktop runtime instead of full fx.
• Support for detecting dotnet core / net5 SquirrelAware binaries

Known Regressions / Considerations before use:

• Only x64 platform is supported by Setup/Update.
• framework-version option is ignored. Currently, only .net 5.0 desktop runtime will be installed.
• loadingGif option is ignored / there is no splash window.
• I have not tested wix/msi support as I don't use it but I assume it will work.
• rcedit can't be used to edit Update.exe icon

Possible improvements:

• Explore ways to reduce binary size for NativeAOT or SelfContained as to remove runtime .net 5.0 dependency from Update.exe (will allow us a lot more flexibility when installing runtimes if the updater itself has no dependencies). Currently a SelfContained single file Update.exe binary is around 40mb but by removing dependencies carefully we should be able to get this comfortably under 10mb which might be good enough.
• Add support for customisable framework-version and loadingGif again. (will likely need to do loadingGif with WinAPI to keep binary size down)
• Fix and deploy as a nuget package: Clowd.Squirrel
• Add .editorconfig and update all formatting in sln as there is now some inconsistency

[IanKemp]

I'm now using this to deploy one of my net5 projects, so I've forked and updated everything to netstandard2/net5 and fixed a bunch of issues.

LIKE A BOSS.

[caesay]

Currently, in my fork, Update.exe ships as a .net5 PublishSingleFile=true, SelfContained=false. This produces a relatively small (2.6mb) single file binary that requires the .net5 runtime is installed during setup.

I have further explored the possibility of removing the runtime .net 5 dependency from Update.exe. This would be a huge win strategically because it would mean that non-dotnet apps (like electron) do not need to have .net core/fx bootstrapped at all during setup. For dotnet apps we would only need to bootstrap the framework the app depends on (or nothing, if the app we're deploying is self contained).

Using NativeAOT+UPX and no code changes we can achieve a final zero-dependency Update.exe binary size of 12mb. Not amazing, but if it saves us from downloading a 60mb dotnet runtime installer, worthwhile! Additional savings could be had by reducing code dependencies. The biggest issue will inevitably be that we use COM interop for creating shortcuts and updating tray icon settings. This is currently not supported by NativeAOT.

This leaves us with the following options then

• Rewrite Update.exe in C++ (an enormous endeavour, but would produce the best end results. I don't have time for this)
• Write a small native helper exe for shell/com interop that ships along side a NativeAOT compiled Update.exe
• Ship a standard SelfContained Update.exe (least work; but Update.exe binary size would be ~50mb)
• Bootstrap .net 5 in addition to framework-version (if different). non-dotnet apps will still need the net5 runtime installed during setup, and for dotnet apps we might end up needing to install two runtimes during setup. (eg. net5 to support Update.exe and netcore31 or net6 etc for the installed app)

My preference is the last option, it is easy to do, is unlikely to introduce issues, and produces the best experience if the target runtime(s) are already installed.

Interested to hear if anyone else has thoughts on the best approach or is willing to help with this? Once this is done, the fork will be close to feature parity, with the only real remaining drawback being the x64 platform requirement. Perhaps we can ship separate x86 and x64 versions of Updater.exe/Setup.exe to solve this if x86 support is needed.

[DavidZidar]

Wouldn't it make more sense to keep Update.exe as a .NET Framework app and make the API surface .NET Standard 2.0? That way Update.exe can stay small and the API can be used in both .NET Core and legacy .NET Framework applications. Microsoft will surely keep supporting .NET Framework for many years to come.

[hez2010]

@caesay COM interop is already (partially) supported by NativeAOT through COM wrappers.

[caesay]

@DavidZidar The API surface (Squirrel itself) is already netstandard2.0 in my fork. Retargeting Update.exe to the full framework (eg net48) instead of net5 is trivial, we'd need to use Fody or ILRepack instead of PublishSingleFile, and we end up in the same situation as 4th option above. We'd need to bootstrap net48 if it's not installed to support Update.exe, in addition to the framework/runtime required by the app we're installing... Although net48 is presumably shipped with all recent versions of Windows, so it might be safe enough to expect it to be installed.

[DavidZidar]

@caesay That's great! I suggested keeping Update.exe as a .NET Framework because I think it might be the most bang for the least effort. C++ or maybe even Go or Rust would probably be better, but as you said it would be a big endeavor.

Here is some interesting reading about the .NET Framework support lifecycle.

https://docs.microsoft.com/en-US/lifecycle/faq/dotnet-framework

Relevant bits:

Beginning with version 4.5.2 and later, .NET Framework is defined as a component of the Windows operating system (OS).

Customers and developers must have completed the in-place update to .NET Framework 4.6.2 or later by April 26, 2022 to continue receiving technical support and security updates.

So .NET Framework is part of the OS which means it's not going away any time soon and even .NET Framework 4.6.2 which is quite old will be supported after 2022.

Is there anything in particular that requires net48 though? If not, then net472 might be a good pick for even better reach.

[gbraad]

Note: We are currently using 4.7.2 for an application ('dumb' tray client using a local go-based server on named pipes), but we want to move away from this. THe reason is that we get a lot of reports of false positives by virus scanners due to the increase of malware using .NET. We did some tests with scanning assemblies on virustotal.com and it was disappointing.

[IanKemp]

• Ship a standard SelfContained Update.exe (least work; but Update.exe binary size would be ~50mb)

Do this. 50MB is nothing in today's world where app updates themselves are routinely hundreds of megabytes (not to mention Windows Update) and it gives you, the maintainer, the freedom to chop-and-change the bundled runtime as you wish.

If other users require smaller update executables, or ones written in .NET Framework, or whatever, they are welcome to submit patches adding that functionality. You've already done enough hard work to reboot Squirrel to a place where it's functional.

[caesay]

@IanKemp Making it a user-configurable option is a nice idea - I'll do that. It would be trivial to compile a SelfContained & a non-SelfContained version of Update.exe and swap which one gets copied into the releases based on a command line switch. Also need to add support to Setup.exe for bootstrapping more than one runtime at setup (eg. net48,net5) if you choose to ship a non-self contained version. Will update here when I've implemented this.

[caesay]

I have released a nuget package and a github release containing all of the previously mentioned changes, as well as a few more, such as:

• Setup.exe now shows the splash image (instead of Update.exe). This means it can be shown quicker (no need to wait for zip to extract), and we don't need to carry around a heavy UI toolkit like WPF. (important for self contained option!)
• Setup.exe now defaults to installing no framework at all.. but can be used to install any modern framework (eg. --framework-version net5) or even more than one framework (eg --framework-version net48,net5).
• Can be used to deploy a fully self-contained and dependency free Updater. This only adds +12mb uncompressed (+7mb compressed). Very reasonable!
• and more!

I have also updated the quickstart guide, and will look to start updating other docs soon.

[kant2002]

@caesay if you still want to pursue NativeAOT and what to solve your COM issue let me know. I think starting using ComWrappers instead of Built-in COM interop is a way to go in long-term. Built-in COM interop would be disable by default in .NET 6 when publishing self-contained applications.
Using ComWrappers make code even more trim-friendly, and maybe shave couple bytes from application itself.

Example how it's done in .NET 6 is here dotnet/runtime#54636 mostly that's pretty mundane changes.

[Lazizbek97]

@anaisbetts Thank you for creating such a great plugin(Squirrel.Windows).
I have tried several times to use and integrate it into my app but didn't manage to do it. I would really appreciate it , if you share any example or doc on how to use it.

Or anyone who used it, please share your experience on usage.