Port GCM Core to Linux

[derrickstolee]

We hope to make a Linux version of GCM Core. Since the tool is built on .NET Core, most of our logic will carry over automatically. There are a few platform-specific bits to handle, though:

1. How should we store secrets on Linux? We need to integrate directly with something here, and the answer might be obvious to an expert.
2. How does the user experience change? To start, we can have command-line and browser prompts only. Do we need to do something special on Linux to open a browser? When we update the GUI components to something like Avalonia, will Linux GUI work immediately?
3. How to install? Likely, we will want to ship a .deb package so users can install via apt-get the same way macOS users can install via brew.

These are all important, technically challenging items.

---

Update!

We have a pre-release of a GCM Core that supports Linux! 🥳 See this comment below for more information.

[mjcheetham]

A few of these questions have answers, but I've lacked time for far to get around to doing it.

1. We use libsecret via P/Invoke to store and retrieve secrets in the user's preferred credential management system. The GNOME Keyring also 'speaks' over the libsecret protocol. The small hiccup I found was that it seems libsecret depends on DBus, which isn't happy in headless environments. The MSAL.NET team also hit this problem.
   It may also be worth, on Linux distributions, providing a second storage mechanism, configurable and unconfigured by default, to store secrets in files under a directory of the user's choosing. The user can be in charge of securing that directory. We might also want to look at (in the future) providing an extension point to plug-in or chain together a secret storage mechanism.

2. xdg-open is probably the best bet here to open the default browser (it's what Process.Start uses when UseShellExecute = true in the .NET Core runtime). Avalonia works on Linux via GTK, so there should be no extra work.

3. Agreed! This will probably end up being the task that takes the most amount of time IMO. Auth might be hard but engineering systems are like expensive china, balanced on top a mile-high pole in a hurricane... 😢

[onitake]

libsecret (or formally, the Freedesktop Secret Service API) is probably the best choice, especially since KeepassXC started supporting it as a credential provider in 2.5.0: https://keepassxc.org/blog/2019-10-26-2.5.0-released/

However, there is still a lack of support in KDE, as KWallet has zero compatibility with libsecret, and the newer KSecretService still seems to be incomplete. Other desktop environment likely have similar problems.

Pure CLI should work, gnome-keyring and secret-tool don't depend on a graphical desktop environment.

[mjcheetham]

Pure CLI should work, gnome-keyring and secret-tool don't depend on a graphical desktop environment.

A partner team at Microsoft also attempted to integrate secret storage via libsecret, but hit issues (at least on Ubuntu Server) with how DBus was configured - it would throw all sorts of errors when trying to call libsecret APIs without X11 configured and running.
IIRC there was some issue with packages having incorrect dependencies between libsecret and DBus and X11.
I agree this should work and is the preferred option for “v1” Linux support.

Storage mechanisms for other popular distros should also be on the backlog.

[onitake]

As for point no. 3: Microsoft is maintaining their own Linux package repositories, why not put it there?

It would also be great if you could work with distributions to get GCM into official package repositories. At least Debian and Fedora should cover a large number of derived distributions.

[atl-mk]

1. libsecret, although this presents issues with kwallet. Chromium and other applications normally support both which will probably cover 90%+ use cases with the ability to set flags to tell them which to use.

2. xdg-open is a safe assumption for launching a GUI, although I would argue falling backing to CLI support is perfectly fine to begin with.

3. Not everyone likes them for philosophical reasons, but a system like snap or flatpak would mean that it should be possible to ship to almost all distributions without having to wade through a dependency nightmare. Granted this adds steps for installation, but with the mainstream distributions adopting them this would still enable less experienced users to install.

[jglick]

As a (fairly experienced) Ubuntu user, from the standpoint of installation and updates I would be happy with a Snap rather than .deb (especially if it used my keyring), so long as it was using “classic” rather than “strict” confinement: the latter would mean, for example, that Git operations outside of $HOME would not work.

[onitake]

I'm not a fan of alternative package managers such as Snap or Flatpack, so my view is a bit biased here.

I do acknowledge that they are useful in cases where large applications with lots of internal dependencies are packaged, but in this case, I see very little benefit.

All dependencies have to be resolved in some way, and by choosing a particular library version, you're risking incompatibilities with the rest of each system. By linking against an OS package instead, you're making sure you're using a version that is supported by the OS distributor. This is particularly true for anything that has strong ties to the system, and GCM Core is a prime example of that: git, ssh-agent, secret-agent are all OS-provided frameworks, so by packaging a different client library than the OS, you're risking breakage on a wide range of systems.

[kyle-rader]

I also am not a fan of snap and neither is my preferred linux flavor: https://www.zdnet.com/article/linux-mint-dumps-ubuntu-snap/ (Thank you Tucker for pointing that out).
I strongly believe we should provide a .deb and maybe we could also do a .rpm if we get GCM working on Fedora or RH as well :).

[mjcheetham]

Update!

We have a pre-release of a GCM Core that supports Linux! 🥳

https://github.com/microsoft/Git-Credential-Manager-Core/releases/tag/v2.0.246-beta

At the moment we provide a Debian package, and a tarball of the git-credential-manager-core single binary. The .deb is currently unsigned and not uploaded anywhere except on GitHub. We are in the process of getting package signing set up and will be publishing it to an official Microsoft feed (so you'll be able to use apt-get!). Currently the pre-built binaries are only provided for 64-bit Intel processors.

If you wish to build from source you can clone and run dotnet build -c LinuxRelease (or LinuxDebug for debug bits), and look for the output binary in out/linux/Packaging.Linux/payload/<configuration>.

After installing the package, or building from source, you’ll want to link the binary to somewhere on your PATH and run git-credential-manager-core configure. Alternatively you can manually set the following config:

[credential]
    helper = <path to git-credential-manager-core>
[credential "https://dev.azure.com"] #required only for Azure DevOps users
    useHttpPath = true

Again, we plan to add these steps to a post install action in the .deb package.

Due to the nature of Linux distributions' open and varied nature, we've had to offer a collection of credential storage options, each with different limitations, advantages, and setup. We've outlined the different options here: https://aka.ms/gcmcore-linuxcredstores. We hope to improve on this in the future, especially where it comes to helping users select and configure an option.

Please let us know what you think, and raise an issue for any bugs you encounter.

[onitake]

How are you building the Debian package?
I'm considering submitting an ITP to the Debian project, but I didn't see a debian/ directory in the source tree...

[derrickstolee]

@onitake: we create a .deb starting with #164, generated as part of dotnet build on Linux. We create these .deb files with every GitHub Actions CI/PR workflow for easy testing. With #174, we also create these in our Azure Pipelines builds that generate full releases.

[kyle-rader]

I'm still pretty new to building packages on/for Debian systems. Is the goal of an ITP to make known that we are building a particular package and that's it. Or is the goal to go further and say, we intend to submit this package to be a part of the Debian packages?

[onitake]

@derrickstolee Thanks. I guess this means that the resulting package includes the .NET Core runtime and all dependencies?

@kyle-rader An ITP (=Intent To Package) is basically saying that the person submitting the ITP will prepare all the necessary work for the project to be included with the Debian project as a supported software package. There are a lot of hurdles to go through, and the tooling is not all that easy to use. But if the upstream project provides a straight-forward way to build a project, then it's often much less work. Note that I'm not neither affiliated with Microsoft, nor am I a Debian developer. But I've maintained a few Debian packages and know my way around a bit.

[kyle-rader]

My only concern is that we have a lot of non-trivial and only internally runnable steps to go through to produce what will be the final product as a signed .deb on https://packages.microsoft.com. I don't know that we want to try to include GCM in Debian directly or not. @derrickstolee / @mjcheetham ?

[kyle-rader]

@onitake re: .NET Core Runtime - yes. If you take a look at the build.sh you'll see we publish this as a single executable but it does contain the whole runtime. The size of that executable IIRC is around 80-100 MB. We might change this to deploy all the Dlls into usr/lib/ and create a symlink in /usr/bin. We also have the option of publishing with trimming to strip out Dlls never used which can help reduce the payload size quite a lot. (I've seen other tools go from 80MB to 18MB)

[derrickstolee]

I don't think that we intend to incorporate this into Debian by default, but do intend for it to be easy to install using apt/apt-get (I still don't know the difference... that's my expertise in this area).

[kyle-rader]

That's what my first thought was. If a package is required to be built and included by a Debian maintainer, then it simply won't be possible for them to include a signed version of GCM (unless they just pull the one we ship).
(apt is just the newer command which is a subset of apt-get with slightly simpler features). apt I think is what most distros now recommend. Was added to Ubuntu in 16.04.

[mjcheetham]

I don't think we want GCM Core to be bundled with Debian, at least not right now. Having it on the https://packages.microsoft.com feed requires it be signed by a Microsoft key, so we won't be able to do that except inside of a MS build process.

We also have the option of publishing with trimming [..]

If we're not trimming, we probably should. I don't think we make (much) use of reflection, so it should be safe.
(The only place I know we use reflection is in discovering the entry assembly name.)

[onitake]

Can you clarify what you mean with "signing"?
If it's about the package, this is something the Debian project will do. All packages published by the Debian project are signed by Debian project keys.
Or do you mean the binaries? Does this have any consequence on functionality?

[mjcheetham]

Can you clarify what you mean with "signing"?

By signing we mean the .deb package, yes. On Mac and Windows we sign both the binaries and installer/packages with Apple's signing service (with the MS team account) and the Microsoft Authenticode certificate, respectively, in our release pipelines. On macOS, binaries must be signed to enable Keychain functionality without constant confirmation prompts.

From a policy level, all 'production' released Microsoft software or packages must be signed as both internal and external customers often take Microsoft software to be trustworthy in their own build systems etc, and must verify the bits are what they expect them to be on disk.

Since GCM Core on Linux is still in the very early stages of release, and we will be making frequent iterations, I don't think it would be appropriate yet to have it bundled with distros or hosted by external projects.

Also I think ITP requires the package be stable:

Reasons why a new package might get rejected nevertheless
...

• The software is very immature (version 0.1-alpha or something like that).

Once the project is more stable, especially on Linux (a new platform for us here 😁 ) this is probably something we can look at.

[onitake]

Technically, GCM Core should be stable - it's at at version 2.0.x after all. 😉

I did a quick skim of what's needed - and I think the bigger problem is something else: There is currently no .NET Core in Debian. We have Mono and a lot of .NET libraries, but I'm not even sure if building .NET Core projects with Mono is even possible at the moment.

[onitake]

And then there will be .NET 5 soon - so let's leave this discussion open for now.

I think putting signed .debs (and possibly .rpms) on the Microsoft package repository is a good first step and will reach many of the prospective users of GCM Core.

[hickford]

Thanks for adding to Microsoft Ubuntu repo https://packages.microsoft.com/ubuntu/21.04/prod repo (https://github.com/microsoft/Git-Credential-Manager-Core/blob/main/README.md#ubuntu-2104-hirsute)

Any chance of inclusion in the Microsoft Debian repo https://packages.microsoft.com/debian/10/prod ? https://docs.microsoft.com/en-us/dotnet/core/install/linux-debian#debian-10-

[dimztimz]

I'd like to point out that Git has very basic credential manager that internally uses libsecret. You can find the source here. On Ubuntu/Debian it is not installed by default, it needs to be compiled with make before it is used. It works as advertised, noting fancy, for GitHub I need to enter personal access token as password.

Would it be possible to make this credential manager compatible with that one, i.e. credential already stored by the old one should be readable by this new one (and maybe the other way around)?

[jglick]

for GitHub I need to enter personal access token as password

If you are using GitHub, I think you are better off with https://cli.github.com/manual/gh_auth_login which sets up

[credential "https://github.com"]
	helper = 
	helper = !/usr/bin/gh auth git-credential

That currently stores a plaintext OAuth token in ~/.config/gh/hosts.yml, which would ideally be improved to use libsecret where available.

[mjcheetham]

Closing this issue as GCM Core can now run on Linux distributions. For further issues/bugs, or support for more distributions please open new issues. Thanks! 🎉

[hickford]

Opened #447 "Publish packages to packages.microsoft.com/debian"