Merge pull request #409 from microsoft/main

Cut new release
git-ecosystem · Aug 11, 2021 · 7ad55fb · 7ad55fb
2 parents 27fbadc + 7fe5379
commit 7ad55fb
Show file tree

Hide file tree

Showing 26 changed files with 622 additions and 126 deletions.
diff --git a/.github/ISSUE_TEMPLATE/auth-problem.md b/.github/ISSUE_TEMPLATE/auth-problem.md
@@ -8,7 +8,7 @@ assignees: ''
 
 **Which version of GCM Core are you using?**
 
-From a terminal, run `git-credential-manager-core version` and paste the output.
+From a terminal, run `git-credential-manager-core --version` and paste the output.
 
 <!-- Ex:
 Git Credential Manager version 2.0.8-beta+e1f8492d04 (macOS, .NET Core 4.6.27129.04)

diff --git a/.github/ISSUE_TEMPLATE/experimental.md b/.github/ISSUE_TEMPLATE/experimental.md
@@ -8,7 +8,7 @@ assignees: ''
 
 **Which version of GCM Core are you using?**
 
-From a terminal, run `git-credential-manager-core version` and paste the output.
+From a terminal, run `git-credential-manager-core --version` and paste the output.
 
 <!-- Ex:
 Git Credential Manager version 2.0.8-beta+e1f8492d04 (macOS, .NET Core 4.6.27129.04)

diff --git a/.github/configure_repoclient.py b/.github/configure_repoclient.py
diff --git a/.github/fetch_release.py b/.github/fetch_release.py
diff --git a/.github/workflows/release-apt-get.yml b/.github/workflows/release-apt-get.yml
@@ -6,47 +6,90 @@ on:
   workflow_dispatch:
     inputs:
       release:
-        description: 'Release Tag'
+        description: 'Release Id'
         required: true
         default: 'latest'
 
 jobs:
   release:
     runs-on: ubuntu-latest
     steps:
-    - name: setup python
-      uses: actions/setup-python@v2
-      with:
-        python-version: 3.8
-
     - uses: actions/checkout@v2
 
     - uses: azure/login@v1
       with:
         creds: ${{ secrets.AZURE_CREDENTIALS }}
 
-    - name: 'Download Repo Client'
+    - name: "Download Repo Client"
       env:
         AZ_SUB: ${{ secrets.AZURE_SUBSCRIPTION }}
       run: |
         az storage blob download --subscription  "$AZ_SUB" --account-name gitcitoolstore -c tools -n azure-repoapi-client_2.0.1_amd64.deb -f repoclient.deb --auth-mode login
 
     - name: "Install Repo Client"
-      env:
-        APT_REPO_ID: ${{ secrets.APT_REPO_ID }}
-        AZURE_AAD_ID: ${{ secrets.AZURE_AAD_ID }}
-        AAD_CLIENT_SECRET: ${{ secrets.AAD_CLIENT_SECRET }}
       run: |
         sudo apt-get install python3-adal --yes
         sudo dpkg -i repoclient.deb
-        python .github/configure_repoclient.py
         rm repoclient.deb
 
+    - name: "Configure Repo Client"
+      uses: actions/github-script@v3
+      env:
+        AZURE_AAD_ID: ${{ secrets.AZURE_AAD_ID }}
+        AAD_CLIENT_SECRET: ${{ secrets.AAD_CLIENT_SECRET }}
+      with:
+        script: |
+          for (const key of ['AZURE_AAD_ID', 'AAD_CLIENT_SECRET']) {
+            if (!process.env[key]) throw new Error(`Required env var ${key} is missing!`)
+          }
+          const config = {
+            AADResource: 'https://microsoft.onmicrosoft.com/945999e9-da09-4b5b-878f-b66c414602c0',
+            AADTenant: '72f988bf-86f1-41af-91ab-2d7cd011db47',
+            AADAuthorityUrl: 'https://login.microsoftonline.com',
+            server: 'azure-apt-cat.cloudapp.net',
+            port: '443',
+            AADClientId: process.env.AZURE_AAD_ID,
+            AADClientSecret: process.env.AAD_CLIENT_SECRET,
+            repositoryId: ''
+          }
+          const fs = require('fs')
+          fs.writeFileSync('config.json', JSON.stringify(config, null, 2))
+
+    - name: "Get Release Asset"
+      id: get-asset
+      env:
+        RELEASE: ${{ github.event.inputs.release }}
+      uses: actions/github-script@v3
+      with:
+        github-token: ${{secrets.GITHUB_TOKEN}}
+        script: |
+          const { data } = await github.repos.getRelease({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              release_id: process.env.RELEASE || 'latest'
+            })
+          const assets = data.assets.filter(asset => asset.name.endsWith('.deb'))
+          if (assets.length !== 1) {
+            throw new Error(`Unexpected number of .deb assets: ${assets.length}`)
+          }
+          const fs = require('fs')
+          const buffer = await github.repos.getReleaseAsset({
+              headers: {
+                accept: 'application/octet-stream'
+              },
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              asset_id: assets[0].id
+            })
+          console.log(buffer)
+          fs.writeFileSync(assets[0].name, Buffer.from(buffer.data))
+          core.setOutput('name', assets[0].name)
+
     - name: "Publish to apt feed"
       env:
         RELEASE: ${{ github.event.inputs.release }}
       run: |
-        pip install requests
-        python .github/fetch_release.py
-        wget "$(cat asset_url.txt)"
-        repoclient -v v3 -c config.json package add --check --wait 300 "$(cat asset_name.txt)" 
+        for id in ${{ secrets.BIONIC_REPO_ID }} ${{ secrets.HIRSUTE_REPO_ID }}
+        do
+          repoclient -v v3 -c config.json package add --check --wait 300 "${{steps.get-asset.outputs.name}}" -r $id
+        done
diff --git a/.github/workflows/release-winget.yaml b/.github/workflows/release-winget.yaml
@@ -21,7 +21,11 @@ jobs:
           Publisher: Microsoft Corporation
           Moniker: git-credential-manager-core
           PackageUrl: https://aka.ms/gcmcore
-          Tags: [ gcm, gcmcore, git, credential ]
+          Tags:
+          - gcm
+          - gcmcore
+          - git
+          - credential 
           License: Copyright (C) Microsoft Corporation
           ShortDescription: Secure, cross-platform Git credential storage with authentication to GitHub, Azure Repos, and other popular Git hosting services.
           Installers:

diff --git a/README.md b/README.md
@@ -86,7 +86,35 @@ sudo /usr/local/share/gcm-core/uninstall.sh
 
 ---
 
-### Linux Debian package (.deb)
+<!-- this explicit anchor should stay stable so that external docs can link here -->
+<a name="linux-install-instructions"></a>
+### Linux
+
+#### Debian package (.deb)
+
+`apt-get` support is available for Ubuntu Bionic Beaver (18.04) and Hirsute 
+Hippo (21.04). Take the following steps to set up and install based on the
+version you are running:
+
+#### Ubuntu 18.04 (Bionic)
+
+```shell
+curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
+sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
+sudo apt-get update
+sudo apt-get install gcmcore
+```
+
+#### Ubuntu 21.04 (Hirsute)
+
+```shell
+curl -sSL https://packages.microsoft.com/config/ubuntu/21.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list
+curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc
+sudo apt-get update
+sudo apt-get install gcmcore
+```
+
+#### Other Ubuntu/Debian distributions
 
 Download the latest [.deb package](https://github.com/microsoft/Git-Credential-Manager-Core/releases/latest), and run the following:
 
@@ -97,9 +125,7 @@ git-credential-manager-core configure
 
 Note that Linux distributions [require additional configuration](https://aka.ms/gcmcore-linuxcredstores) to use GCM Core.
 
----
-
-### Linux tarball (.tar.gz)
+#### Other distributions
 
 Download the latest [tarball](https://github.com/microsoft/Git-Credential-Manager-Core/releases/latest), and run the following:
 
@@ -144,8 +170,11 @@ To uninstall, open Control Panel and navigate to the Programs and Features scree
 
 ## How to use
 
-Git Credential Manager Core is called implicitly by Git, when so configured. It is not intended to be called directly by the user.
-For example, when pushing (`git push`) to [Azure DevOps](https://dev.azure.com), a window is automatically opened and an OAuth2 flow is started to get your personal access token.
+Once it's installed and configured, Git Credential Manager Core is called implicitly by Git.
+You don't have to do anything special, and GCM Core isn't intended to be called directly by the user.
+For example, when pushing (`git push`) to [Azure DevOps](https://dev.azure.com), [Bitbucket](https://bitbucket.org), or [GitHub](https://github.com), a window will automatically open and walk you through the sign-in process.
+(This process will look slightly different for each Git host, and even in some cases, whether you've connected to an on-premises or cloud-hosted Git host.)
+Later Git commands in the same repository will re-use existing credentials or tokens that GCM Core has stored for as long as they're valid.
 
 Read full command line usage [here](docs/usage.md).
 
@@ -160,6 +189,7 @@ See detailed information [here](https://aka.ms/gcmcore-httpproxy).
 - [Command-line usage](docs/usage.md)
 - [Configuration options](docs/configuration.md)
 - [Environment variables](docs/environment.md)
+- [Enterprise configuration](docs/enterprise-config.md)
 - [Network and HTTP configuration](docs/netconfig.md)
 - [Architectural overview](docs/architecture.md)
 - [Host provider specification](docs/hostprovider.md)

diff --git a/docs/bitbucket-development.md b/docs/bitbucket-development.md
@@ -0,0 +1,80 @@
+# Bitbucket Authentication, 2FA and OAuth
+
+By default for authenticating against private Git repositories Bitbucket supports SSH and username/password Basic Auth over HTTPS.
+Username/password Basic Auth over HTTPS is also available for REST API access.
+Additionally Bitbucket supports App-specific passwords which can be used via Basic Auth as username/app-specific-password.
+
+To enhance security Bitbucket offers optional Two-Factor Authentication (2FA). When 2FA is enabled username/password Basic Auth access to the REST APIs and to Git repositories is suspended.
+At that point users are left with the choice of username/apps-specific-password Basic Auth for REST APIs and Git interactions, OAuth for REST APIs and Git/Hg interactions or SSH for Git/HG interactions and one of the previous choices for REST APIs.
+SSH and REST API access are beyond the scope of this document.
+Read about [Bitbucket's 2FA implementation](https://confluence.atlassian.com/bitbucket/two-step-verification-777023203.html).
+
+App-specific passwords are not particularly user friendly as once created Bitbucket hides their value, even from the owner.
+They are intended for use within application that talk to Bitbucket where application can remember and use the app-specific-password.
+[Additional information](https://confluence.atlassian.com/display/BITBUCKET/App+passwords).
+
+OAuth is the intended authentication method for user interactions with HTTPS remote URL for Git repositories when 2FA is active.
+Essentially once a client application has an OAuth access token it can be used in place of a user's password.
+Read more about information [Bitbucket's OAuth implementation](https://confluence.atlassian.com/bitbucket/oauth-on-bitbucket-cloud-238027431.html).
+
+Bitbucket's OAuth implementation follows the standard specifications for OAuth 2.0, which is out of scope for this document.
+However it implements a comparatively rare part of OAuth 2.0 Refresh Tokens.
+Bitbucket's Access Token's expire after 1 hour if not revoked, as opposed to GitHub's that expire after 1 year.
+When GitHub's Access Tokens expire the user must anticipate in the standard OAuth authentication flow to get a new Access Token. Since this occurs, in theory, once per year this is not too onerous.
+Since Bitbucket's Access Tokens expire every hour it is too much to expect a user to go through the OAuth authentication flow every hour.
+Bitbucket implements refresh Tokens.
+Refresh Tokens are issued to the client application at the same time as Access Tokens.
+They can only be used to request a new Access Token, and then only if they have not been revoked.
+As such the support for Bitbucket and the use of its OAuth in the Git Credentials Manager differs significantly from how VSTS and GitHub are implemented.
+This is explained in more detail below.
+
+## Multiple User Accounts
+
+Unlike the GitHub implementation within the Git Credential Manager, the Bitbucket implementation stores 'secrets', passwords, app-specific passwords, or OAuth tokens, with usernames in the [Windows Credential Manager](https://msdn.microsoft.com/en-us/library/windows/desktop/aa374792(v=vs.85).aspx) vault.
+
+Depending on the circumstances this means either saving an explicit username in to the Windows Credential Manager/Vault or including the username in the URL used as the identifying key of entries in the Windows Credential Manager vault, i.e. using a key such as `git:https://mminns@bitbucket.org/` rather than `git:https://bitbucket.org`.
+This means that the Bitbucket implementation in the GCM can support multiple accounts, and usernames,  for a single user against Bitbucket, e.g. a personal account and a work account.
+
+## Authentication User Experience
+
+When the GCM is triggered by Git, the GCM will check the `host` parameter passed to it.
+If it contains `bitbucket.org` it will trigger the Bitbucket related processes.
+
+### Basic Authentication
+
+If the GCM needs to prompt the user for credentials they will always be shown an initial dialog where they can enter a username and password. If the `username` parameter was passed into the GCM it is used to pre-populate the username field, although it can be overridden.
+When username and password credentials are submitted the GCM will use them to attempt to retrieve a token, for Basic Authentication this token is in effect the password the user just entered.
+The GCM retrieves this `token` by checking the password can be used to successfully retrieve the User profile via the Bitbucket REST API.
+
+If the username and password credentials sent as Basic Authentication credentials works, then the password is identified as the token. The credentials, the username and the password/token, are then stored and the values returned to Git.
+
+If the request for the User profile via the REST API fails with a 401 return code it indicates the username/password combination is invalid, nothing is stored and nothing is returned to Git.
+
+However if the request fails with a 403 (Forbidden) return code, this indicates that the username and password are valid but 2FA is enabled on the Bitbucket Account.
+When this occurs the user it prompted to complete the OAuth authentication process.
+
+### OAuth
+
+OAuth authentication prompts the User with a new dialog where they can trigger OAuth authentication.
+This involves opening a browser request to `_https://bitbucket.org/site/oauth2/authorize?response_type=code&client_id={consumerkey}&state=authenticated&scope={scopes}&redirect_uri=http://localhost:34106/_`.
+This will trigger a flow on Bitbucket where the user must login, potentially including a 2FA prompt, and authorize the GCM to access Bitbucket with the specified scopes.
+The GCM will spawn a temporary, local webserver, listening on port 34106, to handle the OAuth redirect/callback.
+Assuming the user successfully logins into Bitbucket and authorizes the GCM this callback will include the Access and Refresh Tokens.
+
+The Access and Refresh Tokens will be stored against the username and the username/Access Token credentials returned to Git.
+
+# On-Premise Bitbucket
+
+On-premise Bitbucket, more correctly known as Bitbucket Server or Bitbucket DC, has a number of differences compared to the cloud instance of Bitbucket, https://bitbucket.org. 
+
+As far as GCMC is concerned the main difference it doesn't support OAuth so only Basic Authentication is available.
+
+It is possible to test with Bitbucket Server by running it locally using the following command from the Atlassian SDK:
+
+	❯ atlas-run-standalone --product bitbucket
+
+See https://developer.atlassian.com/server/framework/atlassian-sdk/atlas-run-standalone/.
+
+This will download and run a standalone instance of Bitbucket Server which can be accessed using the credentials `admin`/`admin` at
+
+	https://localhost:7990/bitbucket
diff --git a/docs/configuration.md b/docs/configuration.md
@@ -3,12 +3,13 @@
 [Git Credential Manager Core](usage.md) works out of the box for most users.
 
 Git Credential Manager Core (GCM Core) can be configured using Git's configuration files, and follows all of the same rules Git does when consuming the files.
+
 Global configuration settings override system configuration settings, and local configuration settings override global settings; and because the configuration details exist within Git's configuration files you can use Git's `git config` utility to set, unset, and alter the setting values. All of GCM Core's configuration settings begin with the term `credential`.
 
 GCM Core honors several levels of settings, in addition to the standard local \> global \> system tiering Git uses.
 URL-specific settings or overrides can be applied to any value in the `credential` namespace with the syntax below.
 
-Additionally, GCM Core respects several GCM-specific [environment variables](environment.md) **which take precedence over configuration options.**
+Additionally, GCM Core respects several GCM-specific [environment variables](environment.md) **which take precedence over configuration options**. System administrators may also configure [default values](enterprise-config.md) for many settings used by GCM Core.
 
 GCM Core will only be used by Git if it is installed and configured. Use `git config --global credential.helper manager-core` to assign GCM Core as your credential helper. Use `git config credential.helper` to see the current configuration.