diff --git a/.github/workflows/keyfactor-starter-workflow.yml b/.github/workflows/keyfactor-starter-workflow.yml index 4c2d327..e291ab4 100644 --- a/.github/workflows/keyfactor-starter-workflow.yml +++ b/.github/workflows/keyfactor-starter-workflow.yml @@ -5,6 +5,19 @@ jobs: call-create-github-release-workflow: uses: Keyfactor/actions/.github/workflows/github-release.yml@main + get-manifest-properties: + runs-on: windows-latest + outputs: + update_catalog: ${{ steps.read-json.outputs.prop }} + steps: + - uses: actions/checkout@v3 + - name: Read json + id: read-json + shell: pwsh + run: | + $json = Get-Content integration-manifest.json | ConvertFrom-Json + echo "::set-output name=prop::$(echo $json.update_catalog)" + call-dotnet-build-and-release-workflow: needs: [call-create-github-release-workflow] uses: Keyfactor/actions/.github/workflows/dotnet-build-and-release.yml@main @@ -18,9 +31,12 @@ jobs: call-generate-readme-workflow: if: github.event_name == 'push' || github.event_name == 'workflow_dispatch' uses: Keyfactor/actions/.github/workflows/generate-readme.yml@main + secrets: + token: ${{ secrets.APPROVE_README_PUSH }} call-update-catalog-workflow: - if: github.event_name == 'push' || github.event_name == 'workflow_dispatch' + needs: get-manifest-properties + if: needs.get-manifest-properties.outputs.update_catalog == 'True' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') uses: Keyfactor/actions/.github/workflows/update-catalog.yml@main secrets: token: ${{ secrets.SDK_SYNC_PAT }} diff --git a/.gitignore b/.gitignore index 9e70390..50f98d1 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ /hashicorp-vault-orchestrator/hashicorp-vault-orchestrator.csproj.user .vs *.licenseheader +README.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..a0dd80d --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,18 @@ +## 2.0.0 + +* Added support for storing certs in sub-paths +* Updated documentation to specify storing the token as a secret. +* Added inventory job support for the Hashicorp PKI secrets engine +* Added inventory job support for the Keyfactor secrets engine + +* **Breaking Change**: the properties have been renamed from: + * `PUBLIC_KEY` to `certificate` + * `PRIVATE_KEY` to `private_key` + * `PUBLIC_KEY_` has been removed. Now the chain is stored in `certificate` if the option is selected. + +* **Breaking Change**: Added a flag on the Keyfactor Certificate store definition to indicate whether to store the full CA chain along with the certificate + + +* **Breaking Change**: the cert store types are now: + * **HCVPKI** for the PKI and Keyfactor secrets engine + * **HCVKV** for the Key-Value secrets engine \ No newline at end of file diff --git a/README.md b/README.md index 257cffd..47866fb 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,26 @@ # Orchestrator Extension for Hashicorp Vault -The Hashicorp Vault Orchestrator extension allows you store certificates in Hashicorp Vault KeyValue secrets engine. +The Hashicorp Vault Orchestrator extension allows you to manage certificates in Hashicorp Vault KeyValue secrets engine and perform inventory on certificates stored in the PKI or Keyfactor secrets engines. #### Integration status: Production - Ready for use in production environments. -## About the Keyfactor Universal Orchestrator Capability -This repository contains a Universal Orchestrator Capability which is a plugin to the Keyfactor Universal Orchestrator. Within the Keyfactor Platform, Orchestrators are used to manage “certificate stores” — collections of certificates and roots of trust that are found within and used by various applications. +## About the Keyfactor Universal Orchestrator Extension -The Universal Orchestrator is part of the Keyfactor software distribution and is available via the Keyfactor customer portal. For general instructions on installing Capabilities, see the “Keyfactor Command Orchestrator Installation and Configuration Guide” section of the Keyfactor documentation. For configuration details of this specific Capability, see below in this readme. +This repository contains a Universal Orchestrator Extension which is a plugin to the Keyfactor Universal Orchestrator. Within the Keyfactor Platform, Orchestrators are used to manage “certificate stores” — collections of certificates and roots of trust that are found within and used by various applications. -The Universal Orchestrator is the successor to the Windows Orchestrator. This Capability plugin only works with the Universal Orchestrator and does not work with the Windows Orchestrator. +The Universal Orchestrator is part of the Keyfactor software distribution and is available via the Keyfactor customer portal. For general instructions on installing Extensions, see the “Keyfactor Command Orchestrator Installation and Configuration Guide” section of the Keyfactor documentation. For configuration details of this specific Extension see below in this readme. +The Universal Orchestrator is the successor to the Windows Orchestrator. This Orchestrator Extension plugin only works with the Universal Orchestrator and does not work with the Windows Orchestrator. + + + + +## Support for Orchestrator Extension for Hashicorp Vault + +Orchestrator Extension for Hashicorp Vault is supported by Keyfactor for Keyfactor customers. If you have a support issue, please open a support ticket with your Keyfactor representative. + +###### To report a problem or suggest a new feature, use the **[Issues](../../issues)** tab. If you want to contribute actual bug fixes or proposed enhancements, use the **[Pull requests](../../pulls)** tab. @@ -19,6 +28,11 @@ The Universal Orchestrator is the successor to the Windows Orchestrator. This Ca + +## Keyfactor Version Supported + +The minimum version of the Keyfactor Universal Orchestrator Framework needed to run this version of the extension is 10.1 + ## Platform Specific Notes The Keyfactor Universal Orchestrator may be installed on either Windows or Linux based platforms. The certificate operations supported by a capability may vary based what platform the capability is installed on. The table below indicates what capabilities are supported based on which platform the encompassing Universal Orchestrator is running. @@ -33,58 +47,71 @@ The Keyfactor Universal Orchestrator may be installed on either Windows or Linux + + --- -This integration for the Keyfactor Universal Orchestrator has been tested against Hashicorp Vault 1.10. It utilizes the *Key/Value* secrets engine to store certificates issues via Keyfactor Command. +This integration for the Keyfactor Universal Orchestrator has been tested against Hashicorp Vault 1.10. It utilizes the **Key/Value** secrets engine to store certificates issues via Keyfactor Command. ## Use Cases -The Hashicorp Vault Orchestrator Integration implements the following capabilities: +This integration supports 3 Hashicorp Secrets Engines; PKI, Key-Value store, and the Keyfactor Hashicorp Plugin (Keyfactor Secrets Engine). + +### The Key-Value secrets engine + +The Following operations are supported by this integration **only** for the Key-Value secrets engine. 1. Discovery - Discover all sub-paths containing certificate. 1. Inventory - Return all certificates stored in a path. 1. Management (Add) - Add a certificate to a defined certificate store. 1. Management (Remove) - Remove a certificate from a defined certificate store. +### The Hashicorp PKI and Keyfactor Plugin secrets engines + +Both the Hashicorp PKI and Keyfactor plugin are designed to allow managing certifications directly on the Hashicorp Vault instance. +This integration does support the following in order to view your certificates from the platform: + +1. Inventory - Return all certificates stored in a path. + +[View the repository on Github](https://github.com/Keyfactor/hashicorp-vault-secretsengine) for more information about the Hashicorp Vault Keyfactor Secrets Engine plugin. + ## Versioning The version number of a the Hashicorp Vault Orchestrator Extension can be verified by right clicking on the `Keyfactor.Extensions.Orchestrator.HCV.dll` file in the extensions installation folder, selecting Properties, and then clicking on the Details tab. ## Keyfactor Version Supported -This integration was built on the .NET Core 3.1 target framework and are compatible for use with the Keyfactor Universal Orchestrator. +This integration was built on the .NET Core 3.1 target framework and are compatible for use with the Keyfactor Universal Orchestrator and the latest version of the Keyfactor platform. ## Security Considerations 1. It is not necessary to use the Vault root token when creating a Certificate Store for HashicorpVault. We recommend creating a token with policies that reflect the minimum permissions necessary to perform the intended operations. -1. The certificates are stored in 3 fields in the Key Value store. - -- `PUBLIC_KEY` - The certificate public key -- `PRIVATE_KEY` - The certificate private key -- `KEY_SECRET` - The certificate private key password - -## Extension Configuration -### On the Orchestrator Agent Machine +1. For the Key-Value secrets engine, the certificates are stored as an entry with these fields. -1. Stop the Orchestrator service. +- `certificate` - The PEM formatted certificate and intermediate CA chain (if selected) +- `private_key` - The certificate private key -- The service will be called "KeyfactorOrchestrator-Default" by default. +**Note**: Key/Value secrets that do not include the keys `certificate` and `private_key` will be ignored during inventory scans. -1. Navigate to the "extensions" sub-folder of your Orchestrator installation directory +## Extension Configuration -- example: `C:\Program Files\Keyfactor\Keyfactor Orchestrator\extensions` +### On the Orchestrator Agent Machine -1. Create a new folder called "HCV" (the name of the folder is not important) -1. Extract the contents of the release zip file into this folder. -1. Re-start the Orchestrator service. +1. Stop the Orchestrator service. + - The service will be called "KeyfactorOrchestrator-Default" by default. +2. Navigate to the "extensions" sub-folder of your Orchestrator installation directory + - example: `C:\Program Files\Keyfactor\Keyfactor Orchestrator\extensions` +3. Create a new folder called "HCV" (the name of the folder is not important) +4. Extract the contents of the release zip file into this folder. +5. Re-start the Orchestrator service. ### In the Keyfactor Platform -1. Add a new Certificate Store Type +#### Add a new Certificate Store Type - **Key-Value Secrets Engine** - Log into Keyfactor as Administrator or a user with permissions to add certificate store types. - Click on the gear icon in the top right and then navigate to the "Certificate Store Types" @@ -93,33 +120,39 @@ This integration was built on the .NET Core 3.1 target framework and are compati ![](images/store_type_add.png) - Set the following values in the "Basic" tab: - - **Name** - "Hashicorp Vault" (or another preferred name) - - **Short Name** - "HCV" + - **Name:** "Hashicorp Vault Key-Value" (or another preferred name) + - **Short Name:** "HCVKV" - **Supported Job Types** - "Inventory", "Add", "Remove", "Discovery" + +![](images/store-type-kv.PNG) + +- Set the following values on the "Advanced" tab: - **Supports Custom Alias** - "Optional" - **Private Key Handling** - "Optional" -![](images/store_type_1.png) +![](images/cert-store-type-advanced.png) - Click the "Custom Fields" tab to add the following custom fields: - **MountPoint** - type: *string* - **VaultServerUrl** - type: *string*, *required* - - **VaultToken** - type: *string*, *required* + - **VaultToken** - type: *secret*, *required* + - **SubfolderInventory** - type: *bool* (By default, this is set to false. Not a required field) + - **IncludeCertChain** - type: *bool* (If true, the available intermediate certificates will also be written to Vault during enrollment) ![](images/store_type_fields.png) - Click **Save** to save the new Store Type. -1. Add the Hashicorp Vault Certificate Store +#### Add the Hashicorp Vault Certificate Store - **Key-Value Secrets Engine** - Navigate to **Locations** > **Certificate Stores** from the main menu - Click **ADD** to open the new Certificate Store Dialog -![](images/cert_store_add_dialog.png) +![](images/cert_store_add_dialog.png) -In Keyfactor Command create a new Certificate Store Type similar to the one below: +In Keyfactor Command create a new Certificate Store that resembles the one below: -![](images/cert_store_fields.png) +![](images/cert_store_fields.png) - **Client Machine** - Enter the URL for the Vault host machine - **Store Path** - This is the path after mount point where the certs will be stored. @@ -128,11 +161,65 @@ In Keyfactor Command create a new Certificate Store Type similar to the one belo - If left blank, will default to "kv-v2". - **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault. - **Vault Server Url** - the full url and port of the Vault server instance +- **Subfolder Inventory** - Set to 'True' if it is a requirement to inventory secrets at the subfolder/component level. The default, 'False' will inventory secrets stored at the root of the "Store Path", but will not look at secrets in subfolders. **Note** that there is a limit on the number of certificates that can be in a certificate store. In certain environments enabling Subfolder Inventory may exceed this limit and cause inventory job failure. Inventory job results are currently submitted to the Command platform as a single HTTP POST. There is not a specific limit on the number of certificates in a store, rather the limit is based on the size of the actual certificates and the HTTP POST size limit configured on the Command web server. + +### For the Keyfactor and PKI plugins -## Testing +- Add a new Certificate Store Type + - Log into Keyfactor as Administrator or a user with permissions to add certificate store types. + - Click on the gear icon in the top right and then navigate to the "Certificate Store Types" + - Click "Add" and enter the following information on the first tab: + +![](images/store_type_add.png) + +- **Name:** "Hashicorp Vault PKI" (or another preferred name) +- **Short Name:** "HCVPKI" +- **Supported Job Types:** "Inventory" + +![](images/store_type_pki.PNG) + +- Set the following values on the "Advanced" tab: + - **Supports Custom Alias** - "Optional" + - **Private Key Handling** - "Optional" + +![](images/cert-store-type-advanced.png) + +- Click the "Custom Fields" tab to add the following custom fields: + - **MountPoint** - type: *string* + - **VaultServerUrl** - type: *string*, *required* + - **VaultToken** - type: *secret*, *required* + +![](images/store_type_fields.png) + +- Click **Save** to save the new Store Type. + +1. Add the Hashicorp Vault Certificate Store + +- Navigate to **Locations** > **Certificate Stores** from the main menu +- Click **ADD** to open the new Certificate Store Dialog + +In Keyfactor Command create a new Certificate Store similar to the one below: + +![](images/store_type_pki.png) + +- **Client Machine** - Enter the URL for the Vault host machine +- **Store Path** - "/" +- **Mount Point** - This is the mount point name for the instance of the PKI or Keyfactor secrets engine plugin. + - If using the PKI plugin, the default in Hashicorp is pki. If using the Keyfactor plugin, it should correspond to the mount point given when the plugin was enabled. + - It is possible to have multiple instances of the Keyfactor plugin running simultaneously, so be sure this corresponds to the one you would like to manage. + +- **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault. +- **Vault Server Url** - the full url and port of the Vault server instance + +At this point, the certificate store should be created and ready to peform inventory on your certificates stored via the Keyfactor or PKI secrets engine plugin for Hashicorp Vault. + +## Testing the Key-Value store ### PFX Enrollment into Vault +**Note** +Enrollment via the platform is only supported by the Key-Value store type + At this point you should be able to enroll a certificate and store it in Vault using the plugin. 1. Navigate to `Enrollment > PFX Enrollment` from the main menu. @@ -153,7 +240,7 @@ At this point you should be able to enroll a certificate and store it in Vault u - Make sure the vault is unsealed first -1. Type `vault kv list kv/cert-store` (where "kv/cert-store" is /) +1. Type `vault kv list kv/cert-store` (where "kv/cert-store" is `/`) - You should see the alias of the newly enrolled certificate @@ -168,6 +255,5 @@ At this point you should be able to enroll a certificate and store it in Vault u ## Notes / Future Enhancements -- Currently we only operate on a single version of the Key Value secret (no versioning capabilities through the Orchesterator Extension / Keyfactor). -- Creating a new certificate store is done implicitly by adding a **store path** value that doesn't currently exist. +- For the Key-Value stores we operate on a single version of the Key Value secret (no versioning capabilities through the Orchesterator Extension / Keyfactor). diff --git a/hashicorp-vault-orchestrator.sln b/hashicorp-vault-orchestrator.sln index a46e58b..af7d618 100644 --- a/hashicorp-vault-orchestrator.sln +++ b/hashicorp-vault-orchestrator.sln @@ -7,6 +7,7 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "hashicorp-vault-orchestrato EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{83623EBF-AC4C-4158-922D-959AEFC75453}" ProjectSection(SolutionItems) = preProject + CHANGELOG.md = CHANGELOG.md integration-manifest.json = integration-manifest.json LICENSE = LICENSE README.md = README.md diff --git a/hashicorp-vault-orchestrator/Constants.cs b/hashicorp-vault-orchestrator/Constants.cs index 9fe095e..da037a4 100644 --- a/hashicorp-vault-orchestrator/Constants.cs +++ b/hashicorp-vault-orchestrator/Constants.cs @@ -9,7 +9,8 @@ namespace Keyfactor.Extensions.Orchestrator.HashicorpVault { static class AzureKeyVaultConstants { - public const string STORE_TYPE_NAME = "HCV"; + public const string KEY_VALUE_STORE_TYPE = "HCVKV"; + public const string PKI_STORE_TYPE = "HCV"; //same for Keyfactor plugin store type } static class JobTypes diff --git a/hashicorp-vault-orchestrator/HcvClient.cs b/hashicorp-vault-orchestrator/HcvClient.cs deleted file mode 100644 index 5bf6e2c..0000000 --- a/hashicorp-vault-orchestrator/HcvClient.cs +++ /dev/null @@ -1,352 +0,0 @@ -// Copyright 2022 Keyfactor -// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. -// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 -// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions -// and limitations under the License. - -using System; -using System.Collections.Generic; -using System.IO; -using System.Linq; -using System.Text; -using System.Threading.Tasks; -using Keyfactor.Logging; -using Keyfactor.Orchestrators.Common.Enums; -using Keyfactor.Orchestrators.Extensions; -using Microsoft.Extensions.Logging; -using Org.BouncyCastle.Crypto; -using Org.BouncyCastle.OpenSsl; -using Org.BouncyCastle.Pkcs; -using VaultSharp; -using VaultSharp.V1.AuthMethods; -using VaultSharp.V1.AuthMethods.Token; -using VaultSharp.V1.Commons; - -namespace Keyfactor.Extensions.Orchestrator.HashicorpVault -{ - public class HcvClient - { - private IVaultClient _vaultClient { get; set; } - - protected IVaultClient VaultClient => _vaultClient; - - private ILogger logger = LogHandler.GetClassLogger(); - - private string _storePath { get; set; } - - private VaultClientSettings clientSettings { get; set; } - - private static readonly string privKeyStart = "-----BEGIN RSA PRIVATE KEY-----\n"; - private static readonly string privKeyEnd = "\n-----END RSA PRIVATE KEY-----"; - - public HcvClient(string vaultToken, string serverUrl) - { - // Initialize one of the several auth methods. - IAuthMethodInfo authMethod = new TokenAuthMethodInfo(vaultToken); - - // Initialize settings. You can also set proxies, custom delegates etc. here. - clientSettings = new VaultClientSettings(serverUrl, authMethod); - - _vaultClient = new VaultClient(clientSettings); - } - - public async Task GetCertificate(string key, string storePath, string mountPoint = null) - { - VaultClient.V1.Auth.ResetVaultToken(); - - Dictionary certData; - Secret res; - - storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; //add the slash back in. - try - { - var fullPath = storePath + "/" + key; - - try - { - if (mountPoint == null) - { - res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath)); - } - else - { - res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath, mountPoint: mountPoint)); - } - } - catch (Exception ex) - { - logger.LogWarning("Error getting certificate (deleted?)", ex); - return null; - } - - certData = (Dictionary)res.Data.Data; - } - catch (Exception ex) - { - logger.LogError("Error getting certificate from Vault", ex); - throw; - } - - try - { - string publicKey = certData["PUBLIC_KEY"]?.ToString() ?? null; - bool hasPrivateKey = certData["PRIVATE_KEY"] != null; - - var certs = new List() { publicKey }; - - var keys = certData.Keys.Where(k => k.StartsWith("PUBLIC_KEY_")).ToList(); - - keys.ForEach(k => certs.Add(certData[k].ToString())); - - return new CurrentInventoryItem() - { - Alias = key, - PrivateKeyEntry = hasPrivateKey, - ItemStatus = OrchestratorInventoryItemStatus.Unknown, - UseChainLevel = true, - Certificates = certs.ToArray() - }; - } - catch (Exception ex) - { - logger.LogError("Error parsing cert data", ex); - throw; - } - } - - public async Task> GetVaults(string storePath, string mountPoint = null) - { - VaultClient.V1.Auth.ResetVaultToken(); - - var vaults = new List(); - - try - { - if (mountPoint == null) - { - vaults = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storePath)).Data.Keys.ToList(); - } - else - { - vaults = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storePath, mountPoint)).Data.Keys.ToList(); - } - - } - catch (Exception ex) - { - logger.LogError(ex.Message); - } - - return vaults; - } - - public async Task PutCertificate(string certName, string contents, string pfxPassword, string storePath, string mountPoint = null) - { - VaultClient.V1.Auth.ResetVaultToken(); - - var certDict = new Dictionary(); - - var pfxBytes = Convert.FromBase64String(contents); - Pkcs12Store p; - using (var pfxBytesMemoryStream = new MemoryStream(pfxBytes)) - { - p = new Pkcs12Store(pfxBytesMemoryStream, - pfxPassword.ToCharArray()); - } - - // Extract private key - string alias; - string privateKeyString; - using (var memoryStream = new MemoryStream()) - { - using (TextWriter streamWriter = new StreamWriter(memoryStream)) - { - logger.LogTrace("Extracting Private Key..."); - var pemWriter = new PemWriter(streamWriter); - logger.LogTrace("Created pemWriter..."); - alias = p.Aliases.Cast().SingleOrDefault(a => p.IsKeyEntry(a)); - logger.LogTrace($"Alias = {alias}"); - var publicKey = p.GetCertificate(alias).Certificate.GetPublicKey(); - logger.LogTrace($"publicKey = {publicKey}"); - var KeyEntry = p.GetKey(alias); - logger.LogTrace($"KeyEntry = {KeyEntry}"); - if (KeyEntry == null) throw new Exception("Unable to retrieve private key"); - - var privateKey = KeyEntry.Key; - logger.LogTrace($"privateKey = {privateKey}"); - var keyPair = new AsymmetricCipherKeyPair(publicKey, privateKey); - - pemWriter.WriteObject(keyPair.Private); - streamWriter.Flush(); - privateKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim() - .Replace("\r", "").Replace("\0", ""); - logger.LogTrace($"Got Private Key String {privateKeyString}"); - memoryStream.Close(); - streamWriter.Close(); - logger.LogTrace("Finished Extracting Private Key..."); - } - } - var pubCertPem = Pemify(Convert.ToBase64String(p.GetCertificate(alias).Certificate.GetEncoded())); - - try - { - privateKeyString = privateKeyString.Replace(privKeyStart, "").Replace(privKeyEnd, ""); - certDict.Add("PRIVATE_KEY", privateKeyString); - certDict.Add("PUBLIC_KEY", pubCertPem); - certDict.Add("KEY_SECRET", pfxPassword); - } - catch (Exception ex) - { - logger.LogError("Error parsing certificate content", ex); - throw; - } - try - { - storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; //add the slash back in. - var fullPath = storePath + "/" + certName; - - if (mountPoint == null) - { - await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(fullPath, certDict); - } - else - { - await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(fullPath, certDict, mountPoint: mountPoint); - } - } - catch (Exception ex) - { - logger.LogError("Error writing cert to Vault", ex); - throw; - } - } - - public async Task DeleteCertificate(string certName, string storePath, string mountPoint = null) - { - VaultClient.V1.Auth.ResetVaultToken(); - - try - { - storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; //add the slash back in. - var fullPath = storePath + "/" + certName; - - if (mountPoint == null) - { - await VaultClient.V1.Secrets.KeyValue.V2.DeleteSecretAsync(fullPath); - } - else - { - await VaultClient.V1.Secrets.KeyValue.V2.DeleteSecretAsync(fullPath, mountPoint); - } - } - catch (Exception ex) - { - logger.LogError("Error removing cert from Vault", ex); - throw; - } - return true; - } - - public async Task> GetCertificates(string storePath, string mountPoint = null) - { - VaultClient.V1.Auth.ResetVaultToken(); - storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; //add the slash back in. - - var certs = new List(); - var certNames = new List(); - try - { - if (string.IsNullOrEmpty(mountPoint)) - { - certNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storePath)).Data.Keys.ToList(); - } - else - { - certNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storePath, mountPoint)).Data.Keys.ToList(); - } - - certNames.ForEach(k => - { - var cert = GetCertificate(k, storePath, mountPoint).Result; - if (cert != null) certs.Add(cert); - }); - } - catch (Exception ex) - { - logger.LogError(ex.Message); - } - - return certs; - } - - private static Func Pemify = ss => - ss.Length <= 64 ? ss : ss.Substring(0, 64) + "\n" + Pemify(ss.Substring(64)); - - //private string GetCertPem(string alias, string contents, string password, ref string privateKeyString) - //{ - // logger.MethodEntry(LogLevel.Debug); - // logger.LogTrace($"alias {alias} privateKeyString {privateKeyString}"); - // string certPem = null; - // try - // { - // if (!string.IsNullOrEmpty(password)) - // { - // logger.LogTrace($"Certificate and Key exist for {alias}"); - // var certData = Convert.FromBase64String(contents); - - // var ms = new MemoryStream(certData); - // Pkcs12Store store = new Pkcs12Store(ms, - // password.ToCharArray()); - - - // string storeAlias; - // TextWriter streamWriter; - // using (var memoryStream = new MemoryStream()) - // { - // streamWriter = new StreamWriter(memoryStream); - // var pemWriter = new PemWriter(streamWriter); - - // storeAlias = store.Aliases.Cast().SingleOrDefault(a => store.IsKeyEntry(a)); - // var publicKey = store.GetCertificate(storeAlias).Certificate.GetPublicKey(); - // var privateKey = store.GetKey(storeAlias).Key; - // var keyPair = new AsymmetricCipherKeyPair(publicKey, privateKey); - - // var pkStart = "-----BEGIN RSA PRIVATE KEY-----\n"; - // var pkEnd = "\n-----END RSA PRIVATE KEY-----"; - - - // pemWriter.WriteObject(keyPair.Private); - // streamWriter.Flush(); - // privateKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim() - // .Replace("\r", "") - // .Replace("\0", ""); - // privateKeyString = privateKeyString.Replace(pkStart, "").Replace(pkEnd, ""); - - // memoryStream.Close(); - // } - - // streamWriter.Close(); - - // // Extract server certificate - // certPem = Pemify( - // Convert.ToBase64String(store.GetCertificate(storeAlias).Certificate.GetEncoded())); - // } - // else - // { - // logger.LogTrace($"Certificate ONLY for {alias}"); - // certPem = Pemify(contents); - // } - // } - // catch (Exception ex) - // { - // logger.LogError($"Error Generating PEM: Error {LogHandler.FlattenException(ex)}"); - // } - - // logger.LogTrace($"PEM {certPem}"); - // logger.MethodEntry(LogLevel.Debug); - // return certPem; - //} - - } -} diff --git a/hashicorp-vault-orchestrator/HcvKeyValueClient.cs b/hashicorp-vault-orchestrator/HcvKeyValueClient.cs new file mode 100644 index 0000000..d7ee988 --- /dev/null +++ b/hashicorp-vault-orchestrator/HcvKeyValueClient.cs @@ -0,0 +1,384 @@ +// Copyright 2022 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Text; +using System.Threading.Tasks; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Common.Enums; +using Keyfactor.Orchestrators.Extensions; +using Microsoft.Extensions.Logging; +using Org.BouncyCastle.Crypto; +using Org.BouncyCastle.OpenSsl; +using Org.BouncyCastle.Pkcs; +using VaultSharp; +using VaultSharp.V1.AuthMethods; +using VaultSharp.V1.AuthMethods.Token; +using VaultSharp.V1.Commons; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault +{ + public class HcvKeyValueClient : IHashiClient + { + private IVaultClient _vaultClient { get; set; } + + protected IVaultClient VaultClient => _vaultClient; + + private ILogger logger = LogHandler.GetClassLogger(); + + private string _storePath { get; set; } + private string _mountPoint { get; set; } + private bool _subfolderInventory { get; set; } + + //private VaultClientSettings clientSettings { get; set; } + + public HcvKeyValueClient(string vaultToken, string serverUrl, string mountPoint, string storePath, bool SubfolderInventory = false) + { + // Initialize one of the several auth methods. + IAuthMethodInfo authMethod = new TokenAuthMethodInfo(vaultToken); + + // Initialize settings. You can also set proxies, custom delegates etc. here. + var clientSettings = new VaultClientSettings(serverUrl, authMethod); + _mountPoint = mountPoint; + _storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; + _vaultClient = new VaultClient(clientSettings); + _subfolderInventory = SubfolderInventory; + } + public async Task> ListComponentPathsAsync(string storagePath) + { + VaultClient.V1.Auth.ResetVaultToken(); + List componentPaths = new List { }; + try + { + Secret listInfo = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storagePath, _mountPoint)); + + foreach (var path in listInfo.Data.Keys) + { + if (!path.EndsWith("/")) + { + continue; + } + + string fullPath = $"{storagePath}{path}"; + componentPaths.Add(fullPath); + + List subPaths = await ListComponentPathsAsync(fullPath); + componentPaths.AddRange(subPaths); + } + } + catch (Exception ex) + { + logger.LogWarning($"Error while listing component paths: {ex}"); + } + return componentPaths; + } + public async Task GetCertificate(string key) + { + VaultClient.V1.Auth.ResetVaultToken(); + + Dictionary certData; + Secret res; + var fullPath = _storePath + key; + var relativePath = fullPath.Substring(_storePath.Length); + + try + { + try + { + if (_mountPoint == null) + { + res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath)); + } + else + { + res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath, mountPoint: _mountPoint)); + } + } + catch (Exception ex) + { + logger.LogError($"Error getting certificate {fullPath}", ex); + + return null; + } + + certData = (Dictionary)res.Data.Data; + } + catch (Exception ex) + { + logger.LogError("Error getting certificate from Vault", ex); + throw; + } + + try + { + string certificate = null; + + //Validates if the "certificate" and "private_key" keys exist in certData + if (certData.TryGetValue("certificate", out object publicKeyObj)) + { + certificate = publicKeyObj as string; + } + + var certs = new List() { certificate }; + + certData.TryGetValue("private_key", out object privateKeyObj); + + // if either field is missing, don't include it in inventory + + if (publicKeyObj == null || privateKeyObj == null) return null; + + //split the chain entries (if chain is included) + + var certFooter = "\n-----END CERTIFICATE-----"; + + certs = certificate.Split(new string[] { certFooter }, StringSplitOptions.RemoveEmptyEntries).ToList(); + + for (int i = 0; i 0) + { + return new CurrentInventoryItem() + { + Alias = key, + PrivateKeyEntry = privateKeyObj != null, + ItemStatus = OrchestratorInventoryItemStatus.Unknown, + UseChainLevel = certs.Count() > 1, + Certificates = certs + }; + } + else + { + return null; + } + } + catch (Exception ex) + { + logger.LogError("Error parsing cert data", ex); + throw; + } + } + + public async Task> GetVaults() + { + VaultClient.V1.Auth.ResetVaultToken(); + + var vaults = new List(); + + try + { + if (_mountPoint == null) + { + vaults = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(_storePath)).Data.Keys.ToList(); + } + else + { + vaults = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(_storePath, _mountPoint)).Data.Keys.ToList(); + } + + } + catch (Exception ex) + { + logger.LogError(ex.Message); + } + + return vaults; + } + + public async Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain) + { + VaultClient.V1.Auth.ResetVaultToken(); + + var certDict = new Dictionary(); + + var pfxBytes = Convert.FromBase64String(contents); + Pkcs12Store p; + using (var pfxBytesMemoryStream = new MemoryStream(pfxBytes)) + { + p = new Pkcs12Store(pfxBytesMemoryStream, + pfxPassword.ToCharArray()); + } + + // Extract private key + string alias; + string privateKeyString; + using (var memoryStream = new MemoryStream()) + { + using (TextWriter streamWriter = new StreamWriter(memoryStream)) + { + logger.LogTrace("Extracting Private Key..."); + var pemWriter = new PemWriter(streamWriter); + logger.LogTrace("Created pemWriter..."); + alias = p.Aliases.Cast().SingleOrDefault(a => p.IsKeyEntry(a)); + logger.LogTrace($"Alias = {alias}"); + var publicKey = p.GetCertificate(alias).Certificate.GetPublicKey(); + + logger.LogTrace($"publicKey = {publicKey}"); + var KeyEntry = p.GetKey(alias); + if (KeyEntry == null) throw new Exception("Unable to retrieve private key"); + + var privateKey = KeyEntry.Key; + var keyPair = new AsymmetricCipherKeyPair(publicKey, privateKey); + + pemWriter.WriteObject(keyPair.Private); + streamWriter.Flush(); + privateKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim() + .Replace("\r", "").Replace("\0", ""); + + logger.LogTrace($"Got Private Key String"); + memoryStream.Close(); + streamWriter.Close(); + logger.LogTrace("Finished Extracting Private Key..."); + } + } + + var pubCert = p.GetCertificate(alias).Certificate.GetEncoded(); + var pubCertPem = Pemify(Convert.ToBase64String(pubCert)); + + // add the certs in the chain + + var pemChain = new List(); + var chain = p.GetCertificateChain(alias).ToList(); + + chain.ForEach(c => + { + var cert = c.Certificate.GetEncoded(); + var encoded = Pemify(Convert.ToBase64String(cert)); + pemChain.Add(encoded); + }); + + try + { + certDict.Add("private_key", privateKeyString); + + // certDict.Add("revocation_time", 0); + + if (includeChain) + { + + certDict.Add("certificate", String.Join("\n", pemChain)); + } + else { + certDict.Add("certificate", pubCertPem); + } + } + catch (Exception ex) + { + logger.LogError("Error parsing certificate content", ex); + throw; + } + try + { + var fullPath = _storePath + certName; + + if (_mountPoint == null) + { + await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(fullPath, certDict); + } + else + { + await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(fullPath, certDict, mountPoint: _mountPoint); + } + } + catch (Exception ex) + { + logger.LogError("Error writing cert to Vault", ex); + throw; + } + } + + public async Task DeleteCertificate(string certName) + { + VaultClient.V1.Auth.ResetVaultToken(); + + try + { + var fullPath = _storePath + certName; + + if (_mountPoint == null) + { + await VaultClient.V1.Secrets.KeyValue.V2.DeleteSecretAsync(fullPath); + } + else + { + await VaultClient.V1.Secrets.KeyValue.V2.DeleteSecretAsync(fullPath, _mountPoint); + } + } + catch (Exception ex) + { + logger.LogError("Error removing cert from Vault", ex); + throw; + } + return true; + } + + public async Task> GetCertificates() + { + VaultClient.V1.Auth.ResetVaultToken(); + _storePath = _storePath.TrimStart('/'); + List subPaths = new List(); + //Grabs the list of subpaths to get certificates from, if SubFolder Inventory is turned on. + //Otherwise just define the single path _storePath + if (_subfolderInventory == true) + { + subPaths = (await ListComponentPathsAsync(_storePath)); + subPaths.Add(_storePath); + } + else + { + subPaths.Add(_storePath); + } + var certs = new List(); + var certNames = new List(); + logger.LogDebug($"SubInventoryEnabled: {_subfolderInventory}"); + foreach (var path in subPaths) + { + var relative_path = path.Substring(_storePath.Length); + try + { + + if (string.IsNullOrEmpty(_mountPoint)) + { + certNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(path)).Data.Keys.ToList(); + } + else + { + certNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(path, mountPoint: _mountPoint)).Data.Keys.ToList(); + } + + certNames.ForEach(k => + { + var cert = GetCertificate($"{relative_path}{k}").Result; + if (cert != null) certs.Add(cert); + }); + } + catch (Exception ex) + { + logger.LogError(ex.Message); + throw ex; + } + } + return certs; + } + private static Func Pemify = base64Cert => + { + string FormatBase64(string ss) => + ss.Length <= 64 ? ss : ss.Substring(0, 64) + "\n" + FormatBase64(ss.Substring(64)); + + string header = "-----BEGIN CERTIFICATE-----\n"; + string footer = "\n-----END CERTIFICATE-----"; + + return header + FormatBase64(base64Cert) + footer; + }; + } +} \ No newline at end of file diff --git a/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs b/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs new file mode 100644 index 0000000..33be1d8 --- /dev/null +++ b/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs @@ -0,0 +1,170 @@ +// Copyright 2022 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Net; +using System.Threading.Tasks; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Common.Enums; +using Keyfactor.Orchestrators.Extensions; +using Microsoft.Extensions.Logging; +using Newtonsoft.Json; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault +{ + public class HcvKeyfactorClient : IHashiClient + { + //private IVaultClient _vaultClient { get; set; } + + private ILogger logger = LogHandler.GetClassLogger(); + + private string _vaultUrl { get; set; } + + private string _vaultToken { get; set; } + + private string _mountPoint { get; set; } + + private string _storePath { get; set; } + + public HcvKeyfactorClient(string vaultToken, string serverUrl, string mountPoint, string storePath) + { + _vaultToken = vaultToken; + _mountPoint = mountPoint ?? "keyfactor"; + _storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; + _vaultUrl = $"{ serverUrl }/v1/{ _mountPoint.Replace("/", string.Empty) }"; + } + + public async Task GetCertificate(string key) + { + var fullPath = $"{ _vaultUrl }/cert/{ key }"; + + try + { + try + { + var req = WebRequest.Create(fullPath); + req.Headers.Add("X-Vault-Request", "true"); + req.Headers.Add("X-Vault-Token", _vaultToken); + req.Method = WebRequestMethods.Http.Get; + var res = await req.GetResponseAsync(); + CertResponse content = JsonConvert.DeserializeObject(new StreamReader(res.GetResponseStream()).ReadToEnd()); + + content.data.TryGetValue("certificate", out object cert); + content.data.TryGetValue("ca_chain", out object caChain); + content.data.TryGetValue("private_key", out object privateKey); + content.data.TryGetValue("revocation_time", out object revokeTime); + + List certList = new List() { cert as string }; + + // if the chain is available, include all certs + + if (!string.IsNullOrEmpty(caChain as string)) + { + string fullChain = caChain.ToString(); + certList = fullChain.Split(new string[] { "\n\n" }, StringSplitOptions.RemoveEmptyEntries).ToList(); + } + + // don't include them in inventory unless they haven't been revoked + + if (revokeTime == null || Equals(revokeTime.ToString(), "0")) + { + var inventoryItem = new CurrentInventoryItem() + { + Alias = key, + Certificates = certList, + ItemStatus = OrchestratorInventoryItemStatus.Unknown, + PrivateKeyEntry = !string.IsNullOrEmpty(privateKey as string), + UseChainLevel = !string.IsNullOrEmpty(caChain as string), + }; + return inventoryItem; + } + return null; + } + catch (Exception ex) + { + logger.LogWarning($"Error getting certificate \"{fullPath}\" from Vault", ex); + + return null; + } + } + catch (Exception ex) + { + logger.LogError($"Error getting certificate \"{fullPath}\" from Vault", ex); + throw; + } + } + + public async Task> GetCertificates() + { + var getKeysPath = $"{ _vaultUrl }/certs?list=true"; + var certs = new List(); + var certNames = new List(); + + try + { + var req = WebRequest.Create(getKeysPath); + req.Headers.Add("X-Vault-Request", "true"); + req.Headers.Add("X-Vault-Token", _vaultToken); + req.Method = WebRequestMethods.Http.Get; + var res = await req.GetResponseAsync(); + var content = JsonConvert.DeserializeObject(new StreamReader(res.GetResponseStream()).ReadToEnd()); + string[] certKeys; + + content.data.TryGetValue("keys", out certKeys); + + certKeys.ToList().ForEach(k => + { + var cert = GetCertificate(k).Result; + if (cert != null) certs.Add(cert); + }); + } + catch (Exception ex) + { + logger.LogError(ex.Message); + } + return certs; + } + + public Task> GetVaults() + { + throw new NotSupportedException(); + } + + public Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain) + { + throw new NotSupportedException(); + } + + public Task DeleteCertificate(string certName) + { + throw new NotSupportedException(); + } + + public class HashiResponse + { + public string request_id { get; set; } + public bool renewable { get; set; } + public int lease_duration { get; set; } + public string wrap_info { get; set; } + public string warnings { get; set; } + public string auth { get; set; } + } + + public class CertResponse : HashiResponse + { + public Dictionary data { get; set; } + } + + public class ListResponse : HashiResponse + { + public Dictionary data { get; set; } + } + } +} \ No newline at end of file diff --git a/hashicorp-vault-orchestrator/IHashiClient.cs b/hashicorp-vault-orchestrator/IHashiClient.cs new file mode 100644 index 0000000..043dd63 --- /dev/null +++ b/hashicorp-vault-orchestrator/IHashiClient.cs @@ -0,0 +1,22 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System.Collections.Generic; +using System.Threading.Tasks; +using Keyfactor.Orchestrators.Extensions; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault +{ + public interface IHashiClient + { + Task> GetCertificates(); + Task GetCertificate(string key); + Task> GetVaults(); + Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain); + Task DeleteCertificate(string certName); + } +} diff --git a/hashicorp-vault-orchestrator/Jobs/Discovery.cs b/hashicorp-vault-orchestrator/Jobs/Discovery.cs index 3dee838..fb40c41 100644 --- a/hashicorp-vault-orchestrator/Jobs/Discovery.cs +++ b/hashicorp-vault-orchestrator/Jobs/Discovery.cs @@ -27,19 +27,27 @@ public JobResult ProcessJob(DiscoveryJobConfiguration config, SubmitDiscoveryUpd try { - vaults = VaultClient.GetVaults(StorePath, MountPoint).Result.ToList(); - + vaults = VaultClient.GetVaults().Result.ToList(); } catch (Exception ex) { - logger.LogError(ex.Message); - - return new JobResult + var result = new JobResult { Result = OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = ex.Message + JobHistoryId = config.JobHistoryId }; + + if (ex.GetType() == typeof(NotSupportedException)) + { + logger.LogError("Attempt to perform discovery on unsupported Secrets Engine backend."); + + result.FailureMessage = $"{SecretsEngine} does not support Discovery jobs."; + } + else + { + result.FailureMessage = ex.Message; + } + return result; } submitDiscoveryUpdate.DynamicInvoke(vaults); diff --git a/hashicorp-vault-orchestrator/Jobs/Inventory.cs b/hashicorp-vault-orchestrator/Jobs/Inventory.cs index a331bd2..6232be6 100644 --- a/hashicorp-vault-orchestrator/Jobs/Inventory.cs +++ b/hashicorp-vault-orchestrator/Jobs/Inventory.cs @@ -26,7 +26,7 @@ public JobResult ProcessJob(InventoryJobConfiguration config, SubmitInventoryUpd IEnumerable certs = null; try { - certs = VaultClient.GetCertificates(StorePath, MountPoint).Result; + certs = VaultClient.GetCertificates().Result; } catch (Exception ex) { diff --git a/hashicorp-vault-orchestrator/Jobs/JobBase.cs b/hashicorp-vault-orchestrator/Jobs/JobBase.cs index dc0cc0b..83e8eb7 100644 --- a/hashicorp-vault-orchestrator/Jobs/JobBase.cs +++ b/hashicorp-vault-orchestrator/Jobs/JobBase.cs @@ -18,44 +18,76 @@ public abstract class JobBase public string VaultToken { get; set; } + public string SecretsEngine { get; set; } // "PKI", "Keyfactor", "Key Value" + public string VaultServerUrl { get; set; } + + public bool SubfolderInventory { get; set; } + + public bool IncludeCertChain { get; set; } public string MountPoint { get; set; } // the mount point of the KV secrets engine. defaults to KV - internal protected HcvClient VaultClient { get; set; } + public string RoleName { get; set; } + internal protected IHashiClient VaultClient { get; set; } + + const string KEY_VALUE_ENGINE = "KV"; + const string KEYFACTOR_ENGINE = "Keyfactor"; + const string PKI_ENGINE = "Hashicorp PKI"; public void InitializeStore(InventoryJobConfiguration config) { var props = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties); //var props = Jsonconfig.CertificateStoreDetails.Properties; - InitProps(props); StorePath = config.CertificateStoreDetails?.StorePath ?? null; + StorePath = StorePath.TrimStart('/'); + StorePath = StorePath.TrimEnd('/'); + StorePath = StorePath == null ? null : StorePath + "/"; //enforce single trailing slash for path + + InitProps(props, config.Capability); } public void InitializeStore(DiscoveryJobConfiguration config) { var props = config.JobProperties; - InitProps(props); + InitProps(props, config.Capability); } public void InitializeStore(ManagementJobConfiguration config) { var props = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties); - InitProps(props); StorePath = config.CertificateStoreDetails?.StorePath ?? null; - StorePath = StorePath.Replace("/", string.Empty); + StorePath = StorePath.TrimStart('/'); + StorePath = StorePath.TrimEnd('/'); + StorePath = StorePath == null ? null : StorePath + "/"; //enforce single trailing slash for path + InitProps(props, config.Capability); } - private void InitProps(dynamic props) { + private void InitProps(dynamic props, string capability) + { if (props == null) throw new System.Exception("Properties is null", props); VaultToken = props["VaultToken"]; VaultServerUrl = props["VaultServerUrl"]; + SecretsEngine = props["SecretsEngine"]; MountPoint = props["MountPoint"] ?? null; - VaultClient = new HcvClient(VaultToken, VaultServerUrl); + SubfolderInventory = props["SubfolderInventory"] ?? false; + IncludeCertChain = props["IncludeCertChain"] ?? false; + + var isPki = capability.Contains("HCVPKI"); + + if (!isPki) + { + VaultClient = new HcvKeyValueClient(VaultToken, VaultServerUrl, MountPoint, StorePath, SubfolderInventory); + } + else + { + VaultClient = new HcvKeyfactorClient(VaultToken, VaultServerUrl, MountPoint, StorePath); + } + } } -} +} \ No newline at end of file diff --git a/hashicorp-vault-orchestrator/Jobs/Management.cs b/hashicorp-vault-orchestrator/Jobs/Management.cs index 5dd0c12..0670d32 100644 --- a/hashicorp-vault-orchestrator/Jobs/Management.cs +++ b/hashicorp-vault-orchestrator/Jobs/Management.cs @@ -29,10 +29,6 @@ public JobResult ProcessJob(ManagementJobConfiguration config) switch (config.OperationType) { - //case CertStoreOperationType.Create: - // logger.LogDebug($"Begin Management > Create..."); - // complete = PerformCreateVault(config.JobHistoryId).Result; - // break; case CertStoreOperationType.Add: logger.LogDebug($"Begin Management > Add..."); complete = PerformAddition(config.JobCertificate.Alias, config.JobCertificate.PrivateKeyPassword, config.JobCertificate.Contents, config.JobHistoryId); @@ -46,32 +42,6 @@ public JobResult ProcessJob(ManagementJobConfiguration config) return complete; } - //protected async Task PerformCreateVault(long jobHistoryId) - //{ - // var jobResult = new JobResult() { JobHistoryId = jobHistoryId, Result = OrchestratorJobStatusJobResult.Failure }; - // bool createVaultResult; - // try - // { - // createVaultResult = await VaultClient.CreateStore(StorePath, MountPoint); - // } - // catch (Exception ex) - // { - // jobResult.FailureMessage = ex.Message; - // return jobResult; - // } - - // if (createVaultResult) - // { - // jobResult.Result = OrchestratorJobStatusJobResult.Success; - // } - // else - // { - // jobResult.FailureMessage = "The creation of the Azure Key Vault failed for an unknown reason. Check your job parameters and ensure permissions are correct."; - // } - - // return jobResult; - //} - protected virtual JobResult PerformAddition(string alias, string pfxPassword, string entryContents, long jobHistoryId) { var complete = new JobResult() { Result = OrchestratorJobStatusJobResult.Failure, JobHistoryId = jobHistoryId }; @@ -87,15 +57,23 @@ protected virtual JobResult PerformAddition(string alias, string pfxPassword, st try { // uploadCollection is either not null or an exception was thrown. - var cert = VaultClient.PutCertificate(alias, entryContents, pfxPassword, StorePath, MountPoint); + var cert = VaultClient.PutCertificate(alias, entryContents, pfxPassword, IncludeCertChain); complete.Result = OrchestratorJobStatusJobResult.Success; } catch (Exception ex) { - complete.FailureMessage = $"An error occured while adding {alias} to {ExtensionName}: " + ex.Message; - - if (ex.InnerException != null) - complete.FailureMessage += " - " + ex.InnerException.Message; + if (ex.GetType() == typeof(NotSupportedException)) + { + logger.LogError("Attempt to Add Certificate on unsupported Secrets Engine backend."); + complete.FailureMessage = $"{SecretsEngine} does not support adding certificates via the Orchestrator."; + } + else + { + complete.FailureMessage = $"An error occured while adding {alias} to {ExtensionName}: " + ex.Message; + + if (ex.InnerException != null) + complete.FailureMessage += " - " + ex.InnerException.Message; + } } } @@ -119,7 +97,7 @@ protected virtual JobResult PerformRemoval(string alias, long jobHistoryId) try { - var success = VaultClient.DeleteCertificate(alias, StorePath, MountPoint).Result; + var success = VaultClient.DeleteCertificate(alias).Result; if (!success) { @@ -133,8 +111,16 @@ protected virtual JobResult PerformRemoval(string alias, long jobHistoryId) catch (Exception ex) { - logger.LogError("Error deleting cert from Vault", ex); - complete.FailureMessage = $"An error occured while removing {alias} from {ExtensionName}: " + ex.Message; + if (ex.GetType() == typeof(NotSupportedException)) + { + logger.LogError("Attempt to Delete Certificate on unsupported Secrets Engine backend."); + complete.FailureMessage = $"{SecretsEngine} does not support removing certificates via the Orchestrator."; + } + else + { + logger.LogError("Error deleting cert from Vault", ex); + complete.FailureMessage = $"An error occured while removing {alias} from {ExtensionName}: " + ex.Message; + } } return complete; } diff --git a/hashicorp-vault-orchestrator/manifest.json b/hashicorp-vault-orchestrator/manifest.json index 837162f..559c5a3 100644 --- a/hashicorp-vault-orchestrator/manifest.json +++ b/hashicorp-vault-orchestrator/manifest.json @@ -1,17 +1,21 @@ { "extensions": { "Keyfactor.Orchestrators.Extensions.IOrchestratorJobExtension": { - "CertStores.HCV.Inventory": { + "CertStores.HCVKV.Inventory": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" }, - "CertStores.HCV.Discovery": { + "CertStores.HCVKV.Discovery": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Discovery" }, - "CertStores.HCV.Management": { + "CertStores.HCVKV.Management": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Management" + }, + "CertStores.HCVPKI.Inventory": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" } } } diff --git a/images/cert-store-kv.PNG b/images/cert-store-kv.PNG new file mode 100644 index 0000000..e88e2d9 Binary files /dev/null and b/images/cert-store-kv.PNG differ diff --git a/images/cert-store-pki.PNG b/images/cert-store-pki.PNG new file mode 100644 index 0000000..c6f7075 Binary files /dev/null and b/images/cert-store-pki.PNG differ diff --git a/images/cert-store-type-advanced.png b/images/cert-store-type-advanced.png new file mode 100644 index 0000000..da28516 Binary files /dev/null and b/images/cert-store-type-advanced.png differ diff --git a/images/cert_store_add_dialog.png b/images/cert_store_add_dialog.png index 3c0561f..3cd444f 100644 Binary files a/images/cert_store_add_dialog.png and b/images/cert_store_add_dialog.png differ diff --git a/images/store-type-kv.PNG b/images/store-type-kv.PNG new file mode 100644 index 0000000..e85c2ad Binary files /dev/null and b/images/store-type-kv.PNG differ diff --git a/images/store_type_fields.png b/images/store_type_fields.png index 5488fb3..c727456 100644 Binary files a/images/store_type_fields.png and b/images/store_type_fields.png differ diff --git a/images/store_type_pki.png b/images/store_type_pki.png new file mode 100644 index 0000000..ff45833 Binary files /dev/null and b/images/store_type_pki.png differ diff --git a/images/vault_cli_read.png b/images/vault_cli_read.png index 3a332db..c092110 100644 Binary files a/images/vault_cli_read.png and b/images/vault_cli_read.png differ diff --git a/integration-manifest.json b/integration-manifest.json index b335284..1ac05aa 100644 --- a/integration-manifest.json +++ b/integration-manifest.json @@ -4,9 +4,13 @@ "name": "Orchestrator Extension for Hashicorp Vault", "status": "production", "link_github": true, - "description": "The Hashicorp Vault Orchestrator extension allows you store certificates in Hashicorp Vault KeyValue secrets engine. ", + "update_catalog": true, + "support_level": "kf-supported", + "description": "The Hashicorp Vault Orchestrator extension allows you to manage certificates in Hashicorp Vault KeyValue secrets engine and perform inventory on certificates stored in the PKI or Keyfactor secrets engines.", "about": { "orchestrator": { + "UOFramework": "10.1", + "pam_support": false, "win": { "supportsCreateStore": true, "supportsDiscovery": true, @@ -24,7 +28,146 @@ "supportsReenrollment": false, "supportsInventory": true, "platformSupport": "Unused" - } + }, + "store_types": [ + { + "Name": "Hashicorp Vault Key-Value", + "ShortName": "HCVKV", + "Capability": "HCVKV", + "StoreType": 110, + "ImportType": 110, + "LocalStore": false, + "SupportedOperations": { + "Add": true, + "Create": false, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "Properties": [ + { + "StoreTypeId": 110, + "Name": "MountPoint", + "DisplayName": "Mount Point", + "Type": "String", + "DependsOn": null, + "DefaultValue": null, + "Required": false + }, + { + "StoreTypeId": 110, + "Name": "VaultServerUrl", + "DisplayName": "Vault Server URL", + "Type": "String", + "DependsOn": null, + "DefaultValue": null, + "Required": true + }, + { + "StoreTypeId": 110, + "Name": "VaultToken", + "DisplayName": "Vault Token", + "Type": "Secret", + "DependsOn": null, + "DefaultValue": null, + "Required": true + }, + { + "StoreTypeId": 110, + "Name": "SubfolderInventory", + "DisplayName": "Subfolder Inventory", + "Type": "Bool", + "DependsOn": null, + "DefaultValue": "true", + "Required": false + }, + { + "StoreTypeId": 110, + "Name": "IncludeCertChain", + "DisplayName": "Include CertChain", + "Type": "Bool", + "DependsOn": null, + "DefaultValue": "true", + "Required": false + } + ], + "EntryParameters": [], + "PasswordOptions": { + "EntrySupported": false, + "StoreRequired": false, + "Style": "Default" + }, + "PrivateKeyAllowed": "Optional", + "JobProperties": [], + "ServerRequired": false, + "PowerShell": false, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Optional", + "InventoryEndpoint": "/AnyInventory/Update", + "InventoryJobType": "ea93bb61-6b57-4c09-848a-3afa0a36507c", + "ManagementJobType": "e14c92fc-64f5-4902-ab62-bfc54af2cb2b", + "DiscoveryJobType": "d35efd13-89e9-402f-b4bf-b8b92b2a4e00" + }, + { + "Name": "Hashicorp Vault PKI", + "ShortName": "HCVPKI", + "Capability": "HCVPKI", + "StoreType": 112, + "ImportType": 112, + "LocalStore": false, + "SupportedOperations": { + "Add": false, + "Create": false, + "Discovery": false, + "Enrollment": false, + "Remove": false + }, + "Properties": [ + { + "StoreTypeId": 112, + "Name": "MountPoint", + "DisplayName": "Mount Point", + "Type": "String", + "DependsOn": null, + "DefaultValue": null, + "Required": false + }, + { + "StoreTypeId": 112, + "Name": "VaultToken", + "DisplayName": "Vault Token", + "Type": "Secret", + "DependsOn": null, + "DefaultValue": null, + "Required": true + }, + { + "StoreTypeId": 112, + "Name": "VaultServerUrl2", + "DisplayName": "Vault Server URL", + "Type": "String", + "DependsOn": null, + "DefaultValue": null, + "Required": false + } + ], + "EntryParameters": [], + "PasswordOptions": { + "EntrySupported": false, + "StoreRequired": false, + "Style": "Default" + }, + "StorePathValue": "/", + "PrivateKeyAllowed": "Optional", + "JobProperties": [], + "ServerRequired": false, + "PowerShell": false, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Optional", + "InventoryEndpoint": "/AnyInventory/Update", + "InventoryJobType": "9171e003-cc8d-4b41-b8d9-c02c4dadcb34" + } + ] } } } diff --git a/readme_source.md b/readme_source.md index 0087675..e69172b 100644 --- a/readme_source.md +++ b/readme_source.md @@ -1,52 +1,63 @@ -This integration for the Keyfactor Universal Orchestrator has been tested against Hashicorp Vault 1.10. It utilizes the *Key/Value* secrets engine to store certificates issues via Keyfactor Command. +This integration for the Keyfactor Universal Orchestrator has been tested against Hashicorp Vault 1.10. It utilizes the **Key/Value** secrets engine to store certificates issues via Keyfactor Command. ## Use Cases -The Hashicorp Vault Orchestrator Integration implements the following capabilities: +This integration supports 3 Hashicorp Secrets Engines; PKI, Key-Value store, and the Keyfactor Hashicorp Plugin (Keyfactor Secrets Engine). + +### The Key-Value secrets engine + +The Following operations are supported by this integration **only** for the Key-Value secrets engine. 1. Discovery - Discover all sub-paths containing certificate. 1. Inventory - Return all certificates stored in a path. 1. Management (Add) - Add a certificate to a defined certificate store. 1. Management (Remove) - Remove a certificate from a defined certificate store. +### The Hashicorp PKI and Keyfactor Plugin secrets engines + +Both the Hashicorp PKI and Keyfactor plugin are designed to allow managing certifications directly on the Hashicorp Vault instance. +This integration does support the following in order to view your certificates from the platform: + +1. Inventory - Return all certificates stored in a path. + +[View the repository on Github](https://github.com/Keyfactor/hashicorp-vault-secretsengine) for more information about the Hashicorp Vault Keyfactor Secrets Engine plugin. + ## Versioning The version number of a the Hashicorp Vault Orchestrator Extension can be verified by right clicking on the `Keyfactor.Extensions.Orchestrator.HCV.dll` file in the extensions installation folder, selecting Properties, and then clicking on the Details tab. ## Keyfactor Version Supported -This integration was built on the .NET Core 3.1 target framework and are compatible for use with the Keyfactor Universal Orchestrator. +This integration was built on the .NET Core 3.1 target framework and are compatible for use with the Keyfactor Universal Orchestrator and the latest version of the Keyfactor platform. ## Security Considerations 1. It is not necessary to use the Vault root token when creating a Certificate Store for HashicorpVault. We recommend creating a token with policies that reflect the minimum permissions necessary to perform the intended operations. -1. The certificates are stored in 3 fields in the Key Value store. -- `PUBLIC_KEY` - The certificate public key -- `PRIVATE_KEY` - The certificate private key -- `KEY_SECRET` - The certificate private key password - -## Extension Configuration - -### On the Orchestrator Agent Machine +1. For the Key-Value secrets engine, the certificates are stored as an entry with these fields. -1. Stop the Orchestrator service. +- `certificate` - The PEM formatted certificate and intermediate CA chain (if selected) +- `private_key` - The certificate private key -- The service will be called "KeyfactorOrchestrator-Default" by default. +**Note**: Key/Value secrets that do not include the keys `certificate` and `private_key` will be ignored during inventory scans. -1. Navigate to the "extensions" sub-folder of your Orchestrator installation directory +## Extension Configuration -- example: `C:\Program Files\Keyfactor\Keyfactor Orchestrator\extensions` +### On the Orchestrator Agent Machine -1. Create a new folder called "HCV" (the name of the folder is not important) -1. Extract the contents of the release zip file into this folder. -1. Re-start the Orchestrator service. +1. Stop the Orchestrator service. + - The service will be called "KeyfactorOrchestrator-Default" by default. +2. Navigate to the "extensions" sub-folder of your Orchestrator installation directory + - example: `C:\Program Files\Keyfactor\Keyfactor Orchestrator\extensions` +3. Create a new folder called "HCV" (the name of the folder is not important) +4. Extract the contents of the release zip file into this folder. +5. Re-start the Orchestrator service. ### In the Keyfactor Platform -1. Add a new Certificate Store Type +#### Add a new Certificate Store Type - **Key-Value Secrets Engine** - Log into Keyfactor as Administrator or a user with permissions to add certificate store types. - Click on the gear icon in the top right and then navigate to the "Certificate Store Types" @@ -55,33 +66,39 @@ This integration was built on the .NET Core 3.1 target framework and are compati ![](images/store_type_add.png) - Set the following values in the "Basic" tab: - - **Name** - "Hashicorp Vault" (or another preferred name) - - **Short Name** - "HCV" + - **Name:** "Hashicorp Vault Key-Value" (or another preferred name) + - **Short Name:** "HCVKV" - **Supported Job Types** - "Inventory", "Add", "Remove", "Discovery" + +![](images/store-type-kv.PNG) + +- Set the following values on the "Advanced" tab: - **Supports Custom Alias** - "Optional" - **Private Key Handling** - "Optional" -![](images/store_type_1.png) +![](images/cert-store-type-advanced.png) - Click the "Custom Fields" tab to add the following custom fields: - **MountPoint** - type: *string* - **VaultServerUrl** - type: *string*, *required* - - **VaultToken** - type: *string*, *required* + - **VaultToken** - type: *secret*, *required* + - **SubfolderInventory** - type: *bool* (By default, this is set to false. Not a required field) + - **IncludeCertChain** - type: *bool* (If true, the available intermediate certificates will also be written to Vault during enrollment) ![](images/store_type_fields.png) - Click **Save** to save the new Store Type. -1. Add the Hashicorp Vault Certificate Store +#### Add the Hashicorp Vault Certificate Store - **Key-Value Secrets Engine** - Navigate to **Locations** > **Certificate Stores** from the main menu - Click **ADD** to open the new Certificate Store Dialog -![](images/cert_store_add_dialog.png) +![](images/cert_store_add_dialog.png) -In Keyfactor Command create a new Certificate Store Type similar to the one below: +In Keyfactor Command create a new Certificate Store that resembles the one below: -![](images/cert_store_fields.png) +![](images/cert_store_fields.png) - **Client Machine** - Enter the URL for the Vault host machine - **Store Path** - This is the path after mount point where the certs will be stored. @@ -90,11 +107,65 @@ In Keyfactor Command create a new Certificate Store Type similar to the one belo - If left blank, will default to "kv-v2". - **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault. - **Vault Server Url** - the full url and port of the Vault server instance +- **Subfolder Inventory** - Set to 'True' if it is a requirement to inventory secrets at the subfolder/component level. The default, 'False' will inventory secrets stored at the root of the "Store Path", but will not look at secrets in subfolders. **Note** that there is a limit on the number of certificates that can be in a certificate store. In certain environments enabling Subfolder Inventory may exceed this limit and cause inventory job failure. Inventory job results are currently submitted to the Command platform as a single HTTP POST. There is not a specific limit on the number of certificates in a store, rather the limit is based on the size of the actual certificates and the HTTP POST size limit configured on the Command web server. + +### For the Keyfactor and PKI plugins + +- Add a new Certificate Store Type + - Log into Keyfactor as Administrator or a user with permissions to add certificate store types. + - Click on the gear icon in the top right and then navigate to the "Certificate Store Types" + - Click "Add" and enter the following information on the first tab: -## Testing +![](images/store_type_add.png) + +- **Name:** "Hashicorp Vault PKI" (or another preferred name) +- **Short Name:** "HCVPKI" +- **Supported Job Types:** "Inventory" + +![](images/store_type_pki.PNG) + +- Set the following values on the "Advanced" tab: + - **Supports Custom Alias** - "Optional" + - **Private Key Handling** - "Optional" + +![](images/cert-store-type-advanced.png) + +- Click the "Custom Fields" tab to add the following custom fields: + - **MountPoint** - type: *string* + - **VaultServerUrl** - type: *string*, *required* + - **VaultToken** - type: *secret*, *required* + +![](images/store_type_fields.png) + +- Click **Save** to save the new Store Type. + +1. Add the Hashicorp Vault Certificate Store + +- Navigate to **Locations** > **Certificate Stores** from the main menu +- Click **ADD** to open the new Certificate Store Dialog + +In Keyfactor Command create a new Certificate Store similar to the one below: + +![](images/store_type_pki.png) + +- **Client Machine** - Enter the URL for the Vault host machine +- **Store Path** - "/" +- **Mount Point** - This is the mount point name for the instance of the PKI or Keyfactor secrets engine plugin. + - If using the PKI plugin, the default in Hashicorp is pki. If using the Keyfactor plugin, it should correspond to the mount point given when the plugin was enabled. + - It is possible to have multiple instances of the Keyfactor plugin running simultaneously, so be sure this corresponds to the one you would like to manage. + +- **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault. +- **Vault Server Url** - the full url and port of the Vault server instance + +At this point, the certificate store should be created and ready to peform inventory on your certificates stored via the Keyfactor or PKI secrets engine plugin for Hashicorp Vault. + +## Testing the Key-Value store ### PFX Enrollment into Vault +**Note** +Enrollment via the platform is only supported by the Key-Value store type + At this point you should be able to enroll a certificate and store it in Vault using the plugin. 1. Navigate to `Enrollment > PFX Enrollment` from the main menu. @@ -115,7 +186,7 @@ At this point you should be able to enroll a certificate and store it in Vault u - Make sure the vault is unsealed first -1. Type `vault kv list kv/cert-store` (where "kv/cert-store" is /) +1. Type `vault kv list kv/cert-store` (where "kv/cert-store" is `/`) - You should see the alias of the newly enrolled certificate @@ -130,5 +201,4 @@ At this point you should be able to enroll a certificate and store it in Vault u ## Notes / Future Enhancements -- Currently we only operate on a single version of the Key Value secret (no versioning capabilities through the Orchesterator Extension / Keyfactor). -- Creating a new certificate store is done implicitly by adding a **store path** value that doesn't currently exist. +- For the Key-Value stores we operate on a single version of the Key Value secret (no versioning capabilities through the Orchesterator Extension / Keyfactor).