diff --git a/CHANGELOG.md b/CHANGELOG.md index a0dd80d..8864109 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,13 @@ +## 3.0.0 + +* Added support for JKS, PKCS12 and PFX file stores in the Hashicorp Vault Key-Value secrets engine. +* Added PAM support for server credentials. + +* **Breaking Changes** + * The server url and Vault Token have been moved to the server username and server password fields of server credentials, respectively. + * The HCVKV store type for PEM files has been renamed to HCVKVPEM + + ## 2.0.0 * Added support for storing certs in sub-paths diff --git a/README.md b/README.md index 37a8640..b2be5c6 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,6 @@ The Universal Orchestrator is part of the Keyfactor software distribution and is The Universal Orchestrator is the successor to the Windows Orchestrator. This Orchestrator Extension plugin only works with the Universal Orchestrator and does not work with the Windows Orchestrator. - - ## Support for Orchestrator Extension for Hashicorp Vault Orchestrator Extension for Hashicorp Vault is supported by Keyfactor for Keyfactor customers. If you have a support issue, please open a support ticket with your Keyfactor representative. @@ -23,7 +21,6 @@ Orchestrator Extension for Hashicorp Vault is supported by Keyfactor for Keyfact ###### To report a problem or suggest a new feature, use the **[Issues](../../issues)** tab. If you want to contribute actual bug fixes or proposed enhancements, use the **[Pull requests](../../pulls)** tab. - --- @@ -62,17 +59,39 @@ This integration supports 3 Hashicorp Secrets Engines; PKI, Key-Value store, and ### The Key-Value secrets engine -The Following operations are supported by this integration **only** for the Key-Value secrets engine. +For the Key-Value secrets engine, we have 4 store types that can be used. -1. Discovery - Discover all sub-paths containing certificate. -1. Inventory - Return all certificates stored in a path. +- *HCVKVJKS* - For JKS certificate files, treats each file as it's own store. +- *HCVKVPFX* - For PFX certificate files, treats each file as it's own store. +- *HCVKVP12* - For PKCS12 certificate files, treats each file as it's own store. +- *HCVKVPEM* - For PEM encoded certificates, treats each _path_ as it's own store. Each certificate exists in a sub-path from the store path. + +The following operations are supported by this integration for all of the Key-Value secrets engine types: + +1. Discovery - Discovery all file repositories for the type +1. Inventory - Inventory all certificates in the path 1. Management (Add) - Add a certificate to a defined certificate store. 1. Management (Remove) - Remove a certificate from a defined certificate store. +1. Create - Create a new, empty certificate store at the path defined in Store Path. + + +Excluding *HCVKVPEM*, the discovery process requires that: +1. The entry for the certificate contain the base64 encoded certificate file. +1. The name (key) for the entry ends with the suffix corresponding to the certificate store type: + 1. *HCVKVJKS* - `*_jks` + 1. *HCVKVPFX* - `*_pfx` + 1. *HCVKVP12* - `*_p12` + 1. *HCVKVPEM* - `certificate` +1. For all except *HCVKVPEM*, there be an entry named `passphrase` that contains the password for the store. +1. For *HCVKVPEM*, there be an entry named `private_key` containing the private key portion of the key-pair. + +**Note**: Key/Value secrets that do not include the expected keys will be ignored during inventory scans. ### The Hashicorp PKI and Keyfactor Plugin secrets engines -Both the Hashicorp PKI and Keyfactor plugin are designed to allow managing certifications directly on the Hashicorp Vault instance. -This integration does support the following in order to view your certificates from the platform: +Both the Hashicorp PKI and Keyfactor Secrets Engine plugins are designed to allow managing certifications directly on the Hashicorp Vault instance. +The store type for the PKI and/or the Keyfactor secrets engine is the same; `HCVPKI`. +This integration supports the following in order to view your certificates from the platform: 1. Inventory - Return all certificates stored in a path. @@ -90,13 +109,6 @@ This integration was built on the .NET Core 3.1 target framework and are compati 1. It is not necessary to use the Vault root token when creating a Certificate Store for HashicorpVault. We recommend creating a token with policies that reflect the minimum permissions necessary to perform the intended operations. -1. For the Key-Value secrets engine, the certificates are stored as an entry with these fields. - -- `certificate` - The PEM formatted certificate and intermediate CA chain (if selected) -- `private_key` - The certificate private key - -**Note**: Key/Value secrets that do not include the keys `certificate` and `private_key` will be ignored during inventory scans. - ## Extension Configuration ### On the Orchestrator Agent Machine @@ -111,7 +123,7 @@ This integration was built on the .NET Core 3.1 target framework and are compati ### In the Keyfactor Platform -#### Add a new Certificate Store Type - **Key-Value Secrets Engine** +#### Add a new Certificate Store Type - **Hashicorp Vault Key-Value PEM** - Log into Keyfactor as Administrator or a user with permissions to add certificate store types. - Click on the gear icon in the top right and then navigate to the "Certificate Store Types" @@ -120,9 +132,11 @@ This integration was built on the .NET Core 3.1 target framework and are compati ![](images/store_type_add.png) - Set the following values in the "Basic" tab: - - **Name:** "Hashicorp Vault Key-Value" (or another preferred name) - - **Short Name:** "HCVKV" + - **Name:** "Hashicorp Vault Key-Value PEM" (or another preferred name) + - **Short Name:** "HCVKVPEM" - **Supported Job Types** - "Inventory", "Add", "Remove", "Discovery" + - **NOTE** If you are setting up "`HCVKVJKS`, `HCVKVPFX`, or `HCVKVP12` the supported job types will be "Inventory, Discovery". + - **Needs Server** - should be checked (true). ![](images/store-type-kv.PNG) @@ -134,13 +148,14 @@ This integration was built on the .NET Core 3.1 target framework and are compati - Click the "Custom Fields" tab to add the following custom fields: - **MountPoint** - type: *string* - - **VaultServerUrl** - type: *string*, *required* - - **VaultToken** - type: *secret*, *required* - **SubfolderInventory** - type: *bool* (By default, this is set to false. Not a required field) - **IncludeCertChain** - type: *bool* (If true, the available intermediate certificates will also be written to Vault during enrollment) ![](images/store_type_fields.png) +**Note** +The 3 highlighted fields above will be added automatically by the platform, you will not need to include them when creating the certificate store type. + - Click **Save** to save the new Store Type. #### Add the Hashicorp Vault Certificate Store - **Key-Value Secrets Engine** @@ -154,16 +169,19 @@ In Keyfactor Command create a new Certificate Store that resembles the one below ![](images/cert_store_fields.png) -- **Client Machine** - Enter the URL for the Vault host machine +- **Client Machine** - Enter an identifier for the client machine. This could be the Orchestrator host name, or anything else useful. This value is not used by the extension. - **Store Path** - This is the path after mount point where the certs will be stored. - example: `kv-v2\kf-secrets\certname` would use the path "\kf-secrets" - **Mount Point** - This is the mount point name for the instance of the Key Value secrets engine. - If left blank, will default to "kv-v2". - If your organization utilizes Vault enterprise namespaces, you should include the namespace here. -- **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault. -- **Vault Server Url** - the full url and port of the Vault server instance - **Subfolder Inventory** - Set to 'True' if it is a requirement to inventory secrets at the subfolder/component level. The default, 'False' will inventory secrets stored at the root of the "Store Path", but will not look at secrets in subfolders. **Note** that there is a limit on the number of certificates that can be in a certificate store. In certain environments enabling Subfolder Inventory may exceed this limit and cause inventory job failure. Inventory job results are currently submitted to the Command platform as a single HTTP POST. There is not a specific limit on the number of certificates in a store, rather the limit is based on the size of the actual certificates and the HTTP POST size limit configured on the Command web server. +#### Set the server name and password + +- The server name should be the full URL to the instance of Vault that will be accessible by the orchestrator. (example: `http://127.0.0.1:8200`) +- The server password should be the Vault token that will be used for authenticating. + ### For the Keyfactor and PKI plugins - Add a new Certificate Store Type diff --git a/hashicorp-vault-orchestrator.sln b/hashicorp-vault-orchestrator.sln index af7d618..bc02e16 100644 --- a/hashicorp-vault-orchestrator.sln +++ b/hashicorp-vault-orchestrator.sln @@ -14,6 +14,11 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution readme_source.md = readme_source.md EndProjectSection EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "readme-src", "readme-src", "{3266961F-0B1D-4DB6-9A58-C0DA958EB832}" + ProjectSection(SolutionItems) = preProject + readme-pam-support.md = readme-pam-support.md + EndProjectSection +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -28,6 +33,9 @@ Global GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE EndGlobalSection + GlobalSection(NestedProjects) = preSolution + {3266961F-0B1D-4DB6-9A58-C0DA958EB832} = {83623EBF-AC4C-4158-922D-959AEFC75453} + EndGlobalSection GlobalSection(ExtensibilityGlobals) = postSolution SolutionGuid = {F1706189-FE1B-4EE6-9AE1-EF4A777B2F06} EndGlobalSection diff --git a/hashicorp-vault-orchestrator/CertUtility.cs b/hashicorp-vault-orchestrator/CertUtility.cs new file mode 100644 index 0000000..1aa77a3 --- /dev/null +++ b/hashicorp-vault-orchestrator/CertUtility.cs @@ -0,0 +1,112 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; +using System.Text; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Extensions; +using Microsoft.Extensions.Logging; +using Org.BouncyCastle.OpenSsl; +using Org.BouncyCastle.Pkcs; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault +{ + public static class CertUtility + { + public static List CurrentInventoryFromPkcs12(Pkcs12Store store) + { + var logger = LogHandler.GetClassLogger(); + + logger.MethodEntry(); + var certs = new List(); + + try + { + using (var memoryStream = new MemoryStream()) + { + using (TextWriter streamWriter = new StreamWriter(memoryStream)) + { + logger.LogTrace("Extracting Private Key..."); + var pemWriter = new PemWriter(streamWriter); + logger.LogTrace("Created pemWriter..."); + var aliases = store.Aliases.Cast().Where(a => store.IsKeyEntry(a)); + //logger.LogTrace($"Alias = {alias}"); + foreach (var alias in aliases) + { + var certInventoryItem = new CurrentInventoryItem { Alias = alias }; + + var entryCerts = new List(); + logger.LogTrace("extracting public key"); + var publicKey = store.GetCertificate(alias).Certificate.GetPublicKey(); + var privateKeyEntry = store.GetKey(alias); + if (privateKeyEntry != null) certInventoryItem.PrivateKeyEntry = true; + pemWriter.WriteObject(publicKey); + streamWriter.Flush(); + var publicKeyString = Encoding.ASCII.GetString(memoryStream.GetBuffer()).Trim() + .Replace("\r", "").Replace("\0", ""); + entryCerts.Add(publicKeyString); + + var pemChain = new List(); + + logger.LogTrace("getting chain certs"); + + var chain = store.GetCertificateChain(alias).ToList(); + + chain.ForEach(c => + { + var cert = c.Certificate.GetEncoded(); + var encoded = Pemify(Convert.ToBase64String(cert)); + pemChain.Add(encoded); + }); + + if (chain.Count() > 0) + { + certInventoryItem.UseChainLevel = true; + entryCerts.AddRange(pemChain); + } + certInventoryItem.Certificates = pemChain; + certs.Add(certInventoryItem); + } + memoryStream.Close(); + streamWriter.Close(); + } + logger.MethodExit(); + return certs; + } + } + catch (Exception ex) + { + logger.LogError("error extracting certs from pkcs12", ex); + throw; + } + } + + public static Func Pemify = base64Cert => + { + string FormatBase64(string ss) => + ss.Length <= 64 ? ss : ss.Substring(0, 64) + "\n" + FormatBase64(ss.Substring(64)); + + string header = "-----BEGIN CERTIFICATE-----\n"; + string footer = "\n-----END CERTIFICATE-----"; + + return header + FormatBase64(base64Cert) + footer; + }; + + public static string GenerateRandomString(int length) + { + using (Aes crypto = Aes.Create()) + { + crypto.GenerateKey(); + return Convert.ToBase64String(crypto.Key).Substring(0, length); + } + } + } +} diff --git a/hashicorp-vault-orchestrator/Constants.cs b/hashicorp-vault-orchestrator/Constants.cs index da037a4..686b8e5 100644 --- a/hashicorp-vault-orchestrator/Constants.cs +++ b/hashicorp-vault-orchestrator/Constants.cs @@ -1,4 +1,4 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, @@ -13,7 +13,7 @@ static class AzureKeyVaultConstants public const string PKI_STORE_TYPE = "HCV"; //same for Keyfactor plugin store type } - static class JobTypes + static class JobType { public const string CREATE = "Create"; public const string DISCOVERY = "Discovery"; @@ -21,4 +21,20 @@ static class JobTypes public const string MANAGEMENT = "Management"; public const string REENROLLMENT = "Enrollment"; } + + static class StoreType + { + public const string HCVKVPEM = "HCVKVPEM"; + public const string KCVKVJKS = "HCVKVJKS"; + public const string HCVKVPKCS12 = "HCVKVP12"; + public const string HCVKVPFX = "HCVKVPFX"; + public const string HCVPKI = "HCVPKI"; + } + + static class StoreFileExtensions { + public const string HCVKVJKS = "_jks"; + public const string HCVKVPKCS12 = "_p12"; + public const string HCVKVPFX = "_pfx"; + public const string HCVKVPEM = "certificate"; + } } diff --git a/hashicorp-vault-orchestrator/FileStores/IFileStore.cs b/hashicorp-vault-orchestrator/FileStores/IFileStore.cs new file mode 100644 index 0000000..135d6b3 --- /dev/null +++ b/hashicorp-vault-orchestrator/FileStores/IFileStore.cs @@ -0,0 +1,20 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System.Collections.Generic; +using Keyfactor.Orchestrators.Extensions; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault +{ + public interface IFileStore + { + string AddCertificate(string alias, string pfxPassword, string entryContents, bool includeChain, string certContent, string passphrase); + string RemoveCertificate(string alias, string passphrase, string storeFileContent); + byte[] CreateFileStore(string passphrase); + IEnumerable GetInventory(Dictionary certFields); + } +} diff --git a/hashicorp-vault-orchestrator/FileStores/JksFileStore.cs b/hashicorp-vault-orchestrator/FileStores/JksFileStore.cs new file mode 100644 index 0000000..d3da6bc --- /dev/null +++ b/hashicorp-vault-orchestrator/FileStores/JksFileStore.cs @@ -0,0 +1,313 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Text; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Extensions; +using Microsoft.Extensions.Logging; +using Org.BouncyCastle.Crypto; +using Org.BouncyCastle.Pkcs; +using Org.BouncyCastle.Security; +using Org.BouncyCastle.X509; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.FileStores +{ + public class JksFileStore : IFileStore + { + internal protected ILogger logger { get; set; } + + public JksFileStore() + { + logger = LogHandler.GetClassLogger(); + } + + public byte[] CreateFileStore(string password) + { + var newStore = new JksStore(); + + using (var outstream = new MemoryStream()) + { + logger.LogDebug("Created new JKS store, saving it to outStream"); + newStore.Save(outstream, password.ToCharArray()); + return outstream.ToArray(); + } + } + + public string AddCertificate(string alias, string pfxPassword, string entryContents, bool includeChain, string storeFileContent, string passphrase) + { + logger.MethodEntry(); + + logger.LogTrace("converting base64 encoded jks store to binary."); + var jksBytes = Convert.FromBase64String(storeFileContent); + + var newCertBytes = Convert.FromBase64String(entryContents); + + logger.LogTrace("adding the new certificate, and getting the new JKS store bytes."); + var newJksBytes = AddOrRemoveCert(alias, pfxPassword, newCertBytes, jksBytes, passphrase); + + return Convert.ToBase64String(newJksBytes); + } + + public string RemoveCertificate(string alias, string passphrase, string storeFileContent) + { + logger.MethodEntry(); + logger.LogTrace("converting base64 encoded jks store to binary."); + var jksBytes = Convert.FromBase64String(storeFileContent); + + logger.LogTrace("removing the certificate, and getting the new JKS store bytes."); + var newJksBytes = AddOrRemoveCert(alias, null, null, jksBytes, passphrase, true); + + return Convert.ToBase64String(newJksBytes); + } + + public IEnumerable GetInventory(Dictionary certFields) + { + logger.MethodEntry(); + // certFields should contain two entries. The certificate with the "_jks" suffix, and "passphrase" + + string password; + string base64EncodedJksStore; + var certs = new List(); + + try + { + var certKey = certFields.Keys.First(f => f.EndsWith(StoreFileExtensions.HCVKVJKS)); + + if (certKey == null) + { + throw new Exception($"No entry with extension '{StoreFileExtensions.HCVKVJKS}' found"); + } + else + { + base64EncodedJksStore = certFields[certKey].ToString(); + } + + if (certFields.TryGetValue("passphrase", out object filePasswordObj)) + { + password = filePasswordObj.ToString(); + } + else + { + throw new Exception($"No passphrase entry found for JKS store '{certKey}'."); + } + + logger.LogTrace("converting base64 encoded cert to binary."); + var jksBytes = Convert.FromBase64String(base64EncodedJksStore); + var pkcs12Store = JksToPkcs12Store(jksBytes, password); + certs = CertUtility.CurrentInventoryFromPkcs12(pkcs12Store); + logger.MethodExit(); + return certs; + + } + catch (Exception ex) + { + logger.LogError("Could not read JKS file", ex); + throw; + } + } + + private byte[] AddOrRemoveCert(string alias, string newCertPassword, byte[] newCertBytes, byte[] existingStore, string existingStorePassword, bool remove = false) + { + logger.MethodEntry(); + + // If existingStore is null, create a new store + var existingJksStore = new JksStore(); + var newJksStore = new JksStore(); + var createdNewStore = false; + + // If existingStore is not null, load it into jksStore + if (existingStore != null) + { + logger.LogDebug("Loading existing JKS store"); + using (var ms = new MemoryStream(existingStore)) + { + try + { + existingJksStore.Load(ms, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray()); + } + catch (Exception ex) + { + logger.LogError(ex, $"Error loading existing JKS store: {ex.Message}"); + } + } + if (existingJksStore.ContainsAlias(alias)) + { + // If alias exists, delete it from existingJksStore + logger.LogDebug("Alias '{Alias}' exists in existing JKS store, deleting it", alias); + existingJksStore.DeleteEntry(alias); + if (remove) + { + // If remove is true, save existingJksStore and return + logger.LogDebug("This is a removal operation, saving existing JKS store"); + using (var mms = new MemoryStream()) + { + existingJksStore.Save(mms, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray()); + logger.LogDebug("Returning existing JKS store"); + return mms.ToArray(); + } + } + } + else if (remove) + { + // If alias does not exist and remove is true, return existingStore + logger.LogDebug("Alias '{Alias}' does not exist in existing JKS store and this is a removal operation, returning existing JKS store as-is", alias); + using (var mms = new MemoryStream()) + { + existingJksStore.Save(mms, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray()); + return mms.ToArray(); + } + } + } + else + { + logger.LogDebug("Existing JKS store is null, creating new JKS store"); + createdNewStore = true; + } + + // Create new Pkcs12Store from newPkcs12Bytes + var storeBuilder = new Pkcs12StoreBuilder(); + var newCert = storeBuilder.Build(); + + try + { + logger.LogDebug("Loading new Pkcs12Store from newPkcs12Bytes"); + using (var pkcs12Ms = new MemoryStream(newCertBytes)) + { + newCert.Load(pkcs12Ms, string.IsNullOrEmpty(newCertPassword) ? Array.Empty() : newCertPassword.ToCharArray()); + } + } + catch (Exception) + { + logger.LogDebug("Loading new Pkcs12Store from newPkcs12Bytes failed, trying to load as X509Certificate"); + var certificateParser = new X509CertificateParser(); + var certificate = certificateParser.ReadCertificate(newCertBytes); + + logger.LogDebug("Creating new Pkcs12Store from certificate"); + // create new Pkcs12Store from certificate + storeBuilder = new Pkcs12StoreBuilder(); + newCert = storeBuilder.Build(); + logger.LogDebug("Setting certificate entry in new Pkcs12Store as alias '{Alias}'", alias); + newCert.SetCertificateEntry(alias, new X509CertificateEntry(certificate)); + } + + + // Iterate through newCert aliases. + logger.LogDebug("Iterating through new Pkcs12Store aliases"); + foreach (var al in newCert.Aliases) + { + logger.LogTrace("Alias: {Alias}", al); + if (newCert.IsKeyEntry(al)) + { + logger.LogDebug("Alias '{Alias}' is a key entry, getting key entry and certificate chain", al); + var keyEntry = newCert.GetKey(al); + logger.LogDebug("Getting certificate chain for alias '{Alias}'", al); + var certificateChain = newCert.GetCertificateChain(al); + + logger.LogDebug("Creating certificate list from certificate chain"); + var certificates = certificateChain.Select(certificateEntry => certificateEntry.Certificate).ToList(); + + if (createdNewStore) + { + // If createdNewStore is true, create a new store + logger.LogDebug("Created new JKS store, setting key entry for alias '{Alias}'", al); + newJksStore.SetKeyEntry(alias, + keyEntry.Key, + string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), + certificates.ToArray()); + } + else + { + // If createdNewStore is false, add to existingJksStore + // check if alias exists in existingJksStore + if (existingJksStore.ContainsAlias(alias)) + { + // If alias exists, delete it from existingJksStore + logger.LogDebug("Alias '{Alias}' exists in existing JKS store, deleting it", alias); + existingJksStore.DeleteEntry(alias); + } + + logger.LogDebug("Setting key entry for alias '{Alias}'", alias); + existingJksStore.SetKeyEntry(alias, + keyEntry.Key, + string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), + certificates.ToArray()); + } + } + else + { + logger.LogDebug("Setting certificate entry for existing JKS store, alias '{Alias}'", alias); + existingJksStore.SetCertificateEntry(alias, newCert.GetCertificate(alias).Certificate); + } + } + + using (var outStream = new MemoryStream()) + { + logger.LogDebug("Saving existing JKS store to outStream"); + existingJksStore.Save(outStream, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray()); + + logger.LogDebug("Returning updated JKS store as byte[]"); + return outStream.ToArray(); + } + } + + private Pkcs12Store JksToPkcs12Store(byte[] storeContents, string storePassword) + { + logger.LogTrace("Entering method to convert JKS store to PKCS12 to work with the contents."); + + Pkcs12StoreBuilder storeBuilder = new Pkcs12StoreBuilder(); + Pkcs12Store pkcs12Store = storeBuilder.Build(); + Pkcs12Store pkcs12StoreNew = storeBuilder.Build(); + + JksStore jksStore = new JksStore(); + + using (MemoryStream ms = new MemoryStream(storeContents)) + { + logger.LogTrace("loading the contents into a jks store"); + jksStore.Load(ms, string.IsNullOrEmpty(storePassword) ? new char[0] : storePassword.ToCharArray()); + } + + foreach (string alias in jksStore.Aliases) + { + if (jksStore.IsKeyEntry(alias)) + { + logger.LogTrace("extracting key pair"); + AsymmetricKeyParameter keyParam = jksStore.GetKey(alias, string.IsNullOrEmpty(storePassword) ? new char[0] : storePassword.ToCharArray()); + AsymmetricKeyEntry keyEntry = new AsymmetricKeyEntry(keyParam); + + logger.LogTrace("extracting certificate chain"); + + X509Certificate[] certificateChain = jksStore.GetCertificateChain(alias); + List certificateChainEntries = new List(); + foreach (X509Certificate certificate in certificateChain) + { + certificateChainEntries.Add(new X509CertificateEntry(certificate)); + } + logger.LogTrace("setting keys on the pkcs12 store."); + pkcs12Store.SetKeyEntry(alias, keyEntry, certificateChainEntries.ToArray()); + } + else + { + logger.LogTrace("setting certificates on the pkcs12 store"); + pkcs12Store.SetCertificateEntry(alias, new X509CertificateEntry(jksStore.GetCertificate(alias))); + } + } + + // Second Pkcs12Store necessary because of an obscure BC bug where creating a Pkcs12Store without .Load (code above using "Set" methods only) does not set all internal hashtables necessary to avoid an error later + // when processing store. + MemoryStream ms2 = new MemoryStream(); + pkcs12Store.Save(ms2, string.IsNullOrEmpty(storePassword) ? new char[0] : storePassword.ToCharArray(), new Org.BouncyCastle.Security.SecureRandom()); + ms2.Position = 0; + + pkcs12StoreNew.Load(ms2, string.IsNullOrEmpty(storePassword) ? new char[0] : storePassword.ToCharArray()); + return pkcs12StoreNew; + } + } +} diff --git a/hashicorp-vault-orchestrator/FileStores/PfxFileStore.cs b/hashicorp-vault-orchestrator/FileStores/PfxFileStore.cs new file mode 100644 index 0000000..ed60036 --- /dev/null +++ b/hashicorp-vault-orchestrator/FileStores/PfxFileStore.cs @@ -0,0 +1,246 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography.X509Certificates; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Extensions; +using Microsoft.Extensions.Logging; +using Org.BouncyCastle.Pkcs; +using Org.BouncyCastle.Security; +using Org.BouncyCastle.X509; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.FileStores +{ + public class PfxFileStore : IFileStore + { + internal protected ILogger logger { get; set; } + + public PfxFileStore() + { + logger = LogHandler.GetClassLogger(); + } + + public byte[] CreateFileStore(string password) + { + Pkcs12Store newStore = null; + using (var outstream = new MemoryStream()) + { + logger.LogDebug("Created new PFX store, saving it to outStream"); + newStore.Save(outstream, password.ToCharArray(), new SecureRandom()); + return outstream.ToArray(); + } + } + + public string AddCertificate(string alias, string pfxPassword, string entryContents, bool includeChain, string storeFileContent, string passphrase) + { + logger.MethodEntry(); + + logger.LogTrace("converting base64 encoded PFX store to binary."); + var pfxBytes = Convert.FromBase64String(storeFileContent); + + + var newCertBytes = Convert.FromBase64String(entryContents); + + logger.LogTrace("adding the new certificate, and getting the new PFX store bytes."); + var newJksBytes = AddOrRemoveCert(alias, pfxPassword, newCertBytes, pfxBytes, passphrase); + + return Convert.ToBase64String(newJksBytes); + } + public string RemoveCertificate(string alias, string passphrase, string storeFileContent) + { + logger.MethodEntry(); + logger.LogTrace("converting base64 encoded PFX store to binary."); + var pfxStoreBytes = Convert.FromBase64String(storeFileContent); + + logger.LogTrace("removing the certificate, and getting the new PFX store bytes."); + var newPfxStoreBytes = AddOrRemoveCert(alias, null, null, pfxStoreBytes, passphrase, true); + + return Convert.ToBase64String(newPfxStoreBytes); + } + + public IEnumerable GetInventory(Dictionary certFields) + { + logger.MethodEntry(); + // certFields should contain two entries. The certificate with the "_pfx" suffix, and "passphrase" + string password; + string base64encodedCert; + var certs = new List(); + + + var certKey = certFields.Keys.First(f => f.Contains(StoreFileExtensions.HCVKVPFX)); + + if (certKey == null) + { + throw new Exception($"No entry with extension '{StoreFileExtensions.HCVKVPFX}' found"); + } + else + { + base64encodedCert = certFields[certKey].ToString(); + } + + if (certFields.TryGetValue("passphrase", out object filePasswordObj)) + { + password = filePasswordObj.ToString(); + } + else + { + throw new Exception($"No password entry found for PFX store '{certKey}'."); + } + logger.LogTrace("converting base64 encoded cert to binary format."); + + var pfxBytes = Convert.FromBase64String(base64encodedCert); + Pkcs12Store p; + using (var pfxBytesMemoryStream = new MemoryStream(pfxBytes)) + { + logger.LogTrace("creating pkcs12 store for working with the certificate."); + Pkcs12StoreBuilder storeBuilder = new Pkcs12StoreBuilder(); + p = storeBuilder.Build(); + p.Load(pfxBytesMemoryStream, password.ToCharArray()); + } + + certs = CertUtility.CurrentInventoryFromPkcs12(p); + logger.MethodExit(); + return certs; + } + + private byte[] AddOrRemoveCert(string alias, string newCertPassword, byte[] newCertBytes, byte[] existingStore, string existingStorePassword, bool remove = false) + { + logger.MethodEntry(); + + Pkcs12Store existingPfxStore = null; + + if (existingStore == null) + { + throw new DirectoryNotFoundException("An existing PFX certificate store was not found."); + } + + logger.LogDebug("Loading existing PFX store from binary data."); + + try + { + using (var pfxBytesMemoryStream = new MemoryStream(existingStore)) + { + logger.LogTrace("creating pkcs12 store for working with the certificate."); + Pkcs12StoreBuilder sb = new Pkcs12StoreBuilder(); + existingPfxStore = sb.Build(); + existingPfxStore.Load(pfxBytesMemoryStream, existingStorePassword.ToCharArray()); + } + } + catch (Exception ex) + { + logger.LogError(ex, $"Error loading existing PFX store: {ex.Message}"); + } + + if (existingPfxStore.ContainsAlias(alias)) + { + // If alias exists, delete it from existingJksStore + logger.LogDebug($"Alias '{alias}' exists in existing PFX store, deleting it"); + existingPfxStore.DeleteEntry(alias); + if (remove) + { + // If remove is true, save existingJksStore and return + logger.LogDebug("This is a removal operation, saving existing PFX store"); + using (var mms = new MemoryStream()) + { + existingPfxStore.Save(mms, + string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), new SecureRandom()); + logger.LogDebug("Returning existing PFX store"); + return mms.ToArray(); + } + } + } + else if (remove) + { + // If alias does not exist and remove is true, return existingStore + logger.LogDebug($"Alias '{alias}' does not exist in existing PFX store and this is a removal operation, returning existing PFX store as-is"); + using (var mms = new MemoryStream()) + { + existingPfxStore.Save(mms, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), new SecureRandom()); + return mms.ToArray(); + } + } + + // adding the new certificate + + // Create new Pkcs12Store from newPkcs12Bytes + var storeBuilder = new Pkcs12StoreBuilder(); + var newCert = storeBuilder.Build(); + + try + { + logger.LogDebug("Loading new certificate as pfx/pkcs12 from newPkcs12Bytes"); + using (var pkcs12Ms = new MemoryStream(newCertBytes)) + { + newCert.Load(pkcs12Ms, string.IsNullOrEmpty(newCertPassword) ? Array.Empty() : newCertPassword.ToCharArray()); + } + } + catch (Exception) + { + logger.LogDebug("Loading new Pkcs12Store from newPkcs12Bytes failed, trying to load as X509Certificate"); + var certificateParser = new X509CertificateParser(); + var certificate = certificateParser.ReadCertificate(newCertBytes); + + logger.LogDebug("Creating new Pkcs12Store from certificate"); + // create new Pkcs12Store from certificate + storeBuilder = new Pkcs12StoreBuilder(); + newCert = storeBuilder.Build(); + logger.LogDebug($"Setting certificate entry in new Pkcs12Store as alias '{alias}'"); + newCert.SetCertificateEntry(alias, new X509CertificateEntry(certificate)); + } + + + // Iterate through newCert aliases. + logger.LogDebug("Iterating through new Pkcs12Store aliases"); + foreach (var al in newCert.Aliases) + { + logger.LogTrace($"Alias: {al}"); + if (newCert.IsKeyEntry(al)) + { + logger.LogDebug($"Alias '{al}' is a key entry, getting key entry and certificate chain"); + var keyEntry = newCert.GetKey(al); + logger.LogDebug($"Getting certificate chain for alias '{al}'"); + var certificateChain = newCert.GetCertificateChain(al); + + logger.LogDebug("Creating certificate list from certificate chain"); + var certificates = certificateChain.ToList(); + + // If createdNewStore is false, add to existingJksStore + // check if alias exists in existingJksStore + if (existingPfxStore.ContainsAlias(alias)) + { + // If alias exists, delete it from existingJksStore + logger.LogDebug($"Alias '{alias}' exists in existing PFX store, deleting it"); + existingPfxStore.DeleteEntry(alias); + } + + logger.LogDebug($"Setting key entry for alias '{alias}'"); + existingPfxStore.SetKeyEntry(alias, + keyEntry, + certificates.ToArray()); + } + else + { + logger.LogDebug($"Setting certificate with alias '{alias}' for existing PFX store"); + existingPfxStore.SetCertificateEntry(alias, newCert.GetCertificate(alias)); + } + } + + using (var outStream = new MemoryStream()) + { + logger.LogDebug("Saving existing PFX store to outStream"); + existingPfxStore.Save(outStream, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), new SecureRandom()); + + logger.LogDebug("Returning updated PFX store as byte[]"); + return outStream.ToArray(); + } + } + } +} diff --git a/hashicorp-vault-orchestrator/FileStores/Pkcs12FileStore.cs b/hashicorp-vault-orchestrator/FileStores/Pkcs12FileStore.cs new file mode 100644 index 0000000..5c90705 --- /dev/null +++ b/hashicorp-vault-orchestrator/FileStores/Pkcs12FileStore.cs @@ -0,0 +1,258 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using Keyfactor.Logging; +using Keyfactor.Orchestrators.Extensions; +using Microsoft.Extensions.Logging; +using Org.BouncyCastle.Pkcs; +using Org.BouncyCastle.Security; +using Org.BouncyCastle.X509; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.FileStores +{ + public class Pkcs12FileStore : IFileStore + { + internal protected ILogger logger { get; set; } + + public Pkcs12FileStore() + { + logger = LogHandler.GetClassLogger(); + } + + public byte[] CreateFileStore(string password) + { + Pkcs12Store newStore = null; + using (var outstream = new MemoryStream()) + { + logger.LogDebug("Created new PKCS12 store, saving it to outStream"); + newStore.Save(outstream, password.ToCharArray(), new SecureRandom()); + return outstream.ToArray(); + } + } + + public string AddCertificate(string alias, string pfxPassword, string entryContents, bool includeChain, string storeFileContent, string passphrase) + { + logger.MethodEntry(); + + logger.LogTrace("converting base64 encoded PKCS12 store to binary."); + var pkcs12bytes = Convert.FromBase64String(storeFileContent); + + + var newCertBytes = Convert.FromBase64String(entryContents); + + logger.LogTrace("adding the new certificate, and getting the new PKCS12 store bytes."); + var newPkcs12Bytes = AddOrRemoveCert(alias, pfxPassword, newCertBytes, pkcs12bytes, passphrase); + + return Convert.ToBase64String(newPkcs12Bytes); + } + + public string RemoveCertificate(string alias, string passphrase, string storeFileContent) + { + logger.MethodEntry(); + logger.LogTrace("converting base64 encoded PKCS12 store to binary."); + var pkcs12StoreBytes = Convert.FromBase64String(storeFileContent); + + logger.LogTrace("removing the certificate, and getting the new PKCS12 store bytes."); + var newPkcs12StoreBytes = AddOrRemoveCert(alias, null, null, pkcs12StoreBytes, passphrase, true); + + return Convert.ToBase64String(newPkcs12StoreBytes); + } + + public IEnumerable GetInventory(Dictionary certFields) + { + logger.MethodEntry(); + // certFields should contain two entries. The certificate with the "_pfx" suffix, and "passphrase" + + string password; + string base64encodedCert; + var certs = new List(); + + try + { + var certKey = certFields.Keys.First(f => f.Contains(StoreFileExtensions.HCVKVPKCS12)); + + if (certKey == null) + { + throw new Exception($"No entry with extension '{StoreFileExtensions.HCVKVPKCS12}' found"); + } + else + { + base64encodedCert = certFields[certKey].ToString(); + } + + if (certFields.TryGetValue("passphrase", out object filePasswordObj)) + { + password = filePasswordObj.ToString(); + } + else + { + throw new Exception($"No password entry found for PKCS12 store '{certKey}'."); + } + + // certFields should contain two entries. The certificate with the "p12-contents" suffix, and "password" + logger.LogTrace("converting base64 encoded cert to binary."); + var bytes = Convert.FromBase64String(base64encodedCert); + + Pkcs12Store pkcs12Store; + + using (var stream = new MemoryStream(bytes)) + { + logger.LogTrace("creating pkcs12 store for working with the certificate."); + Pkcs12StoreBuilder storeBuilder = new Pkcs12StoreBuilder(); + pkcs12Store = storeBuilder.Build(); + pkcs12Store.Load(stream, password.ToCharArray()); + } + certs = CertUtility.CurrentInventoryFromPkcs12(pkcs12Store); + logger.MethodExit(); + return certs; + } + catch (Exception ex) + { + logger.LogError("Unable to read PKCS12 file.", ex); + throw; + } + } + + private byte[] AddOrRemoveCert(string alias, string newCertPassword, byte[] newCertBytes, byte[] existingStore, string existingStorePassword, bool remove = false) + { + logger.MethodEntry(); + + Pkcs12Store existingPkcs12Store = null; + + // If existingStore is not null, load it into existingPkcs12Store + + if (existingStore == null) + { + throw new DirectoryNotFoundException("An existing PKCS12 certificate store was not found."); + } + + logger.LogDebug("Loading existing PKCS12 store from binary data."); + + try + { + using (var pfxBytesMemoryStream = new MemoryStream(existingStore)) + { + logger.LogTrace("creating pkcs12 store for working with the certificate."); + Pkcs12StoreBuilder sb = new Pkcs12StoreBuilder(); + existingPkcs12Store = sb.Build(); + existingPkcs12Store.Load(pfxBytesMemoryStream, existingStorePassword.ToCharArray()); + } + } + catch (Exception ex) + { + logger.LogError(ex, $"Error loading existing PKCS12 store: {ex.Message}"); + } + + if (existingPkcs12Store.ContainsAlias(alias)) + { + // If alias exists, delete it from existingJksStore + logger.LogDebug($"Alias '{alias}' exists in existing PKCS12 store, deleting it"); + existingPkcs12Store.DeleteEntry(alias); + if (remove) + { + // If remove is true, save existingJksStore and return + logger.LogDebug("This is a removal operation, saving existing PKCS12 store"); + using (var mms = new MemoryStream()) + { + existingPkcs12Store.Save(mms, + string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), new SecureRandom()); + logger.LogDebug("Returning existing PKCS12 store"); + return mms.ToArray(); + } + } + } + else if (remove) + { + // If alias does not exist and remove is true, return existingStore + logger.LogDebug($"Alias '{alias}' does not exist in existing PKCS12 store and this is a removal operation, returning existing PKCS12 store as-is"); + using (var mms = new MemoryStream()) + { + existingPkcs12Store.Save(mms, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), new SecureRandom()); + return mms.ToArray(); + } + } + + // adding the new certificate + + // Create new Pkcs12Store from newPkcs12Bytes + var storeBuilder = new Pkcs12StoreBuilder(); + var newCert = storeBuilder.Build(); + + try + { + logger.LogDebug("Loading new certificate as pfx/pkcs12 from newPkcs12Bytes"); + using (var pkcs12Ms = new MemoryStream(newCertBytes)) + { + newCert.Load(pkcs12Ms, string.IsNullOrEmpty(newCertPassword) ? Array.Empty() : newCertPassword.ToCharArray()); + } + } + catch (Exception) + { + logger.LogDebug("Loading new Pkcs12Store from newPkcs12Bytes failed, trying to load as X509Certificate"); + var certificateParser = new X509CertificateParser(); + var certificate = certificateParser.ReadCertificate(newCertBytes); + + logger.LogDebug("Creating new Pkcs12Store from certificate"); + // create new Pkcs12Store from certificate + storeBuilder = new Pkcs12StoreBuilder(); + newCert = storeBuilder.Build(); + logger.LogDebug($"Setting certificate entry in new Pkcs12Store as alias '{alias}'"); + newCert.SetCertificateEntry(alias, new X509CertificateEntry(certificate)); + } + + + // Iterate through newCert aliases. + logger.LogDebug("Iterating through new Pkcs12Store aliases"); + foreach (var al in newCert.Aliases) + { + logger.LogTrace($"Alias: {al}"); + if (newCert.IsKeyEntry(al)) + { + logger.LogDebug($"Alias '{al}' is a key entry, getting key entry and certificate chain"); + var keyEntry = newCert.GetKey(al); + logger.LogDebug($"Getting certificate chain for alias '{al}'"); + var certificateChain = newCert.GetCertificateChain(al); + + logger.LogDebug("Creating certificate list from certificate chain"); + var certificates = certificateChain.ToList(); + + // If createdNewStore is false, add to existingJksStore + // check if alias exists in existingJksStore + if (existingPkcs12Store.ContainsAlias(alias)) + { + // If alias exists, delete it from existingJksStore + logger.LogDebug($"Alias '{alias}' exists in existing PKCS12 store, deleting it"); + existingPkcs12Store.DeleteEntry(alias); + } + + logger.LogDebug($"Setting key entry for alias '{alias}'"); + existingPkcs12Store.SetKeyEntry(alias, + keyEntry, + certificates.ToArray()); + } + else + { + logger.LogDebug($"Setting certificate with alias '{alias}' for existing PKCS12 store"); + existingPkcs12Store.SetCertificateEntry(alias, newCert.GetCertificate(alias)); + } + } + + using (var outStream = new MemoryStream()) + { + logger.LogDebug("Saving existing PKCS12 store to outStream"); + existingPkcs12Store.Save(outStream, string.IsNullOrEmpty(existingStorePassword) ? Array.Empty() : existingStorePassword.ToCharArray(), new SecureRandom()); + + logger.LogDebug("Returning updated PKCS12 store as byte[]"); + return outStream.ToArray(); + } + } + } +} diff --git a/hashicorp-vault-orchestrator/HcvKeyValueClient.cs b/hashicorp-vault-orchestrator/HcvKeyValueClient.cs index d7ee988..dba7990 100644 --- a/hashicorp-vault-orchestrator/HcvKeyValueClient.cs +++ b/hashicorp-vault-orchestrator/HcvKeyValueClient.cs @@ -1,4 +1,4 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, @@ -9,8 +9,11 @@ using System.Collections.Generic; using System.IO; using System.Linq; +using System.Net.Http; using System.Text; +using System.Text.Json; using System.Threading.Tasks; +using Keyfactor.Extensions.Orchestrator.HashicorpVault.FileStores; using Keyfactor.Logging; using Keyfactor.Orchestrators.Common.Enums; using Keyfactor.Orchestrators.Extensions; @@ -22,6 +25,7 @@ using VaultSharp.V1.AuthMethods; using VaultSharp.V1.AuthMethods.Token; using VaultSharp.V1.Commons; +using VaultSharp.V1.SecretsEngines.KeyValue.V2; namespace Keyfactor.Extensions.Orchestrator.HashicorpVault { @@ -36,10 +40,9 @@ public class HcvKeyValueClient : IHashiClient private string _storePath { get; set; } private string _mountPoint { get; set; } private bool _subfolderInventory { get; set; } + private string _storeType { get; set; } - //private VaultClientSettings clientSettings { get; set; } - - public HcvKeyValueClient(string vaultToken, string serverUrl, string mountPoint, string storePath, bool SubfolderInventory = false) + public HcvKeyValueClient(string vaultToken, string serverUrl, string mountPoint, string storePath, string storeType, bool SubfolderInventory = false) { // Initialize one of the several auth methods. IAuthMethodInfo authMethod = new TokenAuthMethodInfo(vaultToken); @@ -47,65 +50,141 @@ public HcvKeyValueClient(string vaultToken, string serverUrl, string mountPoint, // Initialize settings. You can also set proxies, custom delegates etc. here. var clientSettings = new VaultClientSettings(serverUrl, authMethod); _mountPoint = mountPoint; - _storePath = !string.IsNullOrEmpty(storePath) ? "/" + storePath : storePath; + _storePath = (!string.IsNullOrEmpty(storePath) && !storePath.StartsWith("/")) ? "/" + storePath.Trim() : storePath?.Trim(); _vaultClient = new VaultClient(clientSettings); _subfolderInventory = SubfolderInventory; + _storeType = storeType?.Split('.')[1]; } - public async Task> ListComponentPathsAsync(string storagePath) + + public async Task CreateCertStore() { - VaultClient.V1.Auth.ResetVaultToken(); - List componentPaths = new List { }; + logger.MethodEntry(); try { - Secret listInfo = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storagePath, _mountPoint)); - - foreach (var path in listInfo.Data.Keys) + if (_storeType != StoreType.HCVKVPEM) { - if (!path.EndsWith("/")) - { - continue; - } + await CreateFileStore(); + return; + } + // for PEM stores, the store path is the container name, not entry name as with file stores - string fullPath = $"{storagePath}{path}"; - componentPaths.Add(fullPath); + await CreatePemStore(); + } + catch (Exception ex) + { + logger.LogError(ex, "Error when adding the new certificate."); + throw; + } + logger.MethodExit(); - List subPaths = await ListComponentPathsAsync(fullPath); - componentPaths.AddRange(subPaths); + } + + private async Task CreateFileStore() + { + IFileStore fileStore; + var parentPath = _storePath.Substring(0, _storePath.LastIndexOf("/")); + logger.LogTrace($"parent path = {parentPath}"); + var entryName = _storePath.Substring(_storePath.LastIndexOf("/")); + entryName = entryName.TrimStart('/'); + + switch (_storeType) + { + case StoreType.HCVKVPFX: + fileStore = new PfxFileStore(); + break; + + case StoreType.HCVKVPKCS12: + fileStore = new Pkcs12FileStore(); + break; + + case StoreType.KCVKVJKS: + fileStore = new JksFileStore(); + break; + + default: + throw new InvalidOperationException($"unrecognized store type value {_storeType}"); + } + + logger.LogTrace("generating a random string for the new store password."); + var passphrase = CertUtility.GenerateRandomString(16); + + logger.LogTrace("Creating the new filestore with the generated passphrase."); + var newStoreBytes = fileStore.CreateFileStore(passphrase); + + logger.LogTrace("Writing the passphrase and store file to the location in the store path."); + + try + { + + VaultClient.V1.Auth.ResetVaultToken(); + + var newData = new Dictionary { { entryName, Convert.ToBase64String(newStoreBytes) }, { "passphrase", passphrase } }; + + if (string.IsNullOrEmpty(_mountPoint)) + { + await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(parentPath, newData); + } + else + { + await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(parentPath, newData, null, _mountPoint); } } catch (Exception ex) { - logger.LogWarning($"Error while listing component paths: {ex}"); + logger.LogError("Error writing cert to Vault", ex); + throw; } - return componentPaths; + } - public async Task GetCertificate(string key) + private async Task CreatePemStore() { + //without a certificate, the only thing to do is create the secret path in Vault with empty values + var newData = new Dictionary { { "certificate", string.Empty }, { "private_key", string.Empty } }; + + try + { + if (_mountPoint == null) + { + await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(_storePath, newData); + } + else + { + await VaultClient.V1.Secrets.KeyValue.V2.WriteSecretAsync(_storePath, newData, mountPoint: _mountPoint); + } + } + catch (Exception ex) { + logger.LogError(ex, $"Error creating the PEM certificate store at path {_storePath}"); + throw; + } + } + + public async Task GetCertificateFromPemStore(string key) + { + logger.MethodEntry(); + VaultClient.V1.Auth.ResetVaultToken(); Dictionary certData; Secret res; var fullPath = _storePath + key; - var relativePath = fullPath.Substring(_storePath.Length); try { try { - if (_mountPoint == null) + if (string.IsNullOrEmpty(_mountPoint)) { - res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath)); + res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath); } else { - res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath, mountPoint: _mountPoint)); + res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(fullPath, mountPoint: _mountPoint); } } catch (Exception ex) { logger.LogError($"Error getting certificate {fullPath}", ex); - - return null; + throw; } certData = (Dictionary)res.Data.Data; @@ -118,12 +197,12 @@ public async Task GetCertificate(string key) try { - string certificate = null; + string certificate = null; //Validates if the "certificate" and "private_key" keys exist in certData - if (certData.TryGetValue("certificate", out object publicKeyObj)) + if (certData.TryGetValue(StoreFileExtensions.HCVKVPEM, out object publicKeyObj)) { - certificate = publicKeyObj as string; + certificate = publicKeyObj.ToString(); } var certs = new List() { certificate }; @@ -131,16 +210,17 @@ public async Task GetCertificate(string key) certData.TryGetValue("private_key", out object privateKeyObj); // if either field is missing, don't include it in inventory - - if (publicKeyObj == null || privateKeyObj == null) return null; + + if (publicKeyObj == null || privateKeyObj == null) return null; //split the chain entries (if chain is included) var certFooter = "\n-----END CERTIFICATE-----"; certs = certificate.Split(new string[] { certFooter }, StringSplitOptions.RemoveEmptyEntries).ToList(); - - for (int i = 0; i GetCertificate(string key) } catch (Exception ex) { - logger.LogError("Error parsing cert data", ex); + logger.LogError(ex, "Error parsing certificate data"); throw; } } - public async Task> GetVaults() + public async Task> GetVaults(string storePath) { - VaultClient.V1.Auth.ResetVaultToken(); + logger.MethodEntry(); + // there are 4 store types that use the KV secrets engine. HCVKVPEM uses the folder as the store path. The others (KCVKVJKS,HCVKVPKCS12,HCVKVPFX) use the full file path. + string suffix = ""; + storePath = storePath ?? _storePath; - var vaults = new List(); + if (string.IsNullOrEmpty(storePath)) { storePath = "/"; } + + var vaultPaths = new List(); + var entryPaths = new List(); + + logger.LogTrace("getting key suffix for store type. ", _storeType); + + switch (_storeType) + { + case StoreType.KCVKVJKS: + suffix = StoreFileExtensions.HCVKVJKS; + break; + case StoreType.HCVKVPFX: + suffix = StoreFileExtensions.HCVKVPFX; + break; + case StoreType.HCVKVPKCS12: + suffix = StoreFileExtensions.HCVKVPKCS12; + break; + default: + suffix = "certificate"; //PEM store + break; + } try { + logger.LogTrace("sending request to Vault."); + if (_mountPoint == null) { - vaults = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(_storePath)).Data.Keys.ToList(); + var res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storePath); + entryPaths = res.Data.Keys.ToList(); } else { - vaults = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(_storePath, _mountPoint)).Data.Keys.ToList(); + var res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storePath, _mountPoint); + entryPaths = res.Data.Keys.ToList(); } } catch (Exception ex) { logger.LogError(ex.Message); + throw; } - return vaults; + logger.LogTrace("checking paths at this level.", entryPaths); + + for (var i = 0; i < entryPaths.Count(); i++) + { + var path = entryPaths[i]; + if (!path.EndsWith("/")) + { // it is a secret, not a folder + // get the sub-keys for the secret entry + + IDictionary keys; + try + { + if (_mountPoint == null) + { + var res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretSubkeysAsync(storePath + path); + keys = res.Data.Subkeys; + } + else + { + var res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretSubkeysAsync(storePath + path, mountPoint: _mountPoint); + keys = res.Data.Subkeys; + } + // does it have an entry with the suffix we are looking for? + var key = keys.FirstOrDefault(k => k.Key.EndsWith(suffix)); + if (key.Key != null) + { + if (_storeType == StoreType.HCVKVPEM) + { + // PEM stores paths are the folder/container name rather than the entry name. + vaultPaths.Add(storePath); + } + else + { + vaultPaths.Add(storePath + path + "/" + key.Key); + } + } + } + catch (Exception ex) + { + logger.LogError("Error reading secret keys.", ex); + throw; + } + } + else + { //it is a sub-folder. Recurse. + var subPaths = await GetVaults(storePath + path); + vaultPaths.AddRange(subPaths); + } + } + logger.MethodExit(); + vaultPaths = vaultPaths.Distinct().ToList(); + return vaultPaths; } + public async Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain) { - VaultClient.V1.Auth.ResetVaultToken(); + logger.MethodEntry(); + try + { + if (_storeType != StoreType.HCVKVPEM) + { + await PutCertificateIntoFileStore(certName, contents, pfxPassword, includeChain); + return; + } + // for PEM stores, the store path is the container name, not entry name as with file stores + await PutCertificateIntoPemStore(certName, contents, pfxPassword, includeChain); + } + catch (Exception ex) + { + logger.LogError(ex, "Error when adding the new certificate."); + throw; + } + logger.MethodExit(); + } + private async Task PutCertificateIntoPemStore(string certName, string contents, string pfxPassword, bool includeChain) + { var certDict = new Dictionary(); - var pfxBytes = Convert.FromBase64String(contents); Pkcs12Store p; + using (var pfxBytesMemoryStream = new MemoryStream(pfxBytes)) { - p = new Pkcs12Store(pfxBytesMemoryStream, - pfxPassword.ToCharArray()); + Pkcs12StoreBuilder storeBuilder = new Pkcs12StoreBuilder(); + p = storeBuilder.Build(); + p.Load(pfxBytesMemoryStream, pfxPassword.ToCharArray()); } // Extract private key @@ -243,9 +424,12 @@ public async Task PutCertificate(string certName, string contents, string pfxPas } var pubCert = p.GetCertificate(alias).Certificate.GetEncoded(); - var pubCertPem = Pemify(Convert.ToBase64String(pubCert)); - // add the certs in the chain + logger.LogTrace("converting to PEM format."); + + var pubCertPem = CertUtility.Pemify(Convert.ToBase64String(pubCert)); + + logger.LogTrace("adding the chain certs"); var pemChain = new List(); var chain = p.GetCertificateChain(alias).ToList(); @@ -253,7 +437,7 @@ public async Task PutCertificate(string certName, string contents, string pfxPas chain.ForEach(c => { var cert = c.Certificate.GetEncoded(); - var encoded = Pemify(Convert.ToBase64String(cert)); + var encoded = CertUtility.Pemify(Convert.ToBase64String(cert)); pemChain.Add(encoded); }); @@ -266,10 +450,11 @@ public async Task PutCertificate(string certName, string contents, string pfxPas if (includeChain) { - certDict.Add("certificate", String.Join("\n", pemChain)); + certDict.Add(StoreFileExtensions.HCVKVPEM, String.Join("\n", pemChain)); } - else { - certDict.Add("certificate", pubCertPem); + else + { + certDict.Add(StoreFileExtensions.HCVKVPEM, pubCertPem); } } catch (Exception ex) @@ -279,6 +464,9 @@ public async Task PutCertificate(string certName, string contents, string pfxPas } try { + logger.LogTrace("writing secret to vault."); + VaultClient.V1.Auth.ResetVaultToken(); + var fullPath = _storePath + certName; if (_mountPoint == null) @@ -295,9 +483,249 @@ public async Task PutCertificate(string certName, string contents, string pfxPas logger.LogError("Error writing cert to Vault", ex); throw; } + logger.MethodExit(); } - public async Task DeleteCertificate(string certName) + private async Task PutCertificateIntoFileStore(string certName, string contents, string pfxPassword, bool includeChain) + { + logger.MethodEntry(); + + IFileStore fileStore; + var parentPath = _storePath.Substring(0, _storePath.LastIndexOf("/")); + logger.LogTrace($"parent path = {parentPath}"); + Secret res; + Dictionary certData; + + switch (_storeType) + { + case StoreType.HCVKVPFX: + fileStore = new PfxFileStore(); + break; + + case StoreType.HCVKVPKCS12: + fileStore = new Pkcs12FileStore(); + break; + + case StoreType.KCVKVJKS: + fileStore = new JksFileStore(); + break; + + default: + throw new InvalidOperationException($"unrecognized store type value {_storeType}"); + } + + try + { + // first get entry contents and passphrase + logger.LogTrace("getting all secrets in the parent container for the store."); + + if (string.IsNullOrEmpty(_mountPoint)) + { + res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(parentPath); + } + else + { + res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(parentPath, mountPoint: _mountPoint); + } + certData = (Dictionary)res.Data.Data; + logger.LogTrace("got secret data.", certData); + + string certificate = null; + string passphrase = null; + + //Validates if the "certificate" and "private_key" keys exist in certData + + var key = _storePath.Substring(_storePath.LastIndexOf("/")); + key = key.TrimStart('/'); + + logger.LogTrace($"getting the contents of {key}"); + + if (!certData.TryGetValue(key, out object certFileObj)) + { + throw new DirectoryNotFoundException($"entry named {key} not found at {parentPath}"); + } + certificate = certFileObj.ToString(); + + if (!certData.TryGetValue("passphrase", out object passphraseObj)) + { + throw new DirectoryNotFoundException($"no passphrase entry found at {parentPath}"); + } + passphrase = passphraseObj.ToString(); + + logger.LogTrace("got passphrase and certificate store secrets from vault."); + + logger.LogTrace("calling method to add certificate to store file."); + // get new store entry + var newEntry = fileStore.AddCertificate(certName, pfxPassword, contents, includeChain, certificate, passphrase); + logger.LogTrace("got new store file."); + // write new store entry + try + { + logger.LogTrace("writing file store with new certificate to vault."); + VaultClient.V1.Auth.ResetVaultToken(); + + var newData = new Dictionary { { key, newEntry } }; + var patchReq = new PatchSecretDataRequest() { Data = newData }; + + // temporary debugging code + var stringContent = new StringContent(JsonSerializer.Serialize(newData), Encoding.UTF8); + // + + if (string.IsNullOrEmpty(_mountPoint)) + { + await VaultClient.V1.Secrets.KeyValue.V2.PatchSecretAsync(parentPath, patchReq); + } + else + { + await VaultClient.V1.Secrets.KeyValue.V2.PatchSecretAsync(parentPath, patchReq, _mountPoint); + } + } + catch (Exception ex) + { + logger.LogError("Error writing cert to Vault", ex); + throw; + } + + } + catch (Exception ex) + { + logger.LogError(ex, $"Error adding certificate to {_storeType}: {ex.Message}"); + throw; + } + } + + public async Task RemoveCertificate(string certName) + { + logger.MethodEntry(); + try + { + if (_storeType != StoreType.HCVKVPEM) + { + await RemoveCertificateFromFileStore(certName); + return true; + } + // for PEM stores, the store path is the container name, not entry name as with file stores + + await RemoveCertificateFromPemStore(certName); + } + catch (Exception ex) + { + logger.LogError(ex, $"Error when removing the certificate with alias {certName}."); + throw; + } + logger.MethodExit(); + return true; + } + + public async Task RemoveCertificateFromFileStore(string certName) + { + logger.MethodEntry(); + + IFileStore fileStore; + var parentPath = _storePath.Substring(0, _storePath.LastIndexOf("/")); + logger.LogTrace($"parent path = {parentPath}"); + Secret res; + Dictionary certData; + + switch (_storeType) + { + case StoreType.HCVKVPFX: + fileStore = new PfxFileStore(); + break; + + case StoreType.HCVKVPKCS12: + fileStore = new Pkcs12FileStore(); + break; + + case StoreType.KCVKVJKS: + fileStore = new JksFileStore(); + break; + + default: + throw new InvalidOperationException($"unrecognized store type value {_storeType}"); + } + + try + { + // first get entry contents and passphrase + logger.LogTrace("getting all secrets in the parent container for the store."); + + if (string.IsNullOrEmpty(_mountPoint)) + { + res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(parentPath); + } + else + { + res = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(parentPath, mountPoint: _mountPoint); + } + certData = (Dictionary)res.Data.Data; + logger.LogTrace("got secret data.", certData); + + string certStoreContents = null; + string passphrase = null; + + //Validates if the "certificate" and "private_key" keys exist in certData + + var key = _storePath.Substring(_storePath.LastIndexOf("/")); + key = key.TrimStart('/'); + + logger.LogTrace($"getting the contents of {key}"); + + if (!certData.TryGetValue(key, out object certFileObj)) + { + throw new DirectoryNotFoundException($"entry named {key} not found at {parentPath}"); + } + certStoreContents = certFileObj.ToString(); + + if (!certData.TryGetValue("passphrase", out object passphraseObj)) + { + throw new DirectoryNotFoundException($"no passphrase entry found at {parentPath}"); + } + passphrase = passphraseObj.ToString(); + + logger.LogTrace("got passphrase and certificate store secrets from vault."); + + logger.LogTrace("calling method to remove certificate from store file."); + // get new store entry + var newEntry = fileStore.RemoveCertificate(certName, passphrase, certStoreContents); + logger.LogTrace("got new store file."); + // write new store entry + try + { + logger.LogTrace("writing file store sans certificate to vault."); + VaultClient.V1.Auth.ResetVaultToken(); + + var newData = new Dictionary { { key, newEntry } }; + var patchReq = new PatchSecretDataRequest() { Data = newData }; + + // temporary debugging code + var stringContent = new StringContent(JsonSerializer.Serialize(newData), Encoding.UTF8); + // + + if (string.IsNullOrEmpty(_mountPoint)) + { + await VaultClient.V1.Secrets.KeyValue.V2.PatchSecretAsync(parentPath, patchReq); + } + else + { + await VaultClient.V1.Secrets.KeyValue.V2.PatchSecretAsync(parentPath, patchReq, _mountPoint); + } + } + catch (Exception ex) + { + logger.LogError("Error writing file to Vault", ex); + throw; + } + + } + catch (Exception ex) + { + logger.LogError(ex, $"Error removing certificate {certName} from {_storeType}: {ex.Message}"); + throw; + } + } + + public async Task RemoveCertificateFromPemStore(string certName) { VaultClient.V1.Auth.ResetVaultToken(); @@ -316,51 +744,69 @@ public async Task DeleteCertificate(string certName) } catch (Exception ex) { - logger.LogError("Error removing cert from Vault", ex); + logger.LogError(ex, "Error removing cert from Vault"); throw; } - return true; } public async Task> GetCertificates() { + if (_storeType != StoreType.HCVKVPEM) + { + return await GetCertificatesFromFileStore(); + } + // for PEM stores, the store path is the container name, not entry name as with file stores + + return await GetCertificatesFromPemStore(); + } + + private async Task> GetCertificatesFromPemStore() + { + logger.MethodEntry(); + VaultClient.V1.Auth.ResetVaultToken(); - _storePath = _storePath.TrimStart('/'); List subPaths = new List(); + var certs = new List(); + var entryNames = new List(); + //Grabs the list of subpaths to get certificates from, if SubFolder Inventory is turned on. //Otherwise just define the single path _storePath + logger.LogDebug($"SubInventoryEnabled: {_subfolderInventory}"); + if (_subfolderInventory == true) { - subPaths = (await ListComponentPathsAsync(_storePath)); + logger.LogTrace("getting all sub-paths for container"); + subPaths = await GetSubPaths(_storePath); subPaths.Add(_storePath); } else { subPaths.Add(_storePath); } - var certs = new List(); - var certNames = new List(); - logger.LogDebug($"SubInventoryEnabled: {_subfolderInventory}"); + + logger.LogTrace($"got all subpaths for container {_storePath}", subPaths); + + foreach (var path in subPaths) { + var relative_path = path.Substring(_storePath.Length); try { - if (string.IsNullOrEmpty(_mountPoint)) { - certNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(path)).Data.Keys.ToList(); + entryNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(path)).Data.Keys.ToList(); } else { - certNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(path, mountPoint: _mountPoint)).Data.Keys.ToList(); + entryNames = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(path, mountPoint: _mountPoint)).Data.Keys.ToList(); } - certNames.ForEach(k => - { - var cert = GetCertificate($"{relative_path}{k}").Result; - if (cert != null) certs.Add(cert); - }); + entryNames.ForEach(k => + { + var cert = GetCertificateFromPemStore($"{relative_path}{k}").Result; + if (cert != null) certs.Add(cert); + }); } catch (Exception ex) { @@ -370,15 +816,96 @@ public async Task> GetCertificates() } return certs; } - private static Func Pemify = base64Cert => + + public async Task> GetCertificatesFromFileStore() { - string FormatBase64(string ss) => - ss.Length <= 64 ? ss : ss.Substring(0, 64) + "\n" + FormatBase64(ss.Substring(64)); + Secret res; + + //file stores for JKS, PKCS12 and PFX will have a "passphrase" entry on the same level by convention. We'll need this in order to extract the certificates for inventory. + var pos = _storePath.LastIndexOf("/"); + var parentPath = _storePath.Substring(0, pos); + + try + { + if (string.IsNullOrEmpty(_mountPoint)) + { + res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(parentPath)); + } + else + { + res = (await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(parentPath, mountPoint: _mountPoint)); + } + } + catch (Exception ex) + { + logger.LogError($"Error getting certificate data from {parentPath}", ex); + return null; + } + + var certFields = (Dictionary)res.Data.Data; + + IFileStore fileStore; + switch (_storeType) + { + case StoreType.HCVKVPFX: + fileStore = new PfxFileStore(); + break; + + case StoreType.HCVKVPKCS12: + fileStore = new Pkcs12FileStore(); + break; - string header = "-----BEGIN CERTIFICATE-----\n"; - string footer = "\n-----END CERTIFICATE-----"; + case StoreType.KCVKVJKS: + fileStore = new JksFileStore(); + break; - return header + FormatBase64(base64Cert) + footer; - }; + default: + throw new InvalidOperationException($"unrecognized store type value {_storeType}"); + } + + try + { + return fileStore.GetInventory(certFields); + } + catch (Exception ex) + { + logger.LogError(ex, $"Error performing inventory on store type {_storeType}: {ex.Message}"); + throw; + } + } + + private async Task> GetSubPaths(string storagePath) + { + logger.MethodEntry(); + + VaultClient.V1.Auth.ResetVaultToken(); + List componentPaths = new List { }; + try + { + logger.LogTrace("getting secret and path entries at this level.", storagePath); + + Secret listInfo = await VaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync(storagePath, _mountPoint); + + foreach (var path in listInfo.Data.Keys) + { + if (!path.EndsWith("/")) + { + continue; + } + + string fullPath = $"{storagePath}{path}"; + componentPaths.Add(fullPath); + + List subPaths = await GetSubPaths(fullPath); + componentPaths.AddRange(subPaths); + } + } + catch (Exception ex) + { + logger.LogWarning($"Error while listing component paths: {ex}"); + } + logger.MethodExit(); + return componentPaths; + } } } \ No newline at end of file diff --git a/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs b/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs index 33be1d8..252986d 100644 --- a/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs +++ b/hashicorp-vault-orchestrator/HcvKeyfactorClient.cs @@ -1,4 +1,4 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, @@ -27,7 +27,7 @@ public class HcvKeyfactorClient : IHashiClient private string _vaultUrl { get; set; } - private string _vaultToken { get; set; } + private string _vaultToken { get; set; } private string _mountPoint { get; set; } @@ -41,7 +41,7 @@ public HcvKeyfactorClient(string vaultToken, string serverUrl, string mountPoint _vaultUrl = $"{ serverUrl }/v1/{ _mountPoint.Replace("/", string.Empty) }"; } - public async Task GetCertificate(string key) + public async Task GetCertificateFromPemStore(string key) { var fullPath = $"{ _vaultUrl }/cert/{ key }"; @@ -55,10 +55,10 @@ public async Task GetCertificate(string key) req.Method = WebRequestMethods.Http.Get; var res = await req.GetResponseAsync(); CertResponse content = JsonConvert.DeserializeObject(new StreamReader(res.GetResponseStream()).ReadToEnd()); - - content.data.TryGetValue("certificate", out object cert); - content.data.TryGetValue("ca_chain", out object caChain); - content.data.TryGetValue("private_key", out object privateKey); + + content.data.TryGetValue("certificate", out object cert); + content.data.TryGetValue("ca_chain", out object caChain); + content.data.TryGetValue("private_key", out object privateKey); content.data.TryGetValue("revocation_time", out object revokeTime); List certList = new List() { cert as string }; @@ -70,7 +70,7 @@ public async Task GetCertificate(string key) string fullChain = caChain.ToString(); certList = fullChain.Split(new string[] { "\n\n" }, StringSplitOptions.RemoveEmptyEntries).ToList(); } - + // don't include them in inventory unless they haven't been revoked if (revokeTime == null || Equals(revokeTime.ToString(), "0")) @@ -103,6 +103,8 @@ public async Task GetCertificate(string key) public async Task> GetCertificates() { + logger.MethodEntry(); + var getKeysPath = $"{ _vaultUrl }/certs?list=true"; var certs = new List(); var certNames = new List(); @@ -113,15 +115,22 @@ public async Task> GetCertificates() req.Headers.Add("X-Vault-Request", "true"); req.Headers.Add("X-Vault-Token", _vaultToken); req.Method = WebRequestMethods.Http.Get; + + logger.LogTrace("sending request to vault for certs", req); + var res = await req.GetResponseAsync(); + + logger.LogTrace("parsing response", res); + var content = JsonConvert.DeserializeObject(new StreamReader(res.GetResponseStream()).ReadToEnd()); string[] certKeys; + content.data.TryGetValue("keys", out certKeys); certKeys.ToList().ForEach(k => { - var cert = GetCertificate(k).Result; + var cert = GetCertificateFromPemStore(k).Result; if (cert != null) certs.Add(cert); }); } @@ -130,22 +139,7 @@ public async Task> GetCertificates() logger.LogError(ex.Message); } return certs; - } - - public Task> GetVaults() - { - throw new NotSupportedException(); - } - - public Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain) - { - throw new NotSupportedException(); - } - - public Task DeleteCertificate(string certName) - { - throw new NotSupportedException(); - } + } public class HashiResponse { @@ -166,5 +160,25 @@ public class ListResponse : HashiResponse { public Dictionary data { get; set; } } + + public Task> GetVaults(string storePath) + { + throw new NotSupportedException(); + } + + public Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain) + { + throw new NotSupportedException(); + } + + public Task RemoveCertificate(string certName) + { + throw new NotSupportedException(); + } + + public Task CreateCertStore() + { + throw new NotSupportedException(); + } } } \ No newline at end of file diff --git a/hashicorp-vault-orchestrator/IHashiClient.cs b/hashicorp-vault-orchestrator/IHashiClient.cs index 043dd63..43cd706 100644 --- a/hashicorp-vault-orchestrator/IHashiClient.cs +++ b/hashicorp-vault-orchestrator/IHashiClient.cs @@ -14,9 +14,10 @@ namespace Keyfactor.Extensions.Orchestrator.HashicorpVault public interface IHashiClient { Task> GetCertificates(); - Task GetCertificate(string key); - Task> GetVaults(); + Task GetCertificateFromPemStore(string key); + Task> GetVaults(string storePath); Task PutCertificate(string certName, string contents, string pfxPassword, bool includeChain); - Task DeleteCertificate(string certName); + Task RemoveCertificate(string certName); + Task CreateCertStore(); } } diff --git a/hashicorp-vault-orchestrator/Jobs/Discovery.cs b/hashicorp-vault-orchestrator/Jobs/Discovery.cs index fb40c41..ecff776 100644 --- a/hashicorp-vault-orchestrator/Jobs/Discovery.cs +++ b/hashicorp-vault-orchestrator/Jobs/Discovery.cs @@ -1,4 +1,4 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, @@ -17,17 +17,17 @@ namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs { public class Discovery : JobBase, IDiscoveryJobExtension { - ILogger logger = LogHandler.GetClassLogger(); - public JobResult ProcessJob(DiscoveryJobConfiguration config, SubmitDiscoveryUpdate submitDiscoveryUpdate) { + logger = LogHandler.GetClassLogger(); + InitializeStore(config); - List vaults = new List(); + List vaults; try { - vaults = VaultClient.GetVaults().Result.ToList(); + vaults = VaultClient.GetVaults(string.Empty).Result.ToList(); } catch (Exception ex) { @@ -41,7 +41,7 @@ public JobResult ProcessJob(DiscoveryJobConfiguration config, SubmitDiscoveryUpd { logger.LogError("Attempt to perform discovery on unsupported Secrets Engine backend."); - result.FailureMessage = $"{SecretsEngine} does not support Discovery jobs."; + result.FailureMessage = $"{_storeType} does not support Discovery jobs."; } else { diff --git a/hashicorp-vault-orchestrator/Jobs/Inventory.cs b/hashicorp-vault-orchestrator/Jobs/Inventory.cs index 6232be6..e464a65 100644 --- a/hashicorp-vault-orchestrator/Jobs/Inventory.cs +++ b/hashicorp-vault-orchestrator/Jobs/Inventory.cs @@ -1,4 +1,4 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, @@ -17,16 +17,29 @@ namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs { public class Inventory : JobBase, IInventoryJobExtension { - ILogger logger = LogHandler.GetClassLogger(); - public JobResult ProcessJob(InventoryJobConfiguration config, SubmitInventoryUpdate submitInventoryUpdate) { + logger = LogHandler.GetClassLogger(); + InitializeStore(config); IEnumerable certs = null; try { certs = VaultClient.GetCertificates().Result; + var success = submitInventoryUpdate.Invoke(certs.ToList()); + + if (!success) + { + logger.LogTrace("failure submitting results to the platform."); + } + + return new JobResult + { + Result = success ? OrchestratorJobStatusJobResult.Success : OrchestratorJobStatusJobResult.Failure, + JobHistoryId = config.JobHistoryId, + FailureMessage = success ? string.Empty : "Error executing SubmitInventoryUpdate" + }; } catch (Exception ex) { @@ -39,15 +52,6 @@ public JobResult ProcessJob(InventoryJobConfiguration config, SubmitInventoryUpd FailureMessage = ex.Message }; } - - var success = submitInventoryUpdate.Invoke(certs.ToList()); - - return new JobResult - { - Result = success ? OrchestratorJobStatusJobResult.Success : OrchestratorJobStatusJobResult.Failure, - JobHistoryId = config.JobHistoryId, - FailureMessage = success ? string.Empty : "Error executing SubmitInventoryUpdate" - }; } } } diff --git a/hashicorp-vault-orchestrator/Jobs/JobBase.cs b/hashicorp-vault-orchestrator/Jobs/JobBase.cs index 83e8eb7..2af5476 100644 --- a/hashicorp-vault-orchestrator/Jobs/JobBase.cs +++ b/hashicorp-vault-orchestrator/Jobs/JobBase.cs @@ -1,11 +1,15 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions // and limitations under the License. +using System; +using System.Collections.Generic; using Keyfactor.Orchestrators.Extensions; +using Keyfactor.Orchestrators.Extensions.Interfaces; +using Microsoft.Extensions.Logging; using Newtonsoft.Json; namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs @@ -13,12 +17,12 @@ namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs public abstract class JobBase { public string ExtensionName => "HCV"; - + public string StorePath { get; set; } public string VaultToken { get; set; } - public string SecretsEngine { get; set; } // "PKI", "Keyfactor", "Key Value" + public string ClientMachine { get; set; } public string VaultServerUrl { get; set; } @@ -26,62 +30,84 @@ public abstract class JobBase public bool IncludeCertChain { get; set; } - public string MountPoint { get; set; } // the mount point of the KV secrets engine. defaults to KV + public string MountPoint { get; set; } // the mount point of the KV secrets engine. defaults to KV by Vault if not provided. - public string RoleName { get; set; } + internal protected IHashiClient VaultClient { get; set; } + internal protected string _storeType { get; set; } + internal protected ILogger logger { get; set; } + internal protected IPAMSecretResolver PamSecretResolver { get; set; } - internal protected IHashiClient VaultClient { get; set; } - - const string KEY_VALUE_ENGINE = "KV"; - const string KEYFACTOR_ENGINE = "Keyfactor"; - const string PKI_ENGINE = "Hashicorp PKI"; public void InitializeStore(InventoryJobConfiguration config) - { - var props = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties); - //var props = Jsonconfig.CertificateStoreDetails.Properties; + { + ClientMachine = config.CertificateStoreDetails.ClientMachine; - StorePath = config.CertificateStoreDetails?.StorePath ?? null; - StorePath = StorePath.TrimStart('/'); - StorePath = StorePath.TrimEnd('/'); - StorePath = StorePath == null ? null : StorePath + "/"; //enforce single trailing slash for path + // ClientId can be omitted for system assigned managed identities, required for user assigned or service principal auth + VaultServerUrl = PAMUtilities.ResolvePAMField(PamSecretResolver, logger, "Server UserName", config.ServerUsername); + // ClientSecret can be omitted for managed identities, required for service principal auth + VaultToken = PAMUtilities.ResolvePAMField(PamSecretResolver, logger, "Server Password", config.ServerPassword); + + StorePath = config.CertificateStoreDetails.StorePath; + ClientMachine = config.CertificateStoreDetails.ClientMachine; + + var props = JsonConvert.DeserializeObject>(config.CertificateStoreDetails.Properties); + InitProps(props, config.Capability); } public void InitializeStore(DiscoveryJobConfiguration config) { - var props = config.JobProperties; - InitProps(props, config.Capability); + ClientMachine = config.ClientMachine; + + // ClientId can be omitted for system assigned managed identities, required for user assigned or service principal auth + VaultServerUrl = PAMUtilities.ResolvePAMField(PamSecretResolver, logger, "Server UserName", config.ServerUsername); + + // ClientSecret can be omitted for managed identities, required for service principal auth + VaultToken = PAMUtilities.ResolvePAMField(PamSecretResolver, logger, "Server Password", config.ServerPassword); + + InitProps(config.JobProperties, config.Capability); } public void InitializeStore(ManagementJobConfiguration config) { - var props = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties); - StorePath = config.CertificateStoreDetails?.StorePath ?? null; - StorePath = StorePath.TrimStart('/'); - StorePath = StorePath.TrimEnd('/'); - StorePath = StorePath == null ? null : StorePath + "/"; //enforce single trailing slash for path + ClientMachine = config.CertificateStoreDetails.ClientMachine; + // ClientId can be omitted for system assigned managed identities, required for user assigned or service principal auth + VaultServerUrl = PAMUtilities.ResolvePAMField(PamSecretResolver, logger, "Server UserName", config.ServerUsername); + + // ClientSecret can be omitted for managed identities, required for service principal auth + VaultToken = PAMUtilities.ResolvePAMField(PamSecretResolver, logger, "Server Password", config.ServerPassword); + + StorePath = config.CertificateStoreDetails.StorePath; + ClientMachine = config.CertificateStoreDetails.ClientMachine; + dynamic props = JsonConvert.DeserializeObject(config.CertificateStoreDetails.Properties.ToString()); InitProps(props, config.Capability); } private void InitProps(dynamic props, string capability) { - if (props == null) throw new System.Exception("Properties is null", props); + _storeType = capability; - VaultToken = props["VaultToken"]; - VaultServerUrl = props["VaultServerUrl"]; - SecretsEngine = props["SecretsEngine"]; - MountPoint = props["MountPoint"] ?? null; + if (props == null) throw new Exception("Properties is null"); - SubfolderInventory = props["SubfolderInventory"] ?? false; - IncludeCertChain = props["IncludeCertChain"] ?? false; + if (props.ContainsKey("StorePath")) { + StorePath = props["StorePath"].ToString(); + StorePath = StorePath.TrimStart('/'); + StorePath = StorePath.TrimEnd('/'); + if (_storeType.Contains(StoreType.HCVKVPEM) || _storeType.Contains(StoreType.HCVPKI)) { + StorePath += "/"; //ensure single trailing slash for path for PKI or PEM stores. Others use the entry value instead of the container. + } + } - var isPki = capability.Contains("HCVPKI"); + MountPoint = props.ContainsKey("MountPoint") ? props["MountPoint"].ToString() : null; + SubfolderInventory = props.ContainsKey("SubfolderInventory") ? bool.Parse(props["SubfolderInventory"].ToString()) : false; + IncludeCertChain = props.ContainsKey("IncludeCertChain") ? bool.Parse(props["IncludeCertChain"].ToString()) : false; + + var isPki = _storeType.Contains("HCVPKI"); if (!isPki) { - VaultClient = new HcvKeyValueClient(VaultToken, VaultServerUrl, MountPoint, StorePath, SubfolderInventory); + VaultClient = new HcvKeyValueClient(VaultToken, VaultServerUrl, MountPoint, StorePath, _storeType, SubfolderInventory); } else { diff --git a/hashicorp-vault-orchestrator/Jobs/Management.cs b/hashicorp-vault-orchestrator/Jobs/Management.cs index 0670d32..76c63af 100644 --- a/hashicorp-vault-orchestrator/Jobs/Management.cs +++ b/hashicorp-vault-orchestrator/Jobs/Management.cs @@ -1,4 +1,4 @@ -// Copyright 2022 Keyfactor +// Copyright 2023 Keyfactor // Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. // You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 // Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, @@ -15,10 +15,10 @@ namespace Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs { public class Management : JobBase, IManagementJobExtension { - readonly ILogger logger = LogHandler.GetClassLogger(); - public JobResult ProcessJob(ManagementJobConfiguration config) { + logger = LogHandler.GetClassLogger(); + InitializeStore(config); JobResult complete = new JobResult() @@ -30,20 +30,45 @@ public JobResult ProcessJob(ManagementJobConfiguration config) switch (config.OperationType) { case CertStoreOperationType.Add: - logger.LogDebug($"Begin Management > Add..."); + logger.LogDebug("Begin Management > Add..."); complete = PerformAddition(config.JobCertificate.Alias, config.JobCertificate.PrivateKeyPassword, config.JobCertificate.Contents, config.JobHistoryId); break; case CertStoreOperationType.Remove: - logger.LogDebug($"Begin Management > Remove..."); + logger.LogDebug("Begin Management > Remove..."); complete = PerformRemoval(config.JobCertificate.Alias, config.JobHistoryId); break; + case CertStoreOperationType.Create: + logger.LogDebug("Begin Management > Create..."); + complete = PerformCreateCertStore(config); + break; } return complete; } + protected virtual JobResult PerformCreateCertStore(ManagementJobConfiguration config) + { + logger.MethodEntry(); + + var complete = new JobResult { Result = OrchestratorJobStatusJobResult.Failure, JobHistoryId = config.JobHistoryId }; + + try + { + VaultClient.CreateCertStore(); + complete.Result = OrchestratorJobStatusJobResult.Success; + } + catch (Exception ex) + { + logger.LogError(ex, "Error when trying to create the new certificate store."); + complete.FailureMessage = $"Error when trying to create the new certificate store. {ex.Message}"; + } + return complete; + } + protected virtual JobResult PerformAddition(string alias, string pfxPassword, string entryContents, long jobHistoryId) { + logger.MethodEntry(); + var complete = new JobResult() { Result = OrchestratorJobStatusJobResult.Failure, JobHistoryId = jobHistoryId }; if (!string.IsNullOrWhiteSpace(pfxPassword)) // This is a PFX Entry @@ -56,8 +81,8 @@ protected virtual JobResult PerformAddition(string alias, string pfxPassword, st try { - // uploadCollection is either not null or an exception was thrown. var cert = VaultClient.PutCertificate(alias, entryContents, pfxPassword, IncludeCertChain); + cert.Wait(); complete.Result = OrchestratorJobStatusJobResult.Success; } catch (Exception ex) @@ -65,11 +90,11 @@ protected virtual JobResult PerformAddition(string alias, string pfxPassword, st if (ex.GetType() == typeof(NotSupportedException)) { logger.LogError("Attempt to Add Certificate on unsupported Secrets Engine backend."); - complete.FailureMessage = $"{SecretsEngine} does not support adding certificates via the Orchestrator."; + complete.FailureMessage = $"{_storeType} does not support adding certificates via the Orchestrator."; } else { - complete.FailureMessage = $"An error occured while adding {alias} to {ExtensionName}: " + ex.Message; + complete.FailureMessage = $"An error occured while adding {alias} to {StorePath}: " + ex.Message; if (ex.InnerException != null) complete.FailureMessage += " - " + ex.InnerException.Message; @@ -97,7 +122,7 @@ protected virtual JobResult PerformRemoval(string alias, long jobHistoryId) try { - var success = VaultClient.DeleteCertificate(alias).Result; + var success = VaultClient.RemoveCertificate(alias).Result; if (!success) { @@ -114,12 +139,12 @@ protected virtual JobResult PerformRemoval(string alias, long jobHistoryId) if (ex.GetType() == typeof(NotSupportedException)) { logger.LogError("Attempt to Delete Certificate on unsupported Secrets Engine backend."); - complete.FailureMessage = $"{SecretsEngine} does not support removing certificates via the Orchestrator."; + complete.FailureMessage = $"{_storeType} does not support removing certificates via the Orchestrator."; } else { logger.LogError("Error deleting cert from Vault", ex); - complete.FailureMessage = $"An error occured while removing {alias} from {ExtensionName}: " + ex.Message; + complete.FailureMessage = $"An error occured while removing {alias} from {StorePath}: " + ex.Message; } } return complete; diff --git a/hashicorp-vault-orchestrator/PamUtilities.cs b/hashicorp-vault-orchestrator/PamUtilities.cs new file mode 100644 index 0000000..cdbd427 --- /dev/null +++ b/hashicorp-vault-orchestrator/PamUtilities.cs @@ -0,0 +1,26 @@ +// Copyright 2023 Keyfactor +// Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. +// You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 +// Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions +// and limitations under the License. + +using Keyfactor.Orchestrators.Extensions.Interfaces; +using Microsoft.Extensions.Logging; + +namespace Keyfactor.Extensions.Orchestrator.HashicorpVault +{ + internal class PAMUtilities + { + internal static string ResolvePAMField(IPAMSecretResolver resolver, ILogger logger, string name, string key) + { + if (resolver == null) return key; + else + { + return resolver.Resolve(key); + } + + } + } +} + diff --git a/hashicorp-vault-orchestrator/hashicorp-vault-orchestrator.csproj b/hashicorp-vault-orchestrator/hashicorp-vault-orchestrator.csproj index 59f1e0b..a7c518b 100644 --- a/hashicorp-vault-orchestrator/hashicorp-vault-orchestrator.csproj +++ b/hashicorp-vault-orchestrator/hashicorp-vault-orchestrator.csproj @@ -25,7 +25,7 @@ - DEBUG;TRACE + Auto false false @@ -33,26 +33,25 @@ + all - + all - + + all - - all - - - - + + all - + + all diff --git a/hashicorp-vault-orchestrator/manifest.json b/hashicorp-vault-orchestrator/manifest.json index 559c5a3..12610d0 100644 --- a/hashicorp-vault-orchestrator/manifest.json +++ b/hashicorp-vault-orchestrator/manifest.json @@ -1,18 +1,59 @@ { "extensions": { "Keyfactor.Orchestrators.Extensions.IOrchestratorJobExtension": { - "CertStores.HCVKV.Inventory": { + + "CertStores.HCVKVPEM.Inventory": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" }, - "CertStores.HCVKV.Discovery": { + "CertStores.HCVKVPEM.Discovery": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Discovery" }, - "CertStores.HCVKV.Management": { + "CertStores.HCVKVPEM.Management": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Management" }, + + "CertStores.HCVKVJKS.Inventory": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" + }, + "CertStores.HCVKVJKS.Discovery": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Discovery" + }, + "CertStores.HCVKVJKS.Management": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Management" + }, + + "CertStores.HCVKVP12.Inventory": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" + }, + "CertStores.HCVKVP12.Discovery": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Discovery" + }, + "CertStores.HCVKVP12.Management": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Management" + }, + + "CertStores.HCVKVPFX.Inventory": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" + }, + "CertStores.HCVKVPFX.Discovery": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Discovery" + }, + "CertStores.HCVKVPFX.Management": { + "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", + "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Management" + }, + "CertStores.HCVPKI.Inventory": { "assemblypath": "Keyfactor.Extensions.Orchestrator.HCV.dll", "TypeFullName": "Keyfactor.Extensions.Orchestrator.HashicorpVault.Jobs.Inventory" diff --git a/images/cert_store_add_dialog.png b/images/cert_store_add_dialog.png index 3cd444f..58ae385 100644 Binary files a/images/cert_store_add_dialog.png and b/images/cert_store_add_dialog.png differ diff --git a/images/store-type-kv.PNG b/images/store-type-kv.PNG index e85c2ad..9296ce4 100644 Binary files a/images/store-type-kv.PNG and b/images/store-type-kv.PNG differ diff --git a/images/store_type_fields.png b/images/store_type_fields.png index c727456..8e62a8b 100644 Binary files a/images/store_type_fields.png and b/images/store_type_fields.png differ diff --git a/integration-manifest.json b/integration-manifest.json index e2aa800..728db26 100644 --- a/integration-manifest.json +++ b/integration-manifest.json @@ -31,9 +31,9 @@ }, "store_types": [ { - "Name": "Hashicorp Vault Key-Value", - "ShortName": "HCVKV", - "Capability": "HCVKV", + "Name": "Hashicorp Vault Key-Value PEM", + "ShortName": "HCVKVPEM", + "Capability": "HCVKVPEM", "LocalStore": false, "SupportedOperations": { "Add": true, @@ -44,44 +44,52 @@ }, "Properties": [ { - "Name": "MountPoint", - "DisplayName": "Mount Point", - "Type": "String", + "Name": "SubfolderInventory", + "DisplayName": "Subfolder Inventory", + "Type": "Bool", "DependsOn": "", - "DefaultValue": null, + "DefaultValue": "false", "Required": false }, { - "Name": "VaultToken", - "DisplayName": "Vault Token", + "Name": "IncludeCertChain", + "DisplayName": "Include Certificate Chain", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "MountPoint", + "DisplayName": "Mount Point", "Type": "String", "DependsOn": "", "DefaultValue": null, "Required": false }, { - "Name": "VaultServerUrl", - "DisplayName": "Vault Server URL", - "Type": "String", + "Name": "ServerUsername", + "DisplayName": "Server Username", + "Type": "Secret", "DependsOn": "", "DefaultValue": null, "Required": false }, { - "Name": "SubfolderInventory", - "DisplayName": "Subfolder Inventory", - "Type": "Bool", + "Name": "ServerPassword", + "DisplayName": "Server Password", + "Type": "Secret", "DependsOn": "", - "DefaultValue": "false", + "DefaultValue": null, "Required": false }, { - "Name": "IncludeCertChain", - "DisplayName": "Include Cert Chain", + "Name": "ServerUseSsl", + "DisplayName": "Use SSL", "Type": "Bool", "DependsOn": "", "DefaultValue": "true", - "Required": false + "Required": true } ], "EntryParameters": null, @@ -90,11 +98,9 @@ "StoreRequired": false, "Style": "Default" }, - "StorePathType": "", - "StorePathValue": "", "PrivateKeyAllowed": "Optional", "JobProperties": [], - "ServerRequired": false, + "ServerRequired": true, "PowerShell": false, "BlueprintAllowed": false, "CustomAliasAllowed": "Optional" @@ -143,16 +149,238 @@ "StoreRequired": false, "Style": "Default" }, - "StorePathType": "", - "StorePathValue": "", "PrivateKeyAllowed": "Optional", "JobProperties": [], "ServerRequired": false, "PowerShell": false, "BlueprintAllowed": false, "CustomAliasAllowed": "Optional" + }, + { + "Name": "Hashicorp Vault Key-Value JKS", + "ShortName": "HCVKVJKS", + "Capability": "HCVKVJKS", + "LocalStore": false, + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "Properties": [ + { + "Name": "SubfolderInventory", + "DisplayName": "Sub-folder Inventory", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "IncludeCertChain", + "DisplayName": "Include Certificate Chain", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "MountPoint", + "DisplayName": "Mount Point", + "Type": "String", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerUsername", + "DisplayName": "Server Username", + "Type": "Secret", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerPassword", + "DisplayName": "Server Password", + "Type": "Secret", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerUseSsl", + "DisplayName": "Use SSL", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "true", + "Required": true + } + ], + "EntryParameters": null, + "PasswordOptions": { + "EntrySupported": false, + "StoreRequired": false, + "Style": "Default" + }, + "PrivateKeyAllowed": "Optional", + "JobProperties": [], + "ServerRequired": true, + "PowerShell": false, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Optional" + }, + { + "Name": "Hashicorp Vault Key-Value PKCS12", + "ShortName": "HCVKVP12", + "Capability": "HCVKVP12", + "LocalStore": false, + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "Properties": [ + { + "Name": "SubfolderInventory", + "DisplayName": "Sub-folder Inventory", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "MountPoint", + "DisplayName": "Mount Point", + "Type": "String", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "IncludeCertChain", + "DisplayName": "Include Certificate Chain", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "ServerUsername", + "DisplayName": "Server Username", + "Type": "Secret", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerPassword", + "DisplayName": "Server Password", + "Type": "Secret", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerUseSsl", + "DisplayName": "Use SSL", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "true", + "Required": true + } + ], + "EntryParameters": null, + "PasswordOptions": { + "EntrySupported": false, + "StoreRequired": false, + "Style": "Default" + }, + "PrivateKeyAllowed": "Optional", + "JobProperties": [], + "ServerRequired": true, + "PowerShell": false, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Optional" + }, + { + "Name": "Hashicorp Vault Key-Value PFX", + "ShortName": "HCVKVPFX", + "Capability": "HCVKVPFX", + "LocalStore": false, + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "Properties": [ + { + "Name": "SubfolderInventory", + "DisplayName": "Sub-folder Inventory", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "IncludeCertChain", + "DisplayName": "Include Certificate Chain", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "false", + "Required": false + }, + { + "Name": "MountPoint", + "DisplayName": "Mount Point", + "Type": "String", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerUsername", + "DisplayName": "Server Username", + "Type": "Secret", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerPassword", + "DisplayName": "Server Password", + "Type": "Secret", + "DependsOn": "", + "DefaultValue": null, + "Required": false + }, + { + "Name": "ServerUseSsl", + "DisplayName": "Use SSL", + "Type": "Bool", + "DependsOn": "", + "DefaultValue": "true", + "Required": true + } + ], + "EntryParameters": null, + "PasswordOptions": { + "EntrySupported": false, + "StoreRequired": false, + "Style": "Default" + }, + "PrivateKeyAllowed": "Optional", + "JobProperties": [], + "ServerRequired": true, + "PowerShell": false, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Optional" } - ] } } diff --git a/readme-src/readme-pam-support.md b/readme-src/readme-pam-support.md new file mode 100644 index 0000000..a040c91 --- /dev/null +++ b/readme-src/readme-pam-support.md @@ -0,0 +1,4 @@ +|Name|Description| +|----|-----------| +|Server Username|The url to the instance of vault from the orchestrator| +|Server Password|The vault token that will be used for authentication| diff --git a/readme_source.md b/readme_source.md index e9a1b44..e9ccdf6 100644 --- a/readme_source.md +++ b/readme_source.md @@ -8,17 +8,39 @@ This integration supports 3 Hashicorp Secrets Engines; PKI, Key-Value store, and ### The Key-Value secrets engine -The Following operations are supported by this integration **only** for the Key-Value secrets engine. +For the Key-Value secrets engine, we have 4 store types that can be used. -1. Discovery - Discover all sub-paths containing certificate. -1. Inventory - Return all certificates stored in a path. +- *HCVKVJKS* - For JKS certificate files, treats each file as it's own store. +- *HCVKVPFX* - For PFX certificate files, treats each file as it's own store. +- *HCVKVP12* - For PKCS12 certificate files, treats each file as it's own store. +- *HCVKVPEM* - For PEM encoded certificates, treats each _path_ as it's own store. Each certificate exists in a sub-path from the store path. + +The following operations are supported by this integration for all of the Key-Value secrets engine types: + +1. Discovery - Discovery all file repositories for the type +1. Inventory - Inventory all certificates in the path 1. Management (Add) - Add a certificate to a defined certificate store. 1. Management (Remove) - Remove a certificate from a defined certificate store. +1. Create - Create a new, empty certificate store at the path defined in Store Path. + + +Excluding *HCVKVPEM*, the discovery process requires that: +1. The entry for the certificate contain the base64 encoded certificate file. +1. The name (key) for the entry ends with the suffix corresponding to the certificate store type: + 1. *HCVKVJKS* - `*_jks` + 1. *HCVKVPFX* - `*_pfx` + 1. *HCVKVP12* - `*_p12` + 1. *HCVKVPEM* - `certificate` +1. For all except *HCVKVPEM*, there be an entry named `passphrase` that contains the password for the store. +1. For *HCVKVPEM*, there be an entry named `private_key` containing the private key portion of the key-pair. + +**Note**: Key/Value secrets that do not include the expected keys will be ignored during inventory scans. ### The Hashicorp PKI and Keyfactor Plugin secrets engines -Both the Hashicorp PKI and Keyfactor plugin are designed to allow managing certifications directly on the Hashicorp Vault instance. -This integration does support the following in order to view your certificates from the platform: +Both the Hashicorp PKI and Keyfactor Secrets Engine plugins are designed to allow managing certifications directly on the Hashicorp Vault instance. +The store type for the PKI and/or the Keyfactor secrets engine is the same; `HCVPKI`. +This integration supports the following in order to view your certificates from the platform: 1. Inventory - Return all certificates stored in a path. @@ -36,13 +58,6 @@ This integration was built on the .NET Core 3.1 target framework and are compati 1. It is not necessary to use the Vault root token when creating a Certificate Store for HashicorpVault. We recommend creating a token with policies that reflect the minimum permissions necessary to perform the intended operations. -1. For the Key-Value secrets engine, the certificates are stored as an entry with these fields. - -- `certificate` - The PEM formatted certificate and intermediate CA chain (if selected) -- `private_key` - The certificate private key - -**Note**: Key/Value secrets that do not include the keys `certificate` and `private_key` will be ignored during inventory scans. - ## Extension Configuration ### On the Orchestrator Agent Machine @@ -57,7 +72,7 @@ This integration was built on the .NET Core 3.1 target framework and are compati ### In the Keyfactor Platform -#### Add a new Certificate Store Type - **Key-Value Secrets Engine** +#### Add a new Certificate Store Type - **Hashicorp Vault Key-Value PEM** - Log into Keyfactor as Administrator or a user with permissions to add certificate store types. - Click on the gear icon in the top right and then navigate to the "Certificate Store Types" @@ -66,9 +81,11 @@ This integration was built on the .NET Core 3.1 target framework and are compati ![](images/store_type_add.png) - Set the following values in the "Basic" tab: - - **Name:** "Hashicorp Vault Key-Value" (or another preferred name) - - **Short Name:** "HCVKV" + - **Name:** "Hashicorp Vault Key-Value PEM" (or another preferred name) + - **Short Name:** "HCVKVPEM" - **Supported Job Types** - "Inventory", "Add", "Remove", "Discovery" + - **NOTE** If you are setting up "`HCVKVJKS`, `HCVKVPFX`, or `HCVKVP12` the supported job types will be "Inventory, Discovery". + - **Needs Server** - should be checked (true). ![](images/store-type-kv.PNG) @@ -80,13 +97,14 @@ This integration was built on the .NET Core 3.1 target framework and are compati - Click the "Custom Fields" tab to add the following custom fields: - **MountPoint** - type: *string* - - **VaultServerUrl** - type: *string*, *required* - - **VaultToken** - type: *secret*, *required* - **SubfolderInventory** - type: *bool* (By default, this is set to false. Not a required field) - **IncludeCertChain** - type: *bool* (If true, the available intermediate certificates will also be written to Vault during enrollment) ![](images/store_type_fields.png) +**Note** +The 3 highlighted fields above will be added automatically by the platform, you will not need to include them when creating the certificate store type. + - Click **Save** to save the new Store Type. #### Add the Hashicorp Vault Certificate Store - **Key-Value Secrets Engine** @@ -100,16 +118,19 @@ In Keyfactor Command create a new Certificate Store that resembles the one below ![](images/cert_store_fields.png) -- **Client Machine** - Enter the URL for the Vault host machine +- **Client Machine** - Enter an identifier for the client machine. This could be the Orchestrator host name, or anything else useful. This value is not used by the extension. - **Store Path** - This is the path after mount point where the certs will be stored. - example: `kv-v2\kf-secrets\certname` would use the path "\kf-secrets" - **Mount Point** - This is the mount point name for the instance of the Key Value secrets engine. - If left blank, will default to "kv-v2". - If your organization utilizes Vault enterprise namespaces, you should include the namespace here. -- **Vault Token** - This is the access token that will be used by the orchestrator for requests to Vault. -- **Vault Server Url** - the full url and port of the Vault server instance - **Subfolder Inventory** - Set to 'True' if it is a requirement to inventory secrets at the subfolder/component level. The default, 'False' will inventory secrets stored at the root of the "Store Path", but will not look at secrets in subfolders. **Note** that there is a limit on the number of certificates that can be in a certificate store. In certain environments enabling Subfolder Inventory may exceed this limit and cause inventory job failure. Inventory job results are currently submitted to the Command platform as a single HTTP POST. There is not a specific limit on the number of certificates in a store, rather the limit is based on the size of the actual certificates and the HTTP POST size limit configured on the Command web server. +#### Set the server name and password + +- The server name should be the full URL to the instance of Vault that will be accessible by the orchestrator. (example: `http://127.0.0.1:8200`) +- The server password should be the Vault token that will be used for authenticating. + ### For the Keyfactor and PKI plugins - Add a new Certificate Store Type