diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f6c4fcc..2566e1e2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,9 @@ +v2.2.0 +- Add ability to manage same windows server as installed without using WinRM +- Check for "core" version of PowerShell for command tweaks +- Bug fix: Preserve store permissions and file ownership when using separate upload file path +- Bug fix: Fixed issue adding certificates to stores with embedded spaces in path (Windows managed stores only) + v2.1.2 - Bug fix: Discovery not working against Windows servers - Bug fix: Issue running Discovery on Windows servers with one or more spaces in the path diff --git a/Certificate Store Type CURL Scripts/PEM.curl b/Certificate Store Type CURL Scripts/PEM.curl index f8755054..4a20f072 100644 --- a/Certificate Store Type CURL Scripts/PEM.curl +++ b/Certificate Store Type CURL Scripts/PEM.curl @@ -68,7 +68,7 @@ curl -X POST {URL}/keyfactorapi/certificatestoretypes -H "Content-Type: applicat "DefaultValue": "" }, { - "Name": "IsRSAPrivateKey" + "Name": "IsRSAPrivateKey", "DisplayName": "Is RSA Private Key", "Required": false, "DependsOn": "", diff --git a/README.md b/README.md index 5b01adee..77a05b70 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,15 @@ The Remote File Orchestrator allows for the remote management of file-based cert #### Integration status: Production - Ready for use in production environments. -## About the Keyfactor Universal Orchestrator Capability -This repository contains a Universal Orchestrator Capability which is a plugin to the Keyfactor Universal Orchestrator. Within the Keyfactor Platform, Orchestrators are used to manage “certificate stores” — collections of certificates and roots of trust that are found within and used by various applications. +## About the Keyfactor Universal Orchestrator Extension -The Universal Orchestrator is part of the Keyfactor software distribution and is available via the Keyfactor customer portal. For general instructions on installing Capabilities, see the “Keyfactor Command Orchestrator Installation and Configuration Guide” section of the Keyfactor documentation. For configuration details of this specific Capability, see below in this readme. +This repository contains a Universal Orchestrator Extension which is a plugin to the Keyfactor Universal Orchestrator. Within the Keyfactor Platform, Orchestrators are used to manage “certificate stores” — collections of certificates and roots of trust that are found within and used by various applications. + +The Universal Orchestrator is part of the Keyfactor software distribution and is available via the Keyfactor customer portal. For general instructions on installing Extensions, see the “Keyfactor Command Orchestrator Installation and Configuration Guide” section of the Keyfactor documentation. For configuration details of this specific Extension see below in this readme. + +The Universal Orchestrator is the successor to the Windows Orchestrator. This Orchestrator Extension plugin only works with the Universal Orchestrator and does not work with the Windows Orchestrator. -The Universal Orchestrator is the successor to the Windows Orchestrator. This Capability plugin only works with the Universal Orchestrator and does not work with the Windows Orchestrator. @@ -50,34 +52,43 @@ The secrets that this orchestrator extension supports for use with a PAM Provide |StorePassword|The optional password used to secure the certificate store being managed| -It is not necessary to implement all of the secrets available to be managed by a PAM provider. For each value that you want managed by a PAM provider, simply enter the key value inside your specific PAM provider that will hold this value into the corresponding field when setting up the certificate store, discovery job, or API call. +It is not necessary to use a PAM Provider for all of the secrets available above. If a PAM Provider should not be used, simply enter in the actual value to be used, as normal. + +If a PAM Provider will be used for one of the fields above, start by referencing the [Keyfactor Integration Catalog](https://keyfactor.github.io/integrations-catalog/content/pam). The GitHub repo for the PAM Provider to be used contains important information such as the format of the `json` needed. What follows is an example but does not reflect the `json` values for all PAM Providers as they have different "instance" and "initialization" parameter names and values. + +
General PAM Provider Configuration +

+ + + +### Example PAM Provider Setup -Setting up a PAM provider for use involves adding an additional section to the manifest.json file for this extension as well as setting up the PAM provider you will be using. Each of these steps is specific to the PAM provider you will use and are documented in the specific GitHub repo for that provider. For a list of Keyfactor supported PAM providers, please reference the [Keyfactor Integration Catalog](https://keyfactor.github.io/integrations-catalog/content/pam). +To use a PAM Provider to resolve a field, in this example the __Server Password__ will be resolved by the `Hashicorp-Vault` provider, first install the PAM Provider extension from the [Keyfactor Integration Catalog](https://keyfactor.github.io/integrations-catalog/content/pam) on the Universal Orchestrator. +Next, complete configuration of the PAM Provider on the UO by editing the `manifest.json` of the __PAM Provider__ (e.g. located at extensions/Hashicorp-Vault/manifest.json). The "initialization" parameters need to be entered here: -### Register the PAM Provider +~~~ json + "Keyfactor:PAMProviders:Hashicorp-Vault:InitializationInfo": { + "Host": "http://127.0.0.1:8200", + "Path": "v1/secret/data", + "Token": "xxxxxx" + } +~~~ -A PAM Provider needs to be registered on the Universal Orchestrator in the same way other extensions are. Create a folder for the specific PAM Provider to be added, and place the contents of the PAM Provider into the folder. There needs to be a manifest.json with the PAM Provider. +After these values are entered, the Orchestrator needs to be restarted to pick up the configuration. Now the PAM Provider can be used on other Orchestrator Extensions. -After a manifest.json is added, the final step for configuration is setting the "provider-level" parameters for the PAM Provider. These are also known as the "initialization-level" parameters. These need to be placed in a json file that gets loaded by the Orchestrator by default. +### Use the PAM Provider +With the PAM Provider configured as an extenion on the UO, a `json` object can be passed instead of an actual value to resolve the field with a PAM Provider. Consult the [Keyfactor Integration Catalog](https://keyfactor.github.io/integrations-catalog/content/pam) for the specific format of the `json` object. -example manifest.json for MY-PROVIDER-NAME -``` -{ - "extensions": { - "Keyfactor.Platform.Extensions.IPAMProvider": { - "PAMProviders.MY-PROVIDER-NAME.PAMProvider": { - "assemblyPath": "my-pam-provider.dll", - "TypeFullName": "Keyfactor.Extensions.Pam.MyPamProviderClass" - } - } - }, - "Keyfactor:PAMProviders:MY-PROVIDER-NAME:InitializationInfo": { - "InitParam1": "InitValue1", - "InitParam2": "InitValue2" - } -} -``` +To have the __Server Password__ field resolved by the `Hashicorp-Vault` provider, the corresponding `json` object from the `Hashicorp-Vault` extension needs to be copied and filed in with the correct information: + +~~~ json +{"Secret":"my-kv-secret","Key":"myServerPassword"} +~~~ + +This text would be entered in as the value for the __Server Password__, instead of entering in the actual password. The Orchestrator will attempt to use the PAM Provider to retrieve the __Server Password__. If PAM should not be used, just directly enter in the value for the field. +

+
@@ -117,22 +128,19 @@ The version number of a the Remote File Orchestrator Extension can be verified b 1. The Remote File Orchestrator Extension makes use of a few common Linux commands when managing stores on Linux servers. If the credentials you will be connecting with need elevated access to run these commands, you must set up the user id as a sudoer with no password necessary and set the config.json "UseSudo" value to "Y" (See "Config File Setup" later in this README for more information on setting up the config.json file). The full list of these commands below: * echo * find - * cp + * tee * rm * chown * install 2. The Remote File Orchestrator Extension makes use of SFTP and/or SCP to transfer files to and from the orchestrated server. SFTP/SCP cannot make use of sudo, so all folders containing certificate stores will need to allow SFTP/SCP file transfer. If this is not possible, set the values in the config.json apprpriately to use an alternative upload/download folder that does allow SFTP/SCP file transfer (See "Config File Setup" later in this README regarding the config.json file). +3. SSH Key Authentication: When creating a Keyfactor certificate store for the remote file orchestrator extension (see "Creating Certificate Stores" later in this README, you may supply either a user id and password for the certificate store credentials (directly or through one of Keyfactor Command's PAM integrations), or a user id and SSH private key. Both PKCS#1 (BEGIN RSA PRIVATE KEY) and PKCS#8 (BEGIN PRIVATE KEY) formats are supported for the SSH private key. If using the normal Keyfactor Command credentials dialog without PAM integration, just copy and paste the full SSH private key into the Password textbox. + **For Windows orchestrated servers:** 1. Make sure that WinRM is set up on the orchestrated server and that the WinRM port is part of the certificate store path when setting up your certificate stores When creating a new certificate store in Keyfactor Command (See "Creating Certificate Stores" later in this README). -2. When creating/configuring a certificate store in Keyfactor Command, you will see a "Change Credentials" link after entering in the destination client machine (IP or DNS). This link **must** be clicked on to present the credentials dialog. However, it is not required that you enter separate credentials. Simply click SAVE in the resulting dialog without entering in credentials to use the credentials that the Keyfactor Orchestrator Service is running under. Alternatively, you may enter separate credentials into this dialog and use those to connect to the orchestrated server. - Please consult with your company's system administrator for more information on configuring SSH/SFTP/SCP or WinRM in your environment. - -**SSH Key-Based Authentiation** -When creating a Keyfactor certificate store for the remote file orchestrator extension (see "Creating Certificate Stores" later in this README, you may supply either a user id and password for the certificate store credentials (directly or through one of Keyfactor Command's PAM integrations), or a user id and SSH private key. Both PKCS#1 (BEGIN RSA PRIVATE KEY) and PKCS#8 (BEGIN PRIVATE KEY) formats are supported for the SSH private key. If using the normal Keyfactor Command credentials dialog without PAM integration, just copy and paste the full SSH private key into the Password textbox.     ## Remote File Orchestrator Extension Installation @@ -355,11 +363,13 @@ CURL script to automate certificate store type creation can be found [here](Cert   ## Creating Certificate Stores and Scheduling Discovery Jobs -Please refer to the Keyfactor Command Reference Guide for information on creating certificate stores and scheduling Discovery jobs in Keyfactor Command. However, there are a few fields that are important to highlight here - Client Machine, Store Path (Creating Certificate Stores), and Directories to search (Discovery jobs) and Extensions (Discovery jobs). For Linux orchestrated servers, "Client Machine" should be the DNS or IP address of the remote orchestrated server while "Store Path" is the full path and file name of the file based store, beginning with a forward slash (/). For Windows orchestrated servers, "Client Machine" should be of the format {protocol}://{dns-or-ip}:{port} where {protocol} is either http or https, {dns-or-ip} is the DNS or IP address of the remote orchestrated server, and {port} is the port where WinRM is listening, by convention usually 5985 for http and 5986 for https. "Store Path" is the full path and file name of the file based store, beginning with a drive letter (i.e. c:\). For example valid values for Client Machine and Store Path for Linux and Windows managed servers may look something like: +Please refer to the Keyfactor Command Reference Guide for information on creating certificate stores and scheduling Discovery jobs in Keyfactor Command. However, there are a few fields that are important to highlight here - Client Machine, Store Path (Creating Certificate Stores), and Directories to search (Discovery jobs) and Extensions (Discovery jobs). For Linux orchestrated servers, "Client Machine" should be the DNS or IP address of the remote orchestrated server while "Store Path" is the full path and file name of the file based store, beginning with a forward slash (/). For Windows orchestrated servers, "Client Machine" should be of the format {protocol}://{dns-or-ip}:{port} where {protocol} is either http or https, {dns-or-ip} is the DNS or IP address of the remote orchestrated server, and {port} is the port where WinRM is listening, by convention usually 5985 for http and 5986 for https. Alternately, entering the keyword "localhost" for "Client Machine" will point to the server where the orchestrator service is installed and WinRM WILL NOT be required. "Store Path" is the full path and file name of the file based store, beginning with a drive letter (i.e. c:\). For example valid values for Client Machine and Store Path for Linux and Windows managed servers may look something like: Linux: Client Machine - 127.0.0.1 or MyLinuxServerName; Store Path - /home/folder/path/storename.ext Windows: Client Machine - https://My.Server.Domain:5986; Store Path - c:\folder\path\storename.ext +Credentials **must** be entered: a user id and either a password or valid PAM key if the password is stored in a KF Command configured PAM integration. Alternatively, this password can be an SSH private key if connecting to a Linux server using SSH private key authentication. + For "Directories to search", you can chain paths with a comma delimiter as documented in the Keyfactor Command Reference Guide, but there is also a special value that can be used instead - fullscan. Entering fullscan in this field will tell the RemoteFile discovery job to search all available drive letters and recursively search all of them for files matching the other search criteria. For "Extensions", a reserved value of noext will cause the RemoteFile discovery job to search for files that do not have an extension. This value can be chained with other extensions using a comma delimiter. For example, entering pem,jks,noext will cause the RemoteFile discovery job to search for files with extensions of PEM or JKS or files that do not have extensions. diff --git a/RemoteFile/ImplementedStoreTypes/JKS/JksStore.cs b/RemoteFile/ImplementedStoreTypes/JKS/JksStore.cs deleted file mode 100644 index 2877134c..00000000 --- a/RemoteFile/ImplementedStoreTypes/JKS/JksStore.cs +++ /dev/null @@ -1,613 +0,0 @@ -using System; -using System.Collections.Generic; -using System.IO; -using System.Text; - -using Org.BouncyCastle.Asn1; -using Org.BouncyCastle.Asn1.Pkcs; -using Org.BouncyCastle.Asn1.X509; -using Org.BouncyCastle.Crypto; -using Org.BouncyCastle.Crypto.IO; -using Org.BouncyCastle.Pkcs; -using Org.BouncyCastle.Security; -using Org.BouncyCastle.Utilities; -using Org.BouncyCastle.Utilities.Date; -using Org.BouncyCastle.Utilities.IO; -using Org.BouncyCastle.X509; - -using Keyfactor.PKI.X509; - -namespace Keyfactor.Extensions.Orchestrator.RemoteFile.JKS -{ - public class JksStore - { - private static readonly int Magic = unchecked((int)0xFEEDFEED); - - private static readonly AlgorithmIdentifier JksObfuscationAlg = new AlgorithmIdentifier( - new DerObjectIdentifier("1.3.6.1.4.1.42.2.17.1.1"), DerNull.Instance); - - private readonly Dictionary m_certificateEntries = - new Dictionary(StringComparer.OrdinalIgnoreCase); - private readonly Dictionary m_keyEntries = - new Dictionary(StringComparer.OrdinalIgnoreCase); - - public JksStore() - { - } - - /// - public bool Probe(Stream stream) - { - using (var br = new BinaryReader(stream)) - try - { - return Magic == ReadInt32(br); - } - catch (EndOfStreamException) - { - return false; - } - } - - /// - public AsymmetricKeyParameter GetKey(string alias, char[] password) - { - if (alias == null) - throw new ArgumentNullException(nameof(alias)); - if (password == null) - throw new ArgumentNullException(nameof(password)); - - if (!m_keyEntries.TryGetValue(alias, out JksKeyEntry keyEntry)) - return null; - - if (!JksObfuscationAlg.Equals(keyEntry.keyData.EncryptionAlgorithm)) - throw new IOException("unknown encryption algorithm"); - - byte[] encryptedData = keyEntry.keyData.GetEncryptedData(); - - // key length is encryptedData - salt - checksum - int pkcs8Len = encryptedData.Length - 40; - - IDigest digest = DigestUtilities.GetDigest("SHA-1"); - - // key decryption - byte[] keyStream = CalculateKeyStream(digest, password, encryptedData, pkcs8Len); - byte[] pkcs8Key = new byte[pkcs8Len]; - for (int i = 0; i < pkcs8Len; ++i) - { - pkcs8Key[i] = (byte)(encryptedData[20 + i] ^ keyStream[i]); - } - Array.Clear(keyStream, 0, keyStream.Length); - - // integrity check - byte[] checksum = GetKeyChecksum(digest, password, pkcs8Key); - - if (!Arrays.ConstantTimeAreEqual(20, encryptedData, pkcs8Len + 20, checksum, 0)) - throw new IOException("cannot recover key"); - - return PrivateKeyFactory.CreateKey(pkcs8Key); - } - - private byte[] GetKeyChecksum(IDigest digest, char[] password, byte[] pkcs8Key) - { - AddPassword(digest, password); - - return DigestUtilities.DoFinal(digest, pkcs8Key); - } - - private byte[] CalculateKeyStream(IDigest digest, char[] password, byte[] salt, int count) - { - byte[] keyStream = new byte[count]; - byte[] hash = Arrays.CopyOf(salt, 20); - - int index = 0; - while (index < count) - { - AddPassword(digest, password); - - digest.BlockUpdate(hash, 0, hash.Length); - digest.DoFinal(hash, 0); - - int length = System.Math.Min(hash.Length, keyStream.Length - index); - Array.Copy(hash, 0, keyStream, index, length); - index += length; - } - - return keyStream; - } - - public X509Certificate[] GetCertificateChain(string alias) - { - if (m_keyEntries.TryGetValue(alias, out var keyEntry)) - return CloneChain(keyEntry.chain); - - return null; - } - - public X509Certificate GetCertificate(string alias) - { - if (m_certificateEntries.TryGetValue(alias, out var certEntry)) - return certEntry.cert; - - if (m_keyEntries.TryGetValue(alias, out var keyEntry)) - return keyEntry.chain?[0]; - - return null; - } - - public DateTime? GetCreationDate(string alias) - { - if (m_certificateEntries.TryGetValue(alias, out var certEntry)) - return certEntry.date; - - if (m_keyEntries.TryGetValue(alias, out var keyEntry)) - return keyEntry.date; - - return null; - } - - /// - public void SetKeyEntry(string alias, AsymmetricKeyParameter key, char[] password, X509Certificate[] chain) - { - alias = ConvertAlias(alias); - - if (ContainsAlias(alias)) - throw new IOException("alias [" + alias + "] already in use"); - - byte[] pkcs8Key = PrivateKeyInfoFactory.CreatePrivateKeyInfo(key).GetEncoded(); - byte[] protectedKey = new byte[pkcs8Key.Length + 40]; - - SecureRandom rnd = new SecureRandom(); - rnd.NextBytes(protectedKey, 0, 20); - - IDigest digest = DigestUtilities.GetDigest("SHA-1"); - - byte[] checksum = GetKeyChecksum(digest, password, pkcs8Key); - Array.Copy(checksum, 0, protectedKey, 20 + pkcs8Key.Length, 20); - - byte[] keyStream = CalculateKeyStream(digest, password, protectedKey, pkcs8Key.Length); - for (int i = 0; i != keyStream.Length; i++) - { - protectedKey[20 + i] = (byte)(pkcs8Key[i] ^ keyStream[i]); - } - Array.Clear(keyStream, 0, keyStream.Length); - - try - { - var epki = new EncryptedPrivateKeyInfo(JksObfuscationAlg, protectedKey); - m_keyEntries.Add(alias, new JksKeyEntry(DateTime.UtcNow, epki.GetEncoded(), CloneChain(chain))); - } - catch (Exception e) - { - throw new IOException("unable to encode encrypted private key", e); - } - } - - /// - public void SetKeyEntry(string alias, byte[] key, X509Certificate[] chain) - { - alias = ConvertAlias(alias); - - if (ContainsAlias(alias)) - throw new IOException("alias [" + alias + "] already in use"); - - m_keyEntries.Add(alias, new JksKeyEntry(DateTime.UtcNow, key, CloneChain(chain))); - } - - /// - public void SetCertificateEntry(string alias, X509Certificate cert) - { - alias = ConvertAlias(alias); - - if (ContainsAlias(alias)) - throw new IOException("alias [" + alias + "] already in use"); - - m_certificateEntries.Add(alias, new JksTrustedCertEntry(DateTime.UtcNow, cert)); - } - - public void DeleteEntry(string alias) - { - if (!m_keyEntries.Remove(alias)) - { - m_certificateEntries.Remove(alias); - } - } - - public IEnumerable Aliases - { - get - { - var aliases = new HashSet(m_certificateEntries.Keys); - aliases.UnionWith(m_keyEntries.Keys); - // FIXME - //return CollectionUtilities.Proxy(aliases); - return aliases; - } - } - - public bool ContainsAlias(string alias) - { - return IsCertificateEntry(alias) || IsKeyEntry(alias); - } - - public int Count - { - get { return m_certificateEntries.Count + m_keyEntries.Count; } - } - - public bool IsKeyEntry(string alias) - { - return m_keyEntries.ContainsKey(alias); - } - - public bool IsCertificateEntry(string alias) - { - return m_certificateEntries.ContainsKey(alias); - } - - public string GetCertificateAlias(X509Certificate cert) - { - foreach (var entry in m_certificateEntries) - { - if (entry.Value.cert.Equals(cert)) - return entry.Key; - } - return null; - } - - /// - public void Save(Stream stream, char[] password) - { - if (stream == null) - throw new ArgumentNullException(nameof(stream)); - if (password == null) - throw new ArgumentNullException(nameof(password)); - - IDigest checksumDigest = CreateChecksumDigest(password); - BinaryWriter bw = new BinaryWriter(new DigestStream(stream, null, checksumDigest)); - - WriteInt32(bw, Magic); - WriteInt32(bw, 2); - - WriteInt32(bw, Count); - - foreach (var entry in m_keyEntries) - { - string alias = entry.Key; - JksKeyEntry keyEntry = entry.Value; - - WriteInt32(bw, 1); - WriteUtf(bw, alias); - WriteDateTime(bw, keyEntry.date); - WriteBufferWithLength(bw, keyEntry.keyData.GetEncoded()); - - X509Certificate[] chain = keyEntry.chain; - int chainLength = chain == null ? 0 : chain.Length; - WriteInt32(bw, chainLength); - for (int i = 0; i < chainLength; ++i) - { - WriteTypedCertificate(bw, chain[i]); - } - } - - foreach (var entry in m_certificateEntries) - { - string alias = entry.Key; - JksTrustedCertEntry certEntry = entry.Value; - - WriteInt32(bw, 2); - WriteUtf(bw, alias); - WriteDateTime(bw, certEntry.date); - WriteTypedCertificate(bw, certEntry.cert); - } - - byte[] checksum = DigestUtilities.DoFinal(checksumDigest); - bw.Write(checksum); - bw.Flush(); - } - - /// - public void Load(Stream stream, char[] password) - { - if (stream == null) - throw new ArgumentNullException(nameof(stream)); - - m_certificateEntries.Clear(); - m_keyEntries.Clear(); - - ErasableByteStream storeStream = ValidateStream(stream, password); - try - { - BinaryReader dIn = new BinaryReader(storeStream); - - int magic = ReadInt32(dIn); - int storeVersion = ReadInt32(dIn); - - if (!(magic == Magic && (storeVersion == 1 || storeVersion == 2))) - throw new IOException("Invalid keystore format"); - - int numEntries = ReadInt32(dIn); - - for (int t = 0; t < numEntries; t++) - { - int tag = ReadInt32(dIn); - - switch (tag) - { - case 1: // keys - { - string alias = ReadUtf(dIn); - DateTime date = ReadDateTime(dIn); - - // encrypted key data - byte[] keyData = ReadBufferWithLength(dIn); - - // certificate chain - int chainLength = ReadInt32(dIn); - X509Certificate[] chain = null; - if (chainLength > 0) - { - var certs = new List(System.Math.Min(10, chainLength)); - for (int certNo = 0; certNo != chainLength; certNo++) - { - certs.Add(ReadTypedCertificate(dIn, storeVersion)); - } - chain = certs.ToArray(); - } - m_keyEntries.Add(alias, new JksKeyEntry(date, keyData, chain)); - break; - } - case 2: // certificate - { - string alias = ReadUtf(dIn); - DateTime date = ReadDateTime(dIn); - - X509Certificate cert = ReadTypedCertificate(dIn, storeVersion); - - m_certificateEntries.Add(alias, new JksTrustedCertEntry(date, cert)); - break; - } - default: - throw new IOException("unable to discern entry type"); - } - } - - if (storeStream.Position != storeStream.Length) - throw new IOException("password incorrect or store tampered with"); - } - finally - { - storeStream.Erase(); - } - } - - /* - * Validate password takes the checksum of the store and will either. - * 1. If password is null, load the store into memory, return the result. - * 2. If password is not null, load the store into memory, test the checksum, and if successful return - * a new input stream instance of the store. - * 3. Fail if there is a password and an invalid checksum. - * - * @param inputStream The input stream. - * @param password the password. - * @return Either the passed in input stream or a new input stream. - */ - /// - private ErasableByteStream ValidateStream(Stream inputStream, char[] password) - { - byte[] rawStore = Streams.ReadAll(inputStream); - int checksumPos = rawStore.Length - 20; - - if (password != null) - { - byte[] checksum = CalculateChecksum(password, rawStore, 0, checksumPos); - - if (!Arrays.ConstantTimeAreEqual(20, checksum, 0, rawStore, checksumPos)) - { - Array.Clear(rawStore, 0, rawStore.Length); - throw new IOException("password incorrect or store tampered with"); - } - } - - return new ErasableByteStream(rawStore, 0, checksumPos); - } - - private static void AddPassword(IDigest digest, char[] password) - { - // Encoding.BigEndianUnicode - for (int i = 0; i < password.Length; ++i) - { - digest.Update((byte)(password[i] >> 8)); - digest.Update((byte)password[i]); - } - } - - private static byte[] CalculateChecksum(char[] password, byte[] buffer, int offset, int length) - { - IDigest checksumDigest = CreateChecksumDigest(password); - checksumDigest.BlockUpdate(buffer, offset, length); - return DigestUtilities.DoFinal(checksumDigest); - } - - private static X509Certificate[] CloneChain(X509Certificate[] chain) - { - return (X509Certificate[])chain?.Clone(); - } - - private static string ConvertAlias(string alias) - { - return alias.ToLowerInvariant(); - } - - private static IDigest CreateChecksumDigest(char[] password) - { - IDigest digest = DigestUtilities.GetDigest("SHA-1"); - AddPassword(digest, password); - - // - // This "Mighty Aphrodite" string goes all the way back to the - // first java betas in the mid 90's, why who knows? But see - // https://cryptosense.com/mighty-aphrodite-dark-secrets-of-the-java-keystore/ - // - byte[] prefix = Encoding.UTF8.GetBytes("Mighty Aphrodite"); - digest.BlockUpdate(prefix, 0, prefix.Length); - return digest; - } - - private static byte[] ReadBufferWithLength(BinaryReader br) - { - int length = ReadInt32(br); - return br.ReadBytes(length); - } - - private static DateTime ReadDateTime(BinaryReader br) - { - DateTime unixMs = DateTimeUtilities.UnixMsToDateTime(Longs.ReverseBytes(br.ReadInt64())); - DateTime utc = new DateTime(unixMs.Ticks, DateTimeKind.Utc); - return utc; - } - - private static short ReadInt16(BinaryReader br) - { - short n = br.ReadInt16(); - n = (short)(((n & 0xFF) << 8) | ((n >> 8) & 0xFF)); - return n; - } - - private static int ReadInt32(BinaryReader br) - { - return Integers.ReverseBytes(br.ReadInt32()); - } - - private static X509Certificate ReadTypedCertificate(BinaryReader br, int storeVersion) - { - if (storeVersion == 2) - { - string certFormat = ReadUtf(br); - if ("X.509" != certFormat) - throw new IOException("Unsupported certificate format: " + certFormat); - } - - byte[] certData = ReadBufferWithLength(br); - try - { - System.Security.Cryptography.X509Certificates.X509Certificate2 cert = new System.Security.Cryptography.X509Certificates.X509Certificate2(certData); - return DotNetUtilities.FromX509Certificate(cert); - } - finally - { - Array.Clear(certData, 0, certData.Length); - } - } - - private static string ReadUtf(BinaryReader br) - { - short length = ReadInt16(br); - byte[] utfBytes = br.ReadBytes(length); - - /* - * FIXME JKS actually uses a "modified UTF-8" format. For the moment we will just support single-byte - * encodings that aren't null bytes. - */ - for (int i = 0; i < utfBytes.Length; ++i) - { - byte utfByte = utfBytes[i]; - if (utfByte == 0 || (utfByte & 0x80) != 0) - throw new NotSupportedException("Currently missing support for modified UTF-8 encoding in JKS"); - } - - return Encoding.UTF8.GetString(utfBytes); - } - - private static void WriteBufferWithLength(BinaryWriter bw, byte[] buffer) - { - WriteInt32(bw, buffer.Length); - bw.Write(buffer); - } - - private static void WriteDateTime(BinaryWriter bw, DateTime dateTime) - { - bw.Write(Longs.ReverseBytes(DateTimeUtilities.DateTimeToUnixMs(dateTime.ToUniversalTime()))); - } - - private static void WriteInt16(BinaryWriter bw, short n) - { - n = (short)(((n & 0xFF) << 8) | ((n >> 8) & 0xFF)); - bw.Write(n); - } - - private static void WriteInt32(BinaryWriter bw, int n) - { - bw.Write(Integers.ReverseBytes(n)); - } - - private static void WriteTypedCertificate(BinaryWriter bw, X509Certificate cert) - { - WriteUtf(bw, "X.509"); - WriteBufferWithLength(bw, cert.GetEncoded()); - } - - private static void WriteUtf(BinaryWriter bw, string s) - { - byte[] utfBytes = Encoding.UTF8.GetBytes(s); - - /* - * FIXME JKS actually uses a "modified UTF-8" format. For the moment we will just support single-byte - * encodings that aren't null bytes. - */ - for (int i = 0; i < utfBytes.Length; ++i) - { - byte utfByte = utfBytes[i]; - if (utfByte == 0 || (utfByte & 0x80) != 0) - throw new NotSupportedException("Currently missing support for modified UTF-8 encoding in JKS"); - } - - WriteInt16(bw, Convert.ToInt16(utfBytes.Length)); - bw.Write(utfBytes); - } - - /** - * JksTrustedCertEntry is a internal container for the certificate entry. - */ - private sealed class JksTrustedCertEntry - { - internal readonly DateTime date; - internal readonly X509Certificate cert; - - internal JksTrustedCertEntry(DateTime date, X509Certificate cert) - { - this.date = date; - this.cert = cert; - } - } - - private sealed class JksKeyEntry - { - internal readonly DateTime date; - internal readonly EncryptedPrivateKeyInfo keyData; - internal readonly X509Certificate[] chain; - - internal JksKeyEntry(DateTime date, byte[] keyData, X509Certificate[] chain) - { - this.date = date; - this.keyData = EncryptedPrivateKeyInfo.GetInstance(Asn1Sequence.GetInstance(keyData)); - this.chain = chain; - } - } - - private sealed class ErasableByteStream - : MemoryStream - { - internal ErasableByteStream(byte[] buffer, int index, int count) - : base(buffer, index, count, true) - { - } - - internal void Erase() - { - Position = 0L; - Streams.WriteZeroes(this, Convert.ToInt32(Length)); - } - } - } -} diff --git a/RemoteFile/RemoteHandlers/SSHHandler.cs b/RemoteFile/RemoteHandlers/SSHHandler.cs index 079e02fd..72b53f5f 100644 --- a/RemoteFile/RemoteHandlers/SSHHandler.cs +++ b/RemoteFile/RemoteHandlers/SSHHandler.cs @@ -211,7 +211,8 @@ public override void UploadCertificateFile(string path, string fileName, byte[] if (!string.IsNullOrEmpty(ApplicationSettings.SeparateUploadFilePath)) { - RunCommand($"cp -a {uploadPath} {path}", null, ApplicationSettings.UseSudo, null); + //RunCommand($"cat {uploadPath} > {path}/{fileName}", null, ApplicationSettings.UseSudo, null); + RunCommand($"tee {path}/{fileName} < {uploadPath} > /dev/null", null, ApplicationSettings.UseSudo, null); RunCommand($"rm {uploadPath}", null, ApplicationSettings.UseSudo, null); } diff --git a/RemoteFile/RemoteHandlers/WinRMHandler.cs b/RemoteFile/RemoteHandlers/WinRMHandler.cs index 122fb579..c197f125 100644 --- a/RemoteFile/RemoteHandlers/WinRMHandler.cs +++ b/RemoteFile/RemoteHandlers/WinRMHandler.cs @@ -8,6 +8,7 @@ using System; using System.Collections.Generic; using System.Management.Automation; +using System.Management.Automation.Remoting; using System.Management.Automation.Runspaces; using System.Net; using System.Text; @@ -31,10 +32,13 @@ internal WinRMHandler(string server, string serverLogin, string serverPassword) _logger.MethodEntry(LogLevel.Debug); Server = server; - connectionInfo = new WSManConnectionInfo(new System.Uri($"{Server}/wsman")); - if (!string.IsNullOrEmpty(serverLogin)) + if (Server.ToLower() != "localhost") { - connectionInfo.Credential = new PSCredential(serverLogin, new NetworkCredential(serverLogin, serverPassword).SecurePassword); + connectionInfo = new WSManConnectionInfo(new System.Uri($"{Server}/wsman")); + if (!string.IsNullOrEmpty(serverLogin)) + { + connectionInfo.Credential = new PSCredential(serverLogin, new NetworkCredential(serverLogin, serverPassword).SecurePassword); + } } _logger.MethodExit(LogLevel.Debug); @@ -46,11 +50,18 @@ public override void Initialize() try { - if (ApplicationSettings.UseNegotiate) + if (Server.ToLower() == "localhost") + { + runspace = RunspaceFactory.CreateRunspace(); + } + else { - connectionInfo.AuthenticationMechanism = AuthenticationMechanism.Negotiate; + if (ApplicationSettings.UseNegotiate) + { + connectionInfo.AuthenticationMechanism = AuthenticationMechanism.Negotiate; + } + runspace = RunspaceFactory.CreateRunspace(connectionInfo); } - runspace = RunspaceFactory.CreateRunspace(connectionInfo); runspace.Open(); } @@ -83,12 +94,6 @@ public override string RunCommand(string commandText, object[] parameters, bool using (PowerShell ps = PowerShell.Create()) { ps.Runspace = runspace; - - if (commandText.ToLower().IndexOf("keytool ") > -1) - { - commandText = ($"& '{commandText}").Replace("keytool", "keytool'"); - commandText = "echo '' | " + commandText; - } ps.AddScript(commandText); string displayCommand = commandText; @@ -100,7 +105,7 @@ public override string RunCommand(string commandText, object[] parameters, bool if (parameters != null) { - foreach(object parameter in parameters) + foreach (object parameter in parameters) ps.AddArgument(parameter); } @@ -120,7 +125,7 @@ public override string RunCommand(string commandText, object[] parameters, bool errors = null; break; } - + errors += (error + " "); } @@ -192,10 +197,12 @@ public override void UploadCertificateFile(string path, string fileName, byte[] _logger.MethodEntry(LogLevel.Debug); _logger.LogDebug($"UploadCertificateFile: {path} {fileName}"); + string cmdOption = RunCommand($@"$PSVersionTable.PSEdition", null, false, null).ToLower().Contains("core") ? "AsByteStream" : "Encoding Byte"; + string scriptBlock = $@" param($contents) - Set-Content {path + fileName} -Encoding Byte -Value $contents + Set-Content ""{path + fileName}"" -{cmdOption} -Value $contents "; object[] arguments = new object[] { certBytes }; @@ -211,7 +218,9 @@ public override byte[] DownloadCertificateFile(string path) _logger.LogDebug($"DownloadCertificateFile: {path}"); _logger.MethodExit(LogLevel.Debug); - return RunCommandBinary($@"Get-Content -Path ""{path}"" -Encoding Byte -Raw"); + string cmdOption = RunCommand($@"$PSVersionTable.PSEdition", null, false, null).ToLower().Contains("core") ? "AsByteStream" : "Encoding Byte"; + + return RunCommandBinary($@"Get-Content -Path ""{path}"" -{cmdOption} -Raw"); } public override void CreateEmptyStoreFile(string path, string linuxFilePermissions, string linuxFileOwner) diff --git a/integration-manifest.json b/integration-manifest.json index 51c3446a..9152294f 100644 --- a/integration-manifest.json +++ b/integration-manifest.json @@ -27,6 +27,253 @@ "supportsReenrollment": false, "supportsInventory": true, "platformSupport": "Unused" + }, + "store_types": { + "RFJKS": { + "Name": "RFJKS", + "ShortName": "RFJKS", + "Capability": "RFJKS", + "ServerRequired": true, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Required", + "PowerShell": false, + "PrivateKeyAllowed": "Optional", + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "PasswordOptions": { + "Style": "Default", + "EntrySupported": false, + "StoreRequired": true + }, + "Properties": [ + { + "Name": "LinuxFilePermissionsOnStoreCreation", + "DisplayName": "Linux File Permissions on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "LinuxFileOwnerOnStoreCreation", + "DisplayName": "Linux File Owner on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + } + ], + "EntryParameters": [] + }, + "RFPEM": { + "Name": "RFPEM", + "ShortName": "RFPEM", + "Capability": "RFPEM", + "ServerRequired": true, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Forbidden", + "PowerShell": false, + "PrivateKeyAllowed": "Optional", + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "PasswordOptions": { + "Style": "Default", + "EntrySupported": false, + "StoreRequired": true + }, + "Properties": [ + { + "Name": "LinuxFilePermissionsOnStoreCreation", + "DisplayName": "Linux File Permissions on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "LinuxFileOwnerOnStoreCreation", + "DisplayName": "Linux File Owner on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "IsTrustStore", + "DisplayName": "Trust Store", + "Required": false, + "DependsOn": "", + "Type": "Bool", + "DefaultValue": false + }, + { + "Name": "IncludesChain", + "DisplayName": "Store Includes Chain", + "Required": false, + "DependsOn": "", + "Type": "Bool", + "DefaultValue": false + }, + { + "Name": "SeparatePrivateKeyFilePath", + "DisplayName": "Separate Private Key File Location", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "IsRSAPrivateKey", + "DisplayName": "Is RSA Private Key", + "Required": false, + "DependsOn": "", + "Type": "Bool", + "DefaultValue": false + } + ], + "EntryParameters": [] + }, + "RFPkcs12": { + "Name": "RFPkcs12", + "ShortName": "RFPkcs12", + "Capability": "RFPkcs12", + "ServerRequired": true, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Required", + "PowerShell": false, + "PrivateKeyAllowed": "Optional", + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "PasswordOptions": { + "Style": "Default", + "EntrySupported": false, + "StoreRequired": true + }, + "Properties": [ + { + "Name": "LinuxFilePermissionsOnStoreCreation", + "DisplayName": "Linux File Permissions on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "LinuxFileOwnerOnStoreCreation", + "DisplayName": "Linux File Owner on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + } + ], + "EntryParameters": [] + }, + "RFDER": { + "Name": "RFDER", + "ShortName": "RFDER", + "Capability": "RFDER", + "ServerRequired": true, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Forbidden", + "PowerShell": false, + "PrivateKeyAllowed": "Optional", + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "PasswordOptions": { + "Style": "Default", + "EntrySupported": false, + "StoreRequired": true + }, + "Properties": [ + { + "Name": "LinuxFilePermissionsOnStoreCreation", + "DisplayName": "Linux File Permissions on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "LinuxFileOwnerOnStoreCreation", + "DisplayName": "Linux File Owner on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "SeparatePrivateKeyFilePath", + "DisplayName": "Separate Private Key File Location", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + } + ], + "EntryParameters": [] + }, + "RFKDB": { + "Name": "RFKDB", + "ShortName": "RFKDB", + "Capability": "RFKDB", + "ServerRequired": true, + "BlueprintAllowed": false, + "CustomAliasAllowed": "Required", + "PowerShell": false, + "PrivateKeyAllowed": "Optional", + "SupportedOperations": { + "Add": true, + "Create": true, + "Discovery": true, + "Enrollment": false, + "Remove": true + }, + "PasswordOptions": { + "Style": "Default", + "EntrySupported": false, + "StoreRequired": true + }, + "Properties": [ + { + "Name": "LinuxFilePermissionsOnStoreCreation", + "DisplayName": "Linux File Permissions on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + }, + { + "Name": "LinuxFileOwnerOnStoreCreation", + "DisplayName": "Linux File Owner on Store Creation", + "Required": false, + "DependsOn": "", + "Type": "String", + "DefaultValue": "" + } + ], + "EntryParameters": [] + } } } } diff --git a/readme_source.md b/readme_source.md index 7cefb029..37caacfa 100644 --- a/readme_source.md +++ b/readme_source.md @@ -30,22 +30,19 @@ The version number of a the Remote File Orchestrator Extension can be verified b 1. The Remote File Orchestrator Extension makes use of a few common Linux commands when managing stores on Linux servers. If the credentials you will be connecting with need elevated access to run these commands, you must set up the user id as a sudoer with no password necessary and set the config.json "UseSudo" value to "Y" (See "Config File Setup" later in this README for more information on setting up the config.json file). The full list of these commands below: * echo * find - * cp + * tee * rm * chown * install 2. The Remote File Orchestrator Extension makes use of SFTP and/or SCP to transfer files to and from the orchestrated server. SFTP/SCP cannot make use of sudo, so all folders containing certificate stores will need to allow SFTP/SCP file transfer. If this is not possible, set the values in the config.json apprpriately to use an alternative upload/download folder that does allow SFTP/SCP file transfer (See "Config File Setup" later in this README regarding the config.json file). +3. SSH Key Authentication: When creating a Keyfactor certificate store for the remote file orchestrator extension (see "Creating Certificate Stores" later in this README, you may supply either a user id and password for the certificate store credentials (directly or through one of Keyfactor Command's PAM integrations), or a user id and SSH private key. Both PKCS#1 (BEGIN RSA PRIVATE KEY) and PKCS#8 (BEGIN PRIVATE KEY) formats are supported for the SSH private key. If using the normal Keyfactor Command credentials dialog without PAM integration, just copy and paste the full SSH private key into the Password textbox. + **For Windows orchestrated servers:** 1. Make sure that WinRM is set up on the orchestrated server and that the WinRM port is part of the certificate store path when setting up your certificate stores When creating a new certificate store in Keyfactor Command (See "Creating Certificate Stores" later in this README). -2. When creating/configuring a certificate store in Keyfactor Command, you will see a "Change Credentials" link after entering in the destination client machine (IP or DNS). This link **must** be clicked on to present the credentials dialog. However, it is not required that you enter separate credentials. Simply click SAVE in the resulting dialog without entering in credentials to use the credentials that the Keyfactor Orchestrator Service is running under. Alternatively, you may enter separate credentials into this dialog and use those to connect to the orchestrated server. - Please consult with your company's system administrator for more information on configuring SSH/SFTP/SCP or WinRM in your environment. - -**SSH Key-Based Authentiation** -When creating a Keyfactor certificate store for the remote file orchestrator extension (see "Creating Certificate Stores" later in this README, you may supply either a user id and password for the certificate store credentials (directly or through one of Keyfactor Command's PAM integrations), or a user id and SSH private key. Both PKCS#1 (BEGIN RSA PRIVATE KEY) and PKCS#8 (BEGIN PRIVATE KEY) formats are supported for the SSH private key. If using the normal Keyfactor Command credentials dialog without PAM integration, just copy and paste the full SSH private key into the Password textbox.     ## Remote File Orchestrator Extension Installation @@ -268,11 +265,13 @@ CURL script to automate certificate store type creation can be found [here](Cert   ## Creating Certificate Stores and Scheduling Discovery Jobs -Please refer to the Keyfactor Command Reference Guide for information on creating certificate stores and scheduling Discovery jobs in Keyfactor Command. However, there are a few fields that are important to highlight here - Client Machine, Store Path (Creating Certificate Stores), and Directories to search (Discovery jobs) and Extensions (Discovery jobs). For Linux orchestrated servers, "Client Machine" should be the DNS or IP address of the remote orchestrated server while "Store Path" is the full path and file name of the file based store, beginning with a forward slash (/). For Windows orchestrated servers, "Client Machine" should be of the format {protocol}://{dns-or-ip}:{port} where {protocol} is either http or https, {dns-or-ip} is the DNS or IP address of the remote orchestrated server, and {port} is the port where WinRM is listening, by convention usually 5985 for http and 5986 for https. "Store Path" is the full path and file name of the file based store, beginning with a drive letter (i.e. c:\). For example valid values for Client Machine and Store Path for Linux and Windows managed servers may look something like: +Please refer to the Keyfactor Command Reference Guide for information on creating certificate stores and scheduling Discovery jobs in Keyfactor Command. However, there are a few fields that are important to highlight here - Client Machine, Store Path (Creating Certificate Stores), and Directories to search (Discovery jobs) and Extensions (Discovery jobs). For Linux orchestrated servers, "Client Machine" should be the DNS or IP address of the remote orchestrated server while "Store Path" is the full path and file name of the file based store, beginning with a forward slash (/). For Windows orchestrated servers, "Client Machine" should be of the format {protocol}://{dns-or-ip}:{port} where {protocol} is either http or https, {dns-or-ip} is the DNS or IP address of the remote orchestrated server, and {port} is the port where WinRM is listening, by convention usually 5985 for http and 5986 for https. Alternately, entering the keyword "localhost" for "Client Machine" will point to the server where the orchestrator service is installed and WinRM WILL NOT be required. "Store Path" is the full path and file name of the file based store, beginning with a drive letter (i.e. c:\). For example valid values for Client Machine and Store Path for Linux and Windows managed servers may look something like: Linux: Client Machine - 127.0.0.1 or MyLinuxServerName; Store Path - /home/folder/path/storename.ext Windows: Client Machine - https://My.Server.Domain:5986; Store Path - c:\folder\path\storename.ext +Credentials **must** be entered: a user id and either a password or valid PAM key if the password is stored in a KF Command configured PAM integration. Alternatively, this password can be an SSH private key if connecting to a Linux server using SSH private key authentication. + For "Directories to search", you can chain paths with a comma delimiter as documented in the Keyfactor Command Reference Guide, but there is also a special value that can be used instead - fullscan. Entering fullscan in this field will tell the RemoteFile discovery job to search all available drive letters and recursively search all of them for files matching the other search criteria. For "Extensions", a reserved value of noext will cause the RemoteFile discovery job to search for files that do not have an extension. This value can be chained with other extensions using a comma delimiter. For example, entering pem,jks,noext will cause the RemoteFile discovery job to search for files with extensions of PEM or JKS or files that do not have extensions.