Exploring possible interoperability between step-ca and SHSM

[Chewhern]

Hi Smallstep community,

I'm an open-source developer working on SHSM (Software-emulated Hardware Security Module), a self-hosted software service for performing cryptographic operations while providing more explicit control over sensitive key material and its memory lifecycle.

Repository: https://github.com/Chewhern/SHSM

I'm interested in whether SHSM could potentially be useful as another cryptographic-protection option for step-ca, particularly for deployments where users want to isolate cryptographic operations from the main application without relying on dedicated HSM hardware or a cloud KMS.

SHSM currently provides:

• An HTTP API for cryptographic operations
• A cross-platform client
• An initial command-line interface (CLI)
• Public-key and secret-key cryptographic operations
• Key lifecycle and permission controls
• Protected-memory handling using libsodium
• Memory permission controls and zeroization
• Self-hosted deployment
• Documentation and integration examples

One of the problems SHSM is designed to explore is sensitive key material and application memory management.

Applications written in managed or garbage-collected languages can have less deterministic control over the lifetime and destruction of sensitive data. Immutable strings and managed objects can also make explicit erasure difficult. SHSM takes a different approach by keeping the cryptographic key operations inside a separate process and exposing operations through an API.

This made me wonder whether there could be an interoperability point with step-ca.

Smallstep already supports several mechanisms for protecting CA signing keys, including PKCS#11 HSMs, cloud KMS providers, YubiKey PIV, and TPM-based protection. I'm wondering whether a self-hosted software HSM service could be useful as another option for environments that do not have access to those mechanisms.

I am not proposing that SHSM should replace hardware HSMs or KMSs, and I'm not claiming that it provides equivalent security guarantees. The goal is to explore whether the software-based approach is useful for particular deployment scenarios.

For example, a possible architecture might look conceptually like:

step-ca → SHSM → protected key operations

where step-ca requests signing or other cryptographic operations from SHSM without directly managing the private key in its own application process.

I'm particularly interested in feedback on:

1. Whether this architecture would make technical sense for step-ca.
2. Whether the existing step-kms-plugin architecture would already provide an appropriate extension point for something like SHSM.
3. Whether there are deployment scenarios where a self-hosted software HSM would be useful despite the availability of PKCS#11, TPM, and cloud KMS integrations.
4. Whether the memory-management issue described above is a meaningful concern for step-ca or its users.
5. If there is a sensible integration point, what interface or architecture would be preferable.

SHSM is currently in Alpha, so I'm primarily looking for technical feedback before investing further in a specific integration direction.

Thanks for taking the time to read this. I'd be very interested in hearing how the Smallstep maintainers and community would approach this problem.

[ECD5A]

The architecture makes sense as a remote signer, provided the deployment gives the signing service a meaningful isolation boundary. There is also a distinction between the CLI plugin and the server integration that would affect where you start.

For step-ca, I'd start with kms/apiv1.KeyManager. Its CreateSigner method returns a Go crypto.Signer. In the CA initialization path, step-ca creates the key manager and obtains the intermediate's signer through that interface. An adapter could return the public key locally and send signing requests to SHSM, keeping the private key on the service side.

step-kms-plugin is a tool/plugin for the step CLI. Adding support there alone would not wire the backend into the running CA. A custom backend would also need to be registered and included in the step-ca build; I'd confirm the preferred upstream integration route with the maintainers before investing in both.

For an initial prototype, I'd pick one key type and an existing intermediate key. The critical compatibility check is the crypto.Signer contract: whether the input is a digest or a message, the requested hash/signature options, and the returned signature encoding. An HTTP signing endpoint isn't automatically compatible. A useful first test would issue a certificate through the adapter and verify it independently against the intermediate, then check that service unavailability and denied authorization return errors without a local-key fallback.

On the security side, I'd describe two separate properties:

• Keeping the key out of the CA process can reduce exposure through that process's memory or crash dumps. Protected memory and zeroization can help inside SHSM, but the benefit depends on process privileges, debugging access, swap/dump handling, and whether both services share an administrator or host.
• A compromised CA may still use its authorized signing connection. A service receiving only a digest cannot inspect certificate names or extensions and enforce issuance policy over them. Protecting against unauthorized issuance needs an additional policy boundary with enough context to make that decision.

That suggests a practical deployment to evaluate: a separately administered signing service, authenticated transport, credentials restricted to one signing key, and separate permissions for key export, deletion, and administration. I'd measure the isolation and operational cost of that setup before making broader claims about managed-memory safety. These are integration suggestions based on the Smallstep interfaces; I haven't tested SHSM's implementation.

[tashian]

@Chewhern thanks for sharing!

One thing that comes to mind for me here is that an SHSM could take advantage of a secure enclave (or TEE). Both for key and memory isolation, but also for binary attestations. I haven't explored this area very much, but the idea is that only a trusted binary can run inside the enclave and there are very tightly scoped channels for communication between the main processor and the enclave. This gives the operator more confidence that the private key material can't be accessed by a third party.

Downside of this approach is that, last I heard, there's a lot of different standards for enclaves that you'd have to deal with, and there's not much in terms of an abstraction layer on top of them. So, it'd be a lot of work to create something functional.

[Chewhern]

Thank you both for the detailed feedback — this is exactly the kind of technical direction I was hoping for.

@tashian Your point about secure enclaves / TEE is well taken. The value you describe — key and memory isolation, binary attestation, tightly scoped communication channels — aligns directly with what SHSM is trying to explore. The fragmentation problem you mention (multiple enclave standards, no common abstraction layer) is real, and I think it's actually part of why a service-level approach like SHSM could be useful: instead of requiring step-ca to integrate with each TEE vendor's SDK separately, SHSM could expose a single HTTP API and handle the TEE-specific backend internally, where available.

To be clear about the current state: SHSM is pure-software today, with protected memory and zeroization via libsodium. TEE integration is a direction I'm exploring, not something implemented yet. The architecture I have in mind is that SHSM could optionally run inside a TEE (Intel SGX, AMD SEV, etc.), gaining hardware-level isolation and attestation without changing the client-facing API.

On the first reply's point about the policy boundary: I agree that a signing service receiving only a digest cannot enforce issuance policy. TEE attestation doesn't solve that by itself — it solves the isolation question, not the authorization question. A complete design would need both: TEE-backed isolation for the key, and a policy layer with enough context to decide whether a given signing request should be honored.

I'll look into the kms/apiv1.KeyManager interface and the crypto.Signer contract as suggested. If I pursue a prototype, I'll start with one key type and a separately administered signing service, and measure the isolation properties before making broader claims.

Thanks again for the concrete pointers — this gives me a much clearer path to evaluate.