Skip to content

Encryption

Jump to bottom
Antonio Membrides Espinosa edited this page Jul 22, 2026 · 6 revisions

🔐 Encryption — Queryable Encryption, DEKs & Key Management

Status: Implemented (v2) PCI DSS coverage: Req 3 (encryption in use / at rest), Req 3.5–3.6 (key management), Req 4 (in transit), Req 7 (least-privilege access)

This page explains how cardholder and personal data are encrypted in the demo: how the Queryable Encryption (QE) fields are configured, how Data Encryption Keys (DEKs) and the master key (CMK) are managed, and how the cardholder-data (CHD) envelope works on the event bus.

1. Two complementary mechanisms

The platform uses two distinct crypto systems that share the same master key surface:

Mechanism Protects Where Managed by
Queryable Encryption (QE) PII & sensitive fields at rest in MongoDB (email, phone, account reference, address, date of birth, government ID, gateway payload, card expiry, and bank data: payout IBAN/routing and destination IBAN) MongoDB collections The MongoDB driver (automatic encryption)
CHD envelope Cardholder data (PAN / CVV / expiry) in transit on the internal event bus The opaque chd token on bus events The application (AES-256-GCM, node:crypto)

The CVV is never stored in MongoDB and is never placed in QE. It crosses the bus encrypted just-in-time on the way to the card issuer, and the built-in issuer also derives it per card on demand (HMAC-SHA256 under the issuer key, see §13) rather than holding a single global value (PCI DSS Req 3.2: no sensitive authentication data stored). The full PAN is not stored by the PSP core either; when the built-in card-issuer module is active it stores the full PAN encrypted with QE:equality in its own module-owned vault (cardIssuerVault, see §13), the issuer CDE. The core keeps only the token + BIN + last4.

2. Key hierarchy

Both mechanisms follow the standard envelope-encryption model — a three-level key hierarchy:

CMK / Master Key            ← lives in the KMS; never encrypts data directly
   │  (wraps / unwraps)
   └─ DEK  (Data Encryption Key)   ← stored encrypted in the key vault
        │  (encrypts / decrypts)
        └─ field value / message   ← the actual ciphertext in the document or on the bus
  • The CMK never leaves the KMS (for AWS) — the driver asks KMS to wrap/unwrap DEKs.
  • Each DEK is stored already-encrypted (wrapped by the CMK) in the key vault collection.
  • The DEK is what actually encrypts a field value; compromising a DEK ciphertext is useless without the CMK.

3. The master key (CMK) — KMS_PROVIDER

Source: backend/src/vendors/encryption/kms.ts

The KMS provider is chosen by the PSP_KMS_PROVIDER environment variable (a legacy KMS_PROVIDER is still read as a fallback):

Local (offline demo — default)

PSP_KMS_PROVIDER=local
PSP_KMS_LOCAL_MASTER_KEY=<96-byte base64 key>

A 96-byte local master key (the size MongoDB QE requires for the local provider). Generate it with:

npm run setup:key:master   # → backend/bin/seed-generate-key.ts → randomBytes(96).toString('base64')

AWS KMS (production)

PSP_KMS_PROVIDER=aws
AWS_CMK_ARN=arn:aws:kms:...:key/...
AWS_REGION=...
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
# AWS_SESSION_TOKEN=...   # optional (temporary credentials)

With AWS the customer controls the CMK; MongoDB has zero access to it.

buildKmsProviders() returns the kmsProviders object passed to the driver; buildCmkOptions() returns the masterKey reference (AWS only — local needs none).

4. Data Encryption Keys (DEKs)

Source: backend/src/vendors/encryption/keyVault.ts

  • One DEK per encrypted field. MongoDB QE forbids sharing a DEK between fields of the same collection (error 6338401).
  • DEKs are stored in the key vault: encryption.__keyVault, each one wrapped by the CMK.
  • DEKs are created with ClientEncryption.createDataKey(provider, { masterKey, keyAltNames: ['DEK-...'] }) and looked up by their keyAltNames alias — so provisioning is idempotent (re-running setup reuses existing keys, never duplicates them).

The DEKs are organised into two access tiers (16 DEKs total: 5 lookup + 11 sensitive):

Tier DEK aliases Backs Audience
Lookup (QE:equality) DEK-tx-account-ref, DEK-party-email, DEK-party-phone, DEK-customer-account-ref, DEK-auth-email searchable encrypted fields All authenticated analyst roles (L1+)
Sensitive (QE:none) DEK-tx-raw-payload, DEK-tx-processor-meta, DEK-customer-address, DEK-customer-gov-id, DEK-customer-risk-notes, DEK-card-expiry, DEK-party-address, DEK-party-dob, DEK-payout-iban, DEK-payout-routing, DEK-exec-dest-iban retrieval-only sensitive fields (high-sensitivity PII, CHD, and GDPR/PSD2 bank data) L2 Investigator (with escalation token) + Security Auditor

Provisioning (backend/src/vendors/setup/provisionDEKs.ts) first creates a unique index on keyAltNames in the vault, then calls provisionDataEncryptionKeys() which getOrCreates each named DEK.

5. QE field configuration

Source: backend/src/vendors/encryption/encryptedFieldsMaps.ts

Each encrypted collection declares an encryptedFields.fields[] map. Every field specifies the DEK that protects it, its path, its BSON type, and (optionally) how it can be queried:

{
  keyId: deks.partyEmail,                 // which DEK encrypts this field
  path: 'partyEmailAddress',
  bsonType: 'string',
  queries: { queryType: 'equality' },     // ← searchable while encrypted; omit ⇒ QE:none
}

Two field categories:

  • QE:equality (has queries) — searchable while encrypted. The driver encrypts the query value with the field's DEK and matches it against the encrypted index, so the server never sees plaintext. Examples: partyEmailAddress, partyMobilePhoneNumber, cardTransactionAccountReference, customerAgreementReference, customerAuthenticationEmailAddress.
  • QE:none (no queries) — retrieval-only, not searchable. Examples: rawGatewayPayload, processorTransactionMetadata, customerAgreementResidentialAddress, governmentIdentificationReference, customerAgreementRiskNotes, paymentCardExpirationDate, partyPostalAddress, partyDateOfBirth, payoutAccountIban, payoutAccountRoutingNumber, destinationIban.

Encrypted collections & their fields

There are 7 QE-encrypted collections (5 QE:equality search keys, 11 QE:none fields):

Collection (BIAN SD) QE:equality (lookup) QE:none (sensitive)
party (SD-13) partyEmailAddress, partyMobilePhoneNumber partyPostalAddress, partyDateOfBirth
cardTransactionLog (SD-254) cardTransactionAccountReference rawGatewayPayload, processorTransactionMetadata
customerAgreementProcedure (SD-53) customerAgreementReference customerAgreementResidentialAddress, governmentIdentificationReference, customerAgreementRiskNotes
paymentCardManagement (SD-88) — paymentCardExpirationDate
customerAuthenticationAssessment (SD-91) customerAuthenticationEmailAddress —
payoutAccountArrangement (SD-66) — payoutAccountIban, payoutAccountRoutingNumber
paymentExecutionProcedure (SD-65) — destinationIban
cardIssuerVault (Card Administration, module-owned) paymentCardNumber (full PAN, CHD), cardServiceCode —

The card token (paymentCardReference) is intentionally not in QE, a network token is not cardholder data under PCI DSS v4.0. The party date of birth / postal address and the payoutAccountArrangement / paymentExecutionProcedure bank fields (IBAN, routing, destination IBAN) are bank data under GDPR Art. 32 / PSD2, not PCI-scoped card data; they are QE:none, Level 2 only.

Two tiers from the same DEKs

buildEncryptedFieldsMaps(deks, tier) produces two maps:

  • level1 — includes only the QE:equality fields. The QE:none fields are not in the map, so the driver returns them as raw Binary ciphertext, and the service layer strips them out.
  • level2 — includes all fields, so the driver auto-decrypts everything before the service even sees the document.

This is the crux of the design: field-level access control is enforced by the QE client itself (which DEKs are in its map) — not by application projection code. A Level 1 client is physically unable to decrypt QE:none fields because it lacks them in its encrypted-fields map.

6. Creating the encrypted collections

Source: backend/src/vendors/setup/createCollections.ts

For each QE collection, setup calls:

clientEncryption.createEncryptedCollection(db, name, {
  provider,                                       // 'local' | 'aws'
  createCollectionOptions: { encryptedFields: map },
  ...(masterKey && { masterKey }),                // AWS only
});

This registers the encryptedFields configuration on the collection and provisions the internal QE metadata/index collections (esc/ecoc) used for encrypted equality search. In v2 the sensitive fields live inline in the main collection (no separate *Sensitive collections); separation is achieved purely through the DEK/tier split.

7. Runtime — role-aware client pools

Source: backend/src/vendors/encryption/roleClients.ts

Two long-lived MongoClient instances are maintained, one per tier, each configured with autoEncryption:

autoEncryption: {
  keyVaultNamespace: 'encryption.__keyVault',
  kmsProviders: buildKmsProviders(),
  encryptedFieldsMap: { 'pcidb.party': maps.party, 'pcidb.cardTransactionLog': maps.cardTransactionLog, ... },
  extraOptions: { cryptSharedLibPath, cryptSharedLibRequired },
}
  • L1 pool → level1 map, connection string MONGODB_URI_LEVEL1.
  • L2 pool → level2 map, connection string MONGODB_URI_LEVEL2.
  • DEKs are resolved first via a plain (non-QE) client (provisionDataEncryptionKeys, idempotent), then the encrypted client is built with the maps.

getDbForRole(role, hasValidToken) returns the right Db:

Role Pool
level2_investigator with a valid escalation token L2 (full decrypt)
security_auditor L2 (full decrypt)
everyone else (customer, L1 analyst, …) L1 (lookup only)

crypt_shared library

Source: backend/src/vendors/encryption/cryptLib.ts

Automatic encryption needs the MongoDB Automatic Encryption Shared Library (mongo_crypt_v1.dll / .so / .dylib). It is located via MONGODB_CRYPT_SHARED_LIB_PATH or platform defaults; if absent the driver attempts auto-discovery / mongocryptd. Download it from the MongoDB Enterprise downloads ("Cryptography Library / crypt_shared").

Defense in depth (Atlas RBAC)

Source: backend/src/vendors/setup/createAtlasRoles.ts

Setup also creates Atlas custom roles and DB users per tier, so the L1/L2 isolation exists at the database-credential level too — not only in the encrypted-fields map.

8. How an encrypted query works

L1 analyst searches partyEmailAddress = "luis@example.com":

  1. The driver encrypts the query value with DEK-party-email.
  2. It matches against the encrypted equality index — the server never decrypts.
  3. Matching documents return with partyEmailAddress auto-decrypted, but governmentIdentificationReference (a QE:none field absent from the L1 map) comes back as Binary and is stripped by the service.

L2 investigator (after an approved escalation, holding a short-lived token) uses the L2 pool → every encrypted field is auto-decrypted.

9. CHD envelope (cardholder data in transit)

Source: backend/src/vendors/encryption/chdCrypto.ts

Independent of QE, but shares the same master-key surface:

  • The local provider derives a 256-bit KEK from the same KMS_LOCAL_MASTER_KEY via HKDF-SHA256.
  • A fresh per-message DEK encrypts the content with AES-256-GCM; the KEK/CMK wraps that DEK (envelope encryption).
  • The result is the opaque chd token that rides card.issuer.validation.requested on the bus. The AAD binds the token to its journey (correlationId + event type), so it cannot be replayed onto another event.
  • The issuer adapter decrypts the chd just-in-time for the wire; plaintext is never re-published, persisted, or logged. The same KMS_PROVIDER switch allows moving to AWS without changing the contract.

10. Key lifecycle & operations

  • Provisioning: npm run setup:key:master (local) → npm run setup:db creates Atlas roles/users, provisions DEKs (one per field, reused by alias), creates encrypted collections, then indexes. Orchestrated by backend/src/vendors/setup/index.ts.
  • Rotation: rotate the CMK in the KMS (AWS) and re-wrap DEKs; DEKs themselves can be rotated by re-keying the vault. The application contract (encrypted-fields maps) is unchanged.
  • Backup: the key vault (encryption.__keyVault) holds the wrapped DEKs — back it up with the CMK reference; ciphertext is meaningless without the CMK.

11. Environment variables

Variable Purpose
PSP_KMS_PROVIDER local or aws (legacy KMS_PROVIDER still read as fallback)
PSP_KMS_LOCAL_MASTER_KEY 96-byte base64 master key (local KMS)
AWS_CMK_ARN, AWS_REGION AWS KMS customer master key (unprefixed)
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN AWS credentials (unprefixed)
MONGODB_URI base/fallback connection string + DEK provisioning
MONGODB_URI_LEVEL1, MONGODB_URI_LEVEL2 per-tier connection strings (Atlas RBAC)
MONGODB_DB_NAME application database name (e.g. fsi-psp-leafy-pay-store)
MONGODB_CRYPT_SHARED_LIB_PATH path to mongo_crypt_v1 shared library
PSP_KMS_KEY_VAULT_URI, PSP_KMS_KEY_VAULT_DATABASE, PSP_KMS_KEY_VAULT_COLLECTION key-vault location overrides (default encryption.__keyVault)

12. Source map

Concern File
KMS providers / CMK options backend/src/vendors/encryption/kms.ts
DEK provisioning + key vault backend/src/vendors/encryption/keyVault.ts
QE field maps (equality / none, tiers) backend/src/vendors/encryption/encryptedFieldsMaps.ts
Role-aware QE client pools backend/src/vendors/encryption/roleClients.ts
crypt_shared library resolution backend/src/vendors/encryption/cryptLib.ts
CHD envelope (bus) backend/src/vendors/encryption/chdCrypto.ts
Encrypted collection creation backend/src/vendors/setup/createCollections.ts
DEK index + provisioning entry backend/src/vendors/setup/provisionDEKs.ts
Atlas roles / DB users backend/src/vendors/setup/createAtlasRoles.ts
Setup orchestration backend/src/vendors/setup/index.ts
Local master-key generator backend/bin/generate-key.ts

13. Derived CVV, issuer CVK, and the module-owned PAN vault

The built-in card-issuer module demonstrates three MongoDB encryption capabilities on genuine issuer data.

13.1 Derived CVV (never stored, SAD)

The per-card CVV is recomputed on demand, never persisted:

perCardCvv = digits( HMAC-SHA256( CVK, cardToken | expiryMMYY | serviceCode ) )[0 : cvvLength]

cvvLength is 3 for Visa / Mastercard and 4 for Amex. A global escape-hatch CVV (validCvv, default 123) stays available for fast demos; cvvMode (both default | global | per_card) selects which values validation accepts. The CVV appears only in ephemeral reveal responses, never in a collection, log, listing, or validation response (PCI DSS Req 3.2).

13.2 CVK envelope encryption (KMS → DEK → CVK)

The Card Verification Key is module-owned issuer key material, provisioned once and stored only wrapped:

CMK / master key  →  DEK (wrapped in encryption.__keyVault)  →  CVK (HKDF from the unwrapped DEK)

Cleartext CVK exists only in process memory. This is the same envelope model as the QE DEKs; source: backend/src/providers/card-issuer/services/cardVerificationKey.service.ts, wired from vendors/setup/provisionDEKs.ts.

13.3 PAN vault (QE:equality) + reveal on demand

cardIssuerVault stores the full PAN (paymentCardNumber) and cardServiceCode with QE:equality (DEKs DEK-vault-pan, DEK-vault-service-code). Equality supports exact PAN lookup (panExact) and dedup while the server sees only ciphertext; substring / suffix QE is intentionally off (equality only, compatible with server 8.0). Day-to-day search stays on the non-sensitive core: last4 (equality) + bin (prefix). The full PAN is revealed on demand, exactly like the IBAN: hidden by default behind an eye icon, revealed ephemerally and audited (card.pan.revealed; IBAN uses account.iban.revealed, CVV card.cvv.revealed). operations_officer reveals directly from the built-in admin (cards:manage + internal-provider gate); the card owner reveals through the provider flow. Step-up MFA/SCA applies in production.

See also: PII (field-protection strategy & options evaluated), Architecture, and the Technical Specification for the full encryptedFieldsMaps and index strategy.

Clone this wiki locally