Encryption

🔐 Encryption — Queryable Encryption, DEKs & Key Management

Status: Implemented (v2) PCI DSS coverage: Req 3 (encryption in use / at rest), Req 3.5–3.6 (key management), Req 4 (in transit), Req 7 (least-privilege access)

This page explains how cardholder and personal data are encrypted in the demo: how the Queryable Encryption (QE) fields are configured, how Data Encryption Keys (DEKs) and the master key (CMK) are managed, and how the cardholder-data (CHD) envelope works on the event bus.

1. Two complementary mechanisms

The platform uses two distinct crypto systems that share the same master key surface:

Mechanism	Protects	Where	Managed by
Queryable Encryption (QE)	PII & sensitive fields at rest in MongoDB (email, phone, account reference, address, date of birth, government ID, gateway payload, card expiry, and bank data: payout IBAN/routing and destination IBAN)	MongoDB collections	The MongoDB driver (automatic encryption)
CHD envelope	Cardholder data (PAN / CVV / expiry) in transit on the internal event bus	The opaque `chd` token on bus events	The application (`AES-256-GCM`, `node:crypto`)

The CVV is never stored in MongoDB and is never placed in QE. It crosses the bus encrypted just-in-time on the way to the card issuer, and the built-in issuer also derives it per card on demand (HMAC-SHA256 under the issuer key, see §13) rather than holding a single global value (PCI DSS Req 3.2: no sensitive authentication data stored). The full PAN is not stored by the PSP core either; when the built-in card-issuer module is active it stores the full PAN encrypted with QE:equality in its own module-owned vault (cardIssuerVault, see §13), the issuer CDE. The core keeps only the token + BIN + last4.

2. Key hierarchy

Both mechanisms follow the standard envelope-encryption model — a three-level key hierarchy:

CMK / Master Key            ← lives in the KMS; never encrypts data directly
   │  (wraps / unwraps)
   └─ DEK  (Data Encryption Key)   ← stored encrypted in the key vault
        │  (encrypts / decrypts)
        └─ field value / message   ← the actual ciphertext in the document or on the bus

The CMK never leaves the KMS (for AWS) — the driver asks KMS to wrap/unwrap DEKs.
Each DEK is stored already-encrypted (wrapped by the CMK) in the key vault collection.
The DEK is what actually encrypts a field value; compromising a DEK ciphertext is useless without the CMK.

3. The master key (CMK) — `KMS_PROVIDER`

Source: backend/src/vendors/encryption/kms.ts

The KMS provider is chosen by the PSP_KMS_PROVIDER environment variable (a legacy KMS_PROVIDER is still read as a fallback):

Local (offline demo — default)

PSP_KMS_PROVIDER=local
PSP_KMS_LOCAL_MASTER_KEY=<96-byte base64 key>

A 96-byte local master key (the size MongoDB QE requires for the local provider). Generate it with:

npm run setup:key:master   # → backend/bin/seed-generate-key.ts → randomBytes(96).toString('base64')

AWS KMS (production)

PSP_KMS_PROVIDER=aws
AWS_CMK_ARN=arn:aws:kms:...:key/...
AWS_REGION=...
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
# AWS_SESSION_TOKEN=...   # optional (temporary credentials)

With AWS the customer controls the CMK; MongoDB has zero access to it.

buildKmsProviders() returns the kmsProviders object passed to the driver; buildCmkOptions() returns the masterKey reference (AWS only — local needs none).

4. Data Encryption Keys (DEKs)

Source: backend/src/vendors/encryption/keyVault.ts

One DEK per encrypted field. MongoDB QE forbids sharing a DEK between fields of the same collection (error 6338401).
DEKs are stored in the key vault: encryption.__keyVault, each one wrapped by the CMK.
DEKs are created with ClientEncryption.createDataKey(provider, { masterKey, keyAltNames: ['DEK-...'] }) and looked up by their keyAltNames alias — so provisioning is idempotent (re-running setup reuses existing keys, never duplicates them).

The DEKs are organised into two access tiers (40 DEKs total: 21 lookup + 19 sensitive):

Tier	DEK aliases	Backs	Audience
Lookup (searchable: equality / range / text)	`DEK-tx-account-ref`, `DEK-party-email`, `DEK-party-phone`, `DEK-customer-account-ref`, `DEK-auth-email`, `DEK-party-name`, `DEK-party-dob`, `DEK-party-nationality`, `DEK-party-place-of-birth`, `DEK-party-sex`, `DEK-ca-govid-type`, `DEK-ca-govid-number`, `DEK-ca-govid-issuing-country`, `DEK-ca-govid-expiry`, `DEK-ca-tax-id`, `DEK-ca-occupation`, `DEK-kyc-risk-score`, `DEK-kyc-risk-rating`, `DEK-kyc-pep-status`, `DEK-kyc-sanctions-result`, `DEK-vault-pan`, `DEK-vault-service-code`	searchable encrypted fields	All authenticated analyst roles (L1+)
Sensitive (QE:none)	`DEK-tx-raw-payload`, `DEK-tx-processor-meta`, `DEK-customer-address`, `DEK-customer-gov-id`, `DEK-customer-risk-notes`, `DEK-card-expiry`, `DEK-party-address`, `DEK-payout-iban`, `DEK-payout-routing`, `DEK-exec-dest-iban`, `DEK-ca-source-of-funds`, `DEK-ca-purpose`, `DEK-kyc-screening-ref`, `DEK-rtp-payee-alias`, `DEK-rtp-payer-alias`, `DEK-rtp-remittance`, `DEK-rtp-address`, `DEK-rtp-payee-name`	retrieval-only sensitive fields (high-sensitivity PII, CHD, and GDPR/PSD2 bank data)	L2 Investigator (with escalation token) + Security Auditor

Two edge cases in that grouping: DEK-party-dob sits in the lookup tier because the field is range-searchable, and DEK-card-expiry is a QE:none field present in both tier maps (expiry alone is not a sensitive-escalation field, and a field with no queries is unsearchable regardless).

Provisioning (backend/src/vendors/setup/provisionDEKs.ts) first creates a unique index on keyAltNames in the vault, then calls provisionDataEncryptionKeys() which getOrCreates each named DEK.

5. QE field configuration

Source: backend/src/vendors/encryption/encryptedFieldsMaps.ts

Each encrypted collection declares an encryptedFields.fields[] map. Every field specifies the DEK that protects it, its path, its BSON type, and (optionally) how it can be queried:

{
  keyId: deks.partyEmail,                 // which DEK encrypts this field
  path: 'partyEmailAddress',
  bsonType: 'string',
  queries: { queryType: 'equality' },     // ← searchable while encrypted; omit ⇒ QE:none
}

The six query modes

What distinguishes one mode from another is the queries object alone. The same DEK, the same bsonType and the same path syntax apply throughout; only queries decides how the encrypted index can be interrogated.

Mode	`queries` configuration	Answers	Query operator
none	property absent (only `keyId`, `path`, `bsonType`)	nothing, retrieval only	—
equality	`{ queryType: 'equality', contention?: n }`	"is it exactly X?"	`$eq` / plain match
range	`{ queryType: 'range', min, max, sparsity, trimFactor }` (`precision` for decimals)	"is it between X and Y?"	`$gt`, `$gte`, `$lt`, `$lte`
substring	`{ queryType: 'substringPreview', strMaxLength, strMinQueryLength, strMaxQueryLength, caseSensitive, diacriticSensitive }`	"does it contain X?"	`$encStrContains`
prefix	`{ queryType: 'prefixPreview', …same params }`	"does it start with X?"	`$encStrStartsWith`
suffix	`{ queryType: 'suffixPreview', …same params }`	"does it end with X?"	`$encStrEndsWith`

Notes on the text modes (substring / prefix / suffix):

They are a MongoDB 8.2 preview feature and need both server 8.2+ and a matching crypt_shared 8.2+ library.
They are gated by PSP_QE_TEXT_SEARCH. When it is off, the textQuery() helper rewrites the field to { queryType: 'equality', contention: 8 }, so the field stays encrypted and searchable, just exact-match only. effectiveMode() in the search service mirrors that degradation.
strMaxQueryLength bounds what the encrypted index can match. A longer operator input is queried with the longest safe window (last N for suffix, first N for prefix/substring) and the surplus is refined in memory over the decrypted values.
Every field can carry at most one mode. There is no field that is both range-searchable and substring-searchable.

Fields grouped by query mode

9 QE-encrypted collections: 16 equality, 3 range, 1 substring, 1 prefix, 1 suffix, 18 QE:none.

Collection (BIAN SD)	equality	range	substring	prefix	suffix	none (L2 only)
`party` (SD-13)	`partyEmailAddress`, `partyMobilePhoneNumber`, `partyNationality`, `partyPlaceOfBirth`, `partySex`	`partyDateOfBirth`	`partyName`	—	—	`partyPostalAddress`
`customerAgreementProcedure` (SD-53)	`customerAgreementReference`, `…GovernmentID.type`, `…GovernmentID.issuingCountry`, `customerAgreementOccupation`, `…KycCheckRiskRating`, `…KycCheckPepStatus`, `…KycCheckSanctionsResult`	`…GovernmentID.expiryDate`, `…KycCheckRiskScore`	—	`customerAgreementTaxIDNumber`	`…GovernmentID.number`	`customerAgreementResidentialAddress`, `customerAgreementSourceOfFunds`, `customerAgreementPurposeOfRelationship`, `…KycCheckScreeningProviderRef`, `governmentIdentificationReference` (deprecated), `customerAgreementRiskNotes` (deprecated)
`cardTransactionLog` (SD-254)	`cardTransactionAccountReference`	—	—	—	—	`rawGatewayPayload`, `processorTransactionMetadata`
`cardIssuerVault` (Card Administration, module-owned)	`paymentCardNumber` (full PAN, CHD), `cardServiceCode`	—	—	—	—	—
`customerAuthenticationAssessment` (SD-91)	`customerAuthenticationEmailAddress`	—	—	—	—	—
`paymentCardManagement` (SD-88)	—	—	—	—	—	`paymentCardExpirationDate`
`payoutAccountArrangement` (SD-66)	—	—	—	—	—	`payoutAccountIban`, `payoutAccountRoutingNumber`
`paymentExecutionProcedure` (SD-65)	—	—	—	—	—	`destinationIban`
`paymentRequestProcedure` (SD-65, RTP)	—	—	—	—	—	`payeeAlias`, `payerAlias`, `unstructuredRemittance`, `structuredAddress`, `payeeName`

Reading the grouping by intent rather than by collection:

equality is for identifiers and low-cardinality enums (email, phone, account reference, full PAN, nationality, ID type, risk rating, PEP flag). Enum-like fields carry an explicit contention (6 or 8) because a handful of distinct values would otherwise make the encrypted index trivially frequency-analyzable.
range is for the three fields an auditor filters by interval: date of birth (minor detection), government-ID expiry (expired-KYC detection) and KYC risk score.
substring is used once, on partyName, the only field where an investigator genuinely types a fragment. It is the only case-insensitive and diacritic-insensitive field in the map.
prefix fits the tax ID, whose leading characters encode the country/issuer.
suffix fits the government-ID number, because the last digits are what appears on a form or is quoted over the phone.
none covers everything that is sensitive but never a search key: free-text notes, structured addresses, raw gateway payloads, bank identifiers and RTP aliases. Not being searchable is the point: a field with no queries has no encrypted index, so it leaks nothing about its own distribution.

The card token (paymentCardReference) is intentionally not in QE, a network token is not cardholder data under PCI DSS v4.0. The party date of birth / postal address and the payoutAccountArrangement / paymentExecutionProcedure / paymentRequestProcedure bank and alias fields (IBAN, routing, destination IBAN, RTP aliases) are bank data under GDPR Art. 32 / PSD2, not PCI-scoped card data; RTP alias lookups go through a plaintext SHA-256 hash instead of a QE index.

Two tiers from the same DEKs

buildEncryptedFieldsMaps(deks, tier) produces two maps:

level1 — includes only the QE:equality fields. The QE:none fields are not in the map, so the driver returns them as raw Binary ciphertext, and the service layer strips them out.
level2 — includes all fields, so the driver auto-decrypts everything before the service even sees the document.

This is the crux of the design: field-level access control is enforced by the QE client itself (which DEKs are in its map) — not by application projection code. A Level 1 client is physically unable to decrypt QE:none fields because it lacks them in its encrypted-fields map.

6. Creating the encrypted collections

Source: backend/src/vendors/setup/createCollections.ts

For each QE collection, setup calls:

clientEncryption.createEncryptedCollection(db, name, {
  provider,                                       // 'local' | 'aws'
  createCollectionOptions: { encryptedFields: map },
  ...(masterKey && { masterKey }),                // AWS only
});

This registers the encryptedFields configuration on the collection and provisions the internal QE metadata/index collections (esc/ecoc) used for encrypted equality search. In v2 the sensitive fields live inline in the main collection (no separate *Sensitive collections); separation is achieved purely through the DEK/tier split.

7. Runtime — role-aware client pools

Source: backend/src/vendors/encryption/roleClients.ts

Two long-lived MongoClient instances are maintained, one per tier, each configured with autoEncryption:

autoEncryption: {
  keyVaultNamespace: 'encryption.__keyVault',
  kmsProviders: buildKmsProviders(),
  encryptedFieldsMap: { 'pcidb.party': maps.party, 'pcidb.cardTransactionLog': maps.cardTransactionLog, ... },
  extraOptions: { cryptSharedLibPath, cryptSharedLibRequired },
}

L1 pool → level1 map, connection string MONGODB_URI_LEVEL1.
L2 pool → level2 map, connection string MONGODB_URI_LEVEL2.
DEKs are resolved first via a plain (non-QE) client (provisionDataEncryptionKeys, idempotent), then the encrypted client is built with the maps.

getDbForRole(role, hasValidToken) returns the right Db:

Role	Pool
`level2_investigator` with a valid escalation token	L2 (full decrypt)
`security_auditor`	L2 (full decrypt)
everyone else (customer, L1 analyst, …)	L1 (lookup only)

crypt_shared library

Source: backend/src/vendors/encryption/cryptLib.ts

Automatic encryption needs the MongoDB Automatic Encryption Shared Library (mongo_crypt_v1.dll / .so / .dylib). It is located via MONGODB_CRYPT_SHARED_LIB_PATH or platform defaults; if absent the driver attempts auto-discovery / mongocryptd. Download it from the MongoDB Enterprise downloads ("Cryptography Library / crypt_shared").

Defense in depth (Atlas RBAC)

Source: backend/src/vendors/setup/createAtlasRoles.ts

Setup also creates Atlas custom roles and DB users per tier, so the L1/L2 isolation exists at the database-credential level too — not only in the encrypted-fields map.

8. How an encrypted query works

L1 analyst searches partyEmailAddress = "luis@example.com":

The driver encrypts the query value with DEK-party-email.
It matches against the encrypted equality index — the server never decrypts.
Matching documents return with partyEmailAddress auto-decrypted, but governmentIdentificationReference (a QE:none field absent from the L1 map) comes back as Binary and is stripped by the service.

L2 investigator (after an approved escalation, holding a short-lived token) uses the L2 pool → every encrypted field is auto-decrypted.

9. CHD envelope (cardholder data in transit)

Source: backend/src/vendors/encryption/chdCrypto.ts

Independent of QE, but shares the same master-key surface:

The local provider derives a 256-bit KEK from the same KMS_LOCAL_MASTER_KEY via HKDF-SHA256.
A fresh per-message DEK encrypts the content with AES-256-GCM; the KEK/CMK wraps that DEK (envelope encryption).
The result is the opaque chd token that rides card.issuer.validation.requested on the bus. The AAD binds the token to its journey (correlationId + event type), so it cannot be replayed onto another event.
The issuer adapter decrypts the chd just-in-time for the wire; plaintext is never re-published, persisted, or logged. The same KMS_PROVIDER switch allows moving to AWS without changing the contract.

10. Key lifecycle & operations

Provisioning: npm run setup:key:master (local) → npm run setup:db creates Atlas roles/users, provisions DEKs (one per field, reused by alias), creates encrypted collections, then indexes. Orchestrated by backend/src/vendors/setup/index.ts.
Rotation: rotate the CMK in the KMS (AWS) and re-wrap DEKs; DEKs themselves can be rotated by re-keying the vault. The application contract (encrypted-fields maps) is unchanged.
Backup: the key vault (encryption.__keyVault) holds the wrapped DEKs — back it up with the CMK reference; ciphertext is meaningless without the CMK.

11. Environment variables

Variable	Purpose
`PSP_KMS_PROVIDER`	`local` or `aws` (legacy `KMS_PROVIDER` still read as fallback)
`PSP_KMS_LOCAL_MASTER_KEY`	96-byte base64 master key (local KMS)
`AWS_CMK_ARN`, `AWS_REGION`	AWS KMS customer master key (unprefixed)
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`	AWS credentials (unprefixed)
`MONGODB_URI`	base/fallback connection string + DEK provisioning
`MONGODB_URI_LEVEL1`, `MONGODB_URI_LEVEL2`	per-tier connection strings (Atlas RBAC)
`MONGODB_DB_NAME`	application database name (e.g. `fsi-psp-leafy-pay-store`)
`MONGODB_CRYPT_SHARED_LIB_PATH`	path to `mongo_crypt_v1` shared library
`PSP_KMS_KEY_VAULT_URI`, `PSP_KMS_KEY_VAULT_DATABASE`, `PSP_KMS_KEY_VAULT_COLLECTION`	key-vault location overrides (default `encryption.__keyVault`)

12. Source map

Concern	File
KMS providers / CMK options	`backend/src/vendors/encryption/kms.ts`
DEK provisioning + key vault	`backend/src/vendors/encryption/keyVault.ts`
QE field maps (query modes + tiers)	`backend/src/vendors/encryption/encryptedFieldsMaps.ts`
Searchable-field registry + query builders	`backend/src/modules/customer/services/customerAgreement.service.ts`
Text-search gate (`PSP_QE_TEXT_SEARCH`)	`backend/src/config.ts`
Role-aware QE client pools	`backend/src/vendors/encryption/roleClients.ts`
crypt_shared library resolution	`backend/src/vendors/encryption/cryptLib.ts`
CHD envelope (bus)	`backend/src/vendors/encryption/chdCrypto.ts`
Encrypted collection creation	`backend/src/vendors/setup/createCollections.ts`
DEK index + provisioning entry	`backend/src/vendors/setup/provisionDEKs.ts`
Atlas roles / DB users	`backend/src/vendors/setup/createAtlasRoles.ts`
Setup orchestration	`backend/src/vendors/setup/index.ts`
Local master-key generator	`backend/bin/generate-key.ts`

13. Derived CVV, issuer CVK, and the module-owned PAN vault

The built-in card-issuer module demonstrates three MongoDB encryption capabilities on genuine issuer data.

13.1 Derived CVV (never stored, SAD)

The per-card CVV is recomputed on demand, never persisted:

perCardCvv = digits( HMAC-SHA256( CVK, cardToken | expiryMMYY | serviceCode ) )[0 : cvvLength]

cvvLength is 3 for Visa / Mastercard and 4 for Amex. A global escape-hatch CVV (validCvv, default 123) stays available for fast demos; cvvMode (both default | global | per_card) selects which values validation accepts. The CVV appears only in ephemeral reveal responses, never in a collection, log, listing, or validation response (PCI DSS Req 3.2).

13.2 CVK envelope encryption (KMS → DEK → CVK)

The Card Verification Key is module-owned issuer key material, provisioned once and stored only wrapped:

CMK / master key  →  DEK (wrapped in encryption.__keyVault)  →  CVK (HKDF from the unwrapped DEK)

Cleartext CVK exists only in process memory. This is the same envelope model as the QE DEKs; source: backend/src/providers/card-issuer/services/cardVerificationKey.service.ts, wired from vendors/setup/provisionDEKs.ts.

13.3 PAN vault (QE:equality) + reveal on demand

cardIssuerVault stores the full PAN (paymentCardNumber) and cardServiceCode with QE:equality (DEKs DEK-vault-pan, DEK-vault-service-code). Equality supports exact PAN lookup (panExact) and dedup while the server sees only ciphertext; substring / suffix QE is intentionally off (equality only, compatible with server 8.0). Day-to-day search stays on the non-sensitive core: last4 (equality) + bin (prefix). The full PAN is revealed on demand, exactly like the IBAN: hidden by default behind an eye icon, revealed ephemerally and audited (card.pan.revealed; IBAN uses account.iban.revealed, CVV card.cvv.revealed). operations_officer reveals directly from the built-in admin (cards:manage + internal-provider gate); the card owner reveals through the provider flow. Step-up MFA/SCA applies in production.

See also: PII (field-protection strategy & options evaluated), Architecture, and the Technical Specification for the full encryptedFieldsMaps and index strategy.

Beneficial owners stay out of QE

merchantAgreementProcedure remains plaintext (no QE map). The new merchantBeneficialOwners embed carries only FK + role + numeric ownership metadata (business data, not PII/CHD). Owner PII is encrypted in the referenced party record (existing QE tiers); it is never copied into the embed, so no double encryption and no new encrypted fields (QE fields are immutable once the collection exists).

Added 2026-07-24.

Uh oh!

Encryption

🔐 Encryption — Queryable Encryption, DEKs & Key Management

1. Two complementary mechanisms

2. Key hierarchy

3. The master key (CMK) — KMS_PROVIDER

Local (offline demo — default)

AWS KMS (production)

4. Data Encryption Keys (DEKs)

5. QE field configuration

The six query modes

Fields grouped by query mode

Two tiers from the same DEKs

6. Creating the encrypted collections

7. Runtime — role-aware client pools

crypt_shared library

Defense in depth (Atlas RBAC)

8. How an encrypted query works

9. CHD envelope (cardholder data in transit)

10. Key lifecycle & operations

11. Environment variables

12. Source map

13. Derived CVV, issuer CVK, and the module-owned PAN vault

13.1 Derived CVV (never stored, SAD)

13.2 CVK envelope encryption (KMS → DEK → CVK)

13.3 PAN vault (QE:equality) + reveal on demand

Beneficial owners stay out of QE

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

3. The master key (CMK) — `KMS_PROVIDER`