From 02c6e3fa7dbc3a64e5312ef820e0f1779069cadb Mon Sep 17 00:00:00 2001 From: vector-of-bool Date: Thu, 10 Nov 2022 14:28:17 -0700 Subject: [PATCH] [DRIVERS-2312] Define a CreateEncryptedCollection helper (#1322) * Define a CreateEncryptedCollection helper * A very basic beginning test for CreateEncryptedCollection --- .../client-side-encryption.rst | 96 ++++++++++++++++++- .../client-side-encryption/tests/README.rst | 60 +++++++++++- 2 files changed, 150 insertions(+), 6 deletions(-) diff --git a/source/client-side-encryption/client-side-encryption.rst b/source/client-side-encryption/client-side-encryption.rst index 8e5681ed8c..ce90207051 100644 --- a/source/client-side-encryption/client-side-encryption.rst +++ b/source/client-side-encryption/client-side-encryption.rst @@ -4,7 +4,7 @@ Client Side Encryption :Status: Accepted :Minimum Server Version: 4.2 (CSFLE), 6.0 (Queryable Encryption) -:Last Modified: 2022-11-04 +:Last Modified: 2022-11-10 :Version: 1.11.0 .. _lmc-c-api: https://github.com/mongodb/libmongocrypt/blob/master/src/mongocrypt.h.in @@ -837,9 +837,13 @@ encryptedFieldsMap ``encryptedFieldsMap`` only applies to Queryable Encryption. -If a collection is present on both the ``encryptedFieldsMap`` and ``schemaMap``, libmongocrypt_ will error on initialization. See :ref:`fle2-and-fle1-error`. +If a collection is present on both the ``encryptedFieldsMap`` and ``schemaMap``, +libmongocrypt_ will error on initialization. See :ref:`fle2-and-fle1-error`. -If a collection is present on the ``encryptedFieldsMap``, the behavior of ``CreateCollection()`` and ``Collection.Drop()`` is altered. See :ref:`fle2-createcollection-drop`. +If a collection has a set of encrypted fields, the behavior of +``CreateCollection()`` and ``Collection.Drop()`` is altered. An additional +helper, ``CreateEncryptedCollection()`` has been defined as a convenience +wrapper around ``CreateCollection()``. See :ref:`fle2-createcollection-drop`. Automatic encryption in Queryable Encryption is configured with the ``encryptedFields``. @@ -864,6 +868,42 @@ A collection supporting Queryable Encryption requires an index and three additio .. _create: https://www.mongodb.com/docs/manual/reference/command/create .. _drop: https://www.mongodb.com/docs/manual/reference/command/drop + +.. _GetEncryptedFields: + +Collection ``encryptedFields`` Lookup (GetEncryptedFields) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The convenience methods support the following lookup process for finding the +``encryptedFields`` associated with a collection. + +.. default-role:: math + +Assume an exposition-only function +`GetEncryptedFields(opts, collName, dbName, askDb)`, where `opts` is a set of +options, `collName` is the name of the collection, `dbName` is the name of the +database associated with that collection, and `askDb` is a boolean value. The +resulting ``encryptedFields`` `EF` is found by: + +1. Let `QualName` be the string formed by joining `dbName` and `collName` with + an ASCII dot ``"."``. +2. If `opts` contains an ``"encryptedFields"`` property, then `EF` is the value + of that property. +3. Otherwise, if ``AutoEncryptionOptions.encryptedFieldsMap`` contains an + element named by `QualName`, then `EF` is the value of that element. +4. Otherwise, if `askDb` is `true`: + + 1. Issue a ``listCollections`` command against the database named by + `dbName`, filtered by ``{name: }``. Let the result be the + document `L`. + 2. If `L` contains an ``options`` document element, and that element contains + an ``encryptedFields`` document element, `EF` is `L`\ + ``["options"]["encryptedFields"]``. + 3. Otherwise, `EF` is *not-found* + +5. Otherwise, `EF` is considered *not-found*. + + Create Collection Helper ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -891,6 +931,45 @@ If the collection namespace has an associated ``encryptedFields``, then do the f - Create the collection ``collectionName`` with ``collectionOptions`` and the option ``encryptedFields`` set to the ``encryptedFields``. - Create the the index ``{"__safeContent__": 1}`` on collection ``collectionName``. + +Create Encrypted Collection Helper +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To support automatic generation of encryption data keys, a helper +`CreateEncryptedCollection(CE, database, collName, collOpts, kmsProvider, dkOpts)` +is defined, where `CE` is a ClientEncryption_ object, `kmsProvider` is a +KMSProviderName_ and `dkOpts` is a DataKeyOpts_. It has the following behavior: + +- Let `dbName` be the name of `database`. Look up the encrypted fields `EF` for + the new collection as `GetEncryptedFields(collOpts, collName, dbName, false)` + (`See here `_). +- If `EF` is *not-found*, report an error that there are no ``encryptedFields`` + defined for the collection. +- Let `EF'` be a copy of `EF`. Update `EF'` in the following manner: + + - Let `Fields` be the ``"fields"`` element within `EF'`. + - If `Fields` is present and is an array value, then for each element `F` of + `Fields`: + + - If `F` is not a document element, skip it. + - Otherwise, if `F` has a ``"keyId"`` named element `K` and `K` is a + ``null`` value: + + - Let `D` be the result of ``CE.createDataKey(kmsProvider, dkOpts)``. + - If generating `D` resulted in an error `E`, the entire + `CreateEncryptedCollection` must now fail with error `E`. Return the + partially-formed `EF'` with the error so that the caller may know what + datakeys have already been created by the helper. + - Replace `K` in `F` with `D`. + +- Create a new set of options `collOpts'` duplicating `collOpts`. Set the + ``"encryptedFields"`` named element of `collOpts'` to `EF'`. + +- Invoke the ``CreateCollection`` helper as + `CreateCollection(database, collName, collOpts')`. Return the resulting + collection and the generated `EF'`. + + Drop Collection Helper ^^^^^^^^^^^^^^^^^^^^^^ @@ -915,6 +994,8 @@ If the collection namespace has an associated ``encryptedFields``, then do the f If ``encryptedFields["ecocCollection"]`` is not set, use the collection name ``enxcol_..ecoc``. - Drop the collection ``collectionName``. +.. default-role:: literal +.. _ClientEncryption: ClientEncryption ---------------- @@ -924,6 +1005,12 @@ ClientEncryption class ClientEncryption { ClientEncryption(opts: ClientEncryptionOpts); + // The "Create Encrypted Collection" helper is a convenience function wrapping CreateCollection. It will + // create a collection with encrypted fields, automatically allocating and assigning new data encryption + // keys. It returns a handle to the new collection, as well as a document consisting of the generated + // "encryptedFields" options. Refer to "Create Encrypted Collection Helper" + createEncryptedCollection(database: Database, collName: string, collOpts, kmsProvider: KMSProviderName, dkOpts: DataKeyOpts): [Collection, Document]; + // Creates a new key document and inserts into the key vault collection. // Returns the _id of the created document as a UUID (BSON binary subtype 0x04). createDataKey(kmsProvider: KMSProviderName, opts: DataKeyOpts | null): Binary; @@ -2515,6 +2602,9 @@ explicit session parameter as described in the Changelog ========= +:2022-11-10: Defined a ``CreateEncryptedCollection`` helper for creating new + encryption keys automatically for the queryable encrypted fields in + a new collection. :2022-11-07: Reformat changelog. :2022-11-04: Permit global cache for Azure credentials. :2022-10-26: Do not connect to ``mongocryptd`` if shared library is loaded. diff --git a/source/client-side-encryption/tests/README.rst b/source/client-side-encryption/tests/README.rst index 6e40d2b0a4..6a5b095841 100644 --- a/source/client-side-encryption/tests/README.rst +++ b/source/client-side-encryption/tests/README.rst @@ -2395,7 +2395,7 @@ The following tests that a mongocryptd client is not created when shared library #. On ``listenerThread``, create a TcpListener on 127.0.0.1 endpoint and port 27021. Start the listener and wait for establishing connections. If any connection is established, then signal about this to the main thread. - + Drivers MAY pass a different port if they expect their testing infrastructure to be using port 27021. Pass a port that should be free. #. Create a MongoClient configured with auto encryption (referred to as ``client_encrypted``) @@ -2415,7 +2415,61 @@ The following tests that a mongocryptd client is not created when shared library { "mongocryptdURI": "mongodb://localhost:27021" } - -#. Use ``client_encrypted`` to insert the document ``{"unencrypted": "test"}`` into ``db.coll``. + +#. Use ``client_encrypted`` to insert the document ``{"unencrypted": "test"}`` into ``db.coll``. #. Expect no signal from ``listenerThread``. + + + +21. Automatic Data Encryption Keys +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Case 1: Simple Creation and Validation +`````````````````````````````````````` + +This test is the most basic to verify that CreateEncryptedCollection_ created a +collection with queryable encryption enabled. It verifies that the server +rejects an attempt to insert plaintext in an encrypted fields. + +.. _CreateEncryptedCollection: ../client-side-encryption.rst#create-encrypted-collection-helper +.. _MongoClient: ../client-side-encryption.rst#mongoclient-changes + +.. highlight:: typescript +.. default-role:: math + +1. Create a ClientEncryption_ object `CE` with the following options:: + + clientEncryptionOptions: { + keyVaultClient: , + keyVaultNamespace: "keyvault.datakeys", + kmsProviders: { + local: { key: base64Decode(LOCAL_MASTERKEY) }, + }, + } + +2. Create a new create-collection options `Opts` including the following:: + + { + encryptedFields: { + fields: [{ + path: "ssn", + bsonType: "string", + keyId: null + }] + } + } + +3. Open a new database handle `DB`. +4. Invoke `CreateEncryptedCollection(CE, DB, "testing1", Opts, "local", null)` + to obtain a new collection `Coll`. Expect success. +5. Attempt to insert the following document into `Coll`:: + + { + ssn: "123-45-6789" + } + +6. Expect an error from the insert operation that indicates that the document + failed validation. This error indicates that the server expects to receive an + encrypted field for ``ssn``, but we tried to insert a plaintext field via a + client that is unaware of the encryption requirements. \ No newline at end of file