[DRIVERS-2241] Wording to add usage and API information for the csfle dynamic library

[vector-of-bool]

These changes include updates for client-side-encryption on how to use and expose the csfle dynamic library to perform command-marking in place of mongocryptd.

This does not yet include test cases for csfle.

Refer: DRIVERS-2241

[addaleax]

In the server shared library planning docs, the expectation was that, in typical situations, the shared library would be deployed as part of application directories, not as part of the system libraries.

Considering this, would it be worth simplifying this interface to merge the csfleSearchPathsPrefer/csfleSearchPathsFallback/csfleDisableSystemLibrary options and only specify a single list of search paths, which may or may not include $SYSTEM? We already need to make users aware that $ORIGIN exists as an expansion.

(If the desire here is to make the shared library an opt-out feature rather than an opt-in one, we could make specifying an empty search path list different from specifying no search path list at all, i.e. make the default ["$SYSTEM"] rather than [].)

[vector-of-bool]

I interpretted the discussion as to whether it would be system-installed to be still tentative (although I would fall on the side of application-level deployment rather than system installation).

Given that we already need to document search path behavior, I think it would be feasible to merge these options and allow "set an empty path list" to be the "disable csfle" option. It will minimize the moving parts.

Although that still leaves an open question as to what the default should be (as related to your other comment). A default of ["$SYSTEM"] is reasonable if we prefer a system-wide installation, but a default of ["$ORIGIN"] would be better for application-level deployment (i.e. the user would only need to have csfle reside alongside the libmongocrypt library).

Either way, I like the idea to reduce the moving parts and merge these options and replace the "disable" toggle.

[kevinAlbs]

In the server shared library planning docs, the expectation was that, in typical situations, the shared library would be deployed as part of application directories, not as part of the system libraries.

I missed that in scoping. I was under the impression that we should prefer a system-wide installation. I now see in the Technical Design a note to "Copy to app directory".

Although that still leaves an open question as to what the default should be (as related to your other comment). A default of ["$SYSTEM"] is reasonable if we prefer a system-wide installation, but a default of ["$ORIGIN"] would be better for application-level deployment (i.e. the user would only need to have csfle reside alongside the libmongocrypt library).

Drivers that a package libmongocrypt may need to override these defaults. That is Java, Python, C#, and Node.js (see How is libmongocrypt packaged for drivers?).

Would it make sense for drivers that rely on system install for libmongocrypt to use $SYSTEM as the default? If so, then using $SYSTEM as the default makes sense to me.

[addaleax]

Is it an explicit error to specify CSFLE library path options and bypassAutoEncryption, or will the CSFLE library path just be ignored? Either way is good for me.

[kevinAlbs]

For precedence, it is not currently an error to specify both extraOptions.mongocryptdBypassSpawn
and extraOptions.mongocryptdSpawnPath. I am OK with either choice.

[vector-of-bool]

I don't think it would be an error, but it might be suspicious. Perhaps a "drivers MAY warn" would be useful?

[kevinAlbs]

I prefer not using "MAY". It creates more possibility of inconsistency between driver implementations.

My vote goes for ignoring. There is little harm in ignoring. There is precedent with mongocryptdBypassSpawn and mongocryptdSpawnPath.

[kevinAlbs]

The additional refactoring is nice. Do you mind moving the larger chunks of refactoring to a separate PR? Other DBX engineers will likely refer to the PR to see the new content. I think it will be easier to interpret if the diff is limited to the new content.

[kevinAlbs]

In the server shared library planning docs, the expectation was that, in typical situations, the shared library would be deployed as part of application directories, not as part of the system libraries.

I missed that in scoping. I was under the impression that we should prefer a system-wide installation. I now see in the Technical Design a note to "Copy to app directory".

Although that still leaves an open question as to what the default should be (as related to your other comment). A default of ["$SYSTEM"] is reasonable if we prefer a system-wide installation, but a default of ["$ORIGIN"] would be better for application-level deployment (i.e. the user would only need to have csfle reside alongside the libmongocrypt library).

Drivers that a package libmongocrypt may need to override these defaults. That is Java, Python, C#, and Node.js (see How is libmongocrypt packaged for drivers?).

Would it make sense for drivers that rely on system install for libmongocrypt to use $SYSTEM as the default? If so, then using $SYSTEM as the default makes sense to me.

[kevinAlbs]

For precedence, it is not currently an error to specify both extraOptions.mongocryptdBypassSpawn
and extraOptions.mongocryptdSpawnPath. I am OK with either choice.

[kevinAlbs]

Suggested change

	`(from the libmongocrypt C API) <lmc-c-api_>`_ based on the search critera that
	`(from the libmongocrypt C API) <lmc-c-api_>`_ based on the search criteria that

[vector-of-bool]

@kevinAlbs I've stripped a lot of the not-necessary cleanup changes. Should be much more streamlined now.

[kevinAlbs]

LGTM pending the open question about $DEFAULT. Great work! Please re-request my review if you need further input.

I would like to wait until there is a C PoC to merge.

I have asked for a reviewer from the from a language team that packages libmongocrypt.

[kevinAlbs]

Added @jyemin as a reviewer.

[kevinAlbs]

After talking with @jyemin, I realize The DBX scope answers the question Should csfle be available in package managers? with "No".

I have two new open questions about simplifying the API. I have left comments inline.

[addaleax]

Just approving again to re-affirm that the current state of this PR LGTM

[vector-of-bool]

I've revived the search-paths functionality in a limited fashion, not for the user-API, but to be an option to driver testing. Being able to allow drivers more control over the csfle search behavior is highly beneficial for testing purposes.