[DRIVERS-2411] Prose Tests for IMDS Communication (#1314)

* Defining initial prose tests for IMDS communication.
* Fix incorrect Azure resource URL in spec doc
* Specify a timeout on the IMDS HTTP request
* Tweak wording on timeout and error handling, fix typo in IP addr

source/client-side-encryption/client-side-encryption.rst

@@ -624,6 +624,12 @@ Identities* associated with them. From within the VM, an identity can be used by
 obtaining an access token via HTTP from the *Azure Instance Metadata Service*
 (IMDS). `See this documentation for more information`__
 
+..note::
+
+   To optimize for testability, it is recommended to implement an isolated
+   abstraction for communication with IMDS. This will aide in the implementation
+   of the prose tests of the communication with an IMDS server.
+
 __ https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-use-vm-token#get-a-token-using-http
 
 .. default-role:: math
@@ -633,24 +639,26 @@ The below steps should be taken:
 1. Let `U` be a new URL, initialized from the URL string
    :ts:`"http://169.254.169.254/metadata/identity/oauth2/token"`
 2. Add a query parameter ``api-version=2018-02-01`` to `U`.
-3. Add a query parameter ``resource=http://vault.azure.com/`` to `U`.
+3. Add a query parameter ``resource=https://vault.azure.net/`` to `U`.
 4. Prepare an HTTP GET request `Req` based on `U`.
 
    .. note:: All query parameters on `U` should be appropriately percent-encoded
 
 5. Add HTTP headers ``Metadata: true`` and ``Accept: application/json`` to
    `Req`.
-6. Issue `Req` to the Azure IMDS server ``168.254.169.254:80``. Let `Resp` be
-   the response from the server.
-7.  If `Resp_{status} ≠ 200`, obtaining the access token has failed, and the
-    HTTP response body of `Resp` encodes information about the error that
-    occurred. Return an error instead of an access token.
-8.  Otherwise, let `J` be the JSON document encoded in the HTTP response body
-    of `Resp`.
-9.  The result access token `T` is given as the ``access_token`` string property
-    of `J`. Return `T` as the resulting access token.
-10. The resulting "expires in" duration `d_{exp}` is a count of seconds given as an
-    ASCII-encoded integer string ``expires_in`` property of `J`.
+6. Issue `Req` to the Azure IMDS server ``169.254.169.254:80``. Let `Resp` be
+   the response from the server. If the HTTP response is not completely received
+   within ten seconds, consider the request to have timed out, and return an
+   error instead of an access token.
+7. If `Resp_{status} ≠ 200`, obtaining the access token has failed, and the HTTP
+   response body of `Resp` encodes information about the error that occurred.
+   Return an error including the HTTP response body instead of an access token.
+8. Otherwise, let `J` be the JSON document encoded in the HTTP response body of
+   `Resp`.
+9. The result access token `T` is given as the ``access_token`` string property
+   of `J`. Return `T` as the resulting access token.
+10. The resulting "expires in" duration `d_{exp}` is a count of seconds given as
+    an ASCII-encoded integer string ``expires_in`` property of `J`.
 
 .. note::
 
@@ -2507,6 +2515,7 @@ Changelog
    :align: left
 
    Date, Description
+   22-10-11, Specify a timeout on Azure IMDS HTTP requests and fix the resource URL
    22-10-05, Remove spec front matter and ``versionadded`` RST macros (since spec version was removed)
    22-09-26, Add behavior for automatic Azure KeyVault credentials for ``kmsProviders``.
    22-09-09, Prohibit ``rewrapManyDataKey`` with libmongocrypt <= 1.5.1.

source/client-side-encryption/tests/README.rst

@@ -2188,3 +2188,130 @@ attached to the virtual machine.
 Expect the key to be successfully created.
 
 .. _Automatic GCP Credentials: ../client-side-encryption.rst#automatic-gcp-credentials
+
+
+18. Azure IMDS Credentials
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Refer: `Automatic Azure Credentials <auto-azure_>`_
+
+.. _auto-azure: ../client-side-encryption.rst#obtaining-an-access-token-for-azure-key-vault
+
+The test cases for IMDS communication are specially designed to not require an
+Azure environment, while still exercising the core of the functionality. The
+design of these test cases encourages an implementation to separate the concerns
+of IMDS communication from the logic of KMS key manipulation. The purpose of
+these test cases is to ensure drivers will behave appropriately regardless of
+the behavior of the IMDS server.
+
+For these IMDS credentials tests, a simple stand-in IMDS-imitating HTTP server
+is available in drivers-evergreen-tools, at ``.evergreen/csfle/fake_azure.py``.
+``fake_azure.py`` is a very simple ``bottle.py`` application. For the easiest
+use, it is recommended to execute it through ``bottle.py`` (which is a sibling
+file in the same directory)::
+
+   python .evergreen/csfle/bottle.py fake_azure:imds
+
+This will run the ``imds`` Bottle application defined in the ``fake_azure``
+Python module. ``bottle.py`` accepts additional command line arguments to
+control the bind host and TCP port (use ``--help`` for more information).
+
+For each test case, follow the process for obtaining the token as outlined in
+the `automatic Azure credentials section <auto-azure_>`_ with the following
+changes:
+
+1. Instead of the standard IMDS TCP endpoint of `169.254.169.254:80`,
+   communicate with the running ``fake_azure`` HTTP server.
+
+2. For each test case, the behavior of the server may be controlled by attaching
+   an additional HTTP header to the sent request: ``X-MongoDB-HTTP-TestParams``.
+
+
+Case 1: Success
+```````````````
+
+Do not set an ``X-MongoDB-HTTP-TestParams`` header.
+
+Upon receiving a response from ``fake_azure``, the driver must decode the
+following information:
+
+1. HTTP status will be ``200 Okay``.
+2. The HTTP body will be a valid JSON string.
+3. The access token will be the string ``"magic-cookie"``.
+4. The expiry duration of the token will be seventy seconds.
+5. The token will have a resource of ``"https://vault.azure.net"``
+
+
+Case 2: Empty JSON
+``````````````````
+
+This case addresses a server returning valid JSON with invalid content.
+
+Set ``X-MongoDB-HTTP-TestParams`` to ``case=empty-json``.
+
+Upon receiving a response:
+
+1. HTTP status will be ``200 Okay``
+2. The HTTP body will be a valid JSON string.
+3. There will be no access token, expiry duration, or resource.
+
+The test case should ensure that this error condition is handled gracefully.
+
+
+Case 3: Bad JSON
+````````````````
+
+This case addresses a server returning malformed JSON.
+
+Set ``X-MongoDB-HTTP-TestParams`` to ``case=bad-json``.
+
+Upon receiving a response:
+
+1. HTTP status will be ``200 Okay``
+2. The response body will contain a malformed JSON string.
+
+The test case should ensure that this error condition is handled gracefully.
+
+
+Case 4: HTTP 404
+````````````````
+
+This case addresses a server returning a "Not Found" response. This is
+documented to occur spuriously within an Azure environment.
+
+Set ``X-MongoDB-HTTP-TestParams`` to ``case=404``.
+
+Upon receiving a response:
+
+1. HTTP status will be ``404 Not Found``.
+2. The response body is unspecified.
+
+The test case should ensure that this error condition is handled gracefully.
+
+
+Case 5: HTTP 500
+````````````````
+
+This case addresses an IMDS server reporting an internal error. This is
+documented to occur spuriously within an Azure environment.
+
+Set ``X-MongoDB-HTTP-TestParams`` to ``case=500``.
+
+Upon receiving a response:
+
+1. HTTP status code will be ``500``.
+2. The response body is unspecified.
+
+The test case should ensure that this error condition is handled gracefully.
+
+
+Case 6: Slow Response
+`````````````````````
+
+This case addresses an IMDS server responding very slowly. Drivers should not
+halt the application waiting on a peer to communicate.
+
+Set ``X-MongoDB-HTTP-TestParams`` to ``case=slow``.
+
+The HTTP response from the ``fake_azure`` server will take at least 1000 seconds
+to complete. The request should fail with a timeout.