cdc: Allow webhook sink to provide client certificates to the remote webhook server

[sherman-grewal]

Summary

Allow the webhook sink to provide client certificates to the remote webhook server.

Resolves #74230

Testing

1. Install certstrap to help with creating the CA and other required certificates

$ brew install certstrap

2. Create a CA

$ certstrap init --common-name "ExampleCA"
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Created out/ExampleCA.key
Created out/ExampleCA.crt
Created out/ExampleCA.crl

3. Create the certificate for the server

$ certstrap request-cert --domain  "localhost"
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Created out/localhost.key
Created out/localhost.csr

4. Create the certificate for the client

$ certstrap request-cert --domain  "client"
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Created out/client.key
Created out/client.csr

5. Sign the certificate for the server

$ certstrap sign localhost --CA ExampleCA
Created out/localhost.crt from out/localhost.csr signed by out/ExampleCA.key

6. Sign the certificate for the client

$ certstrap sign client --CA ExampleCA 
Created out/client.crt from out/client.csr signed by out/ExampleCA.key

7. Pull the server code in this file, and then run the following in your console

$ go build server.go
$ ./server -host "localhost" -srvcert "<path-to-server-cert>" -cacert "<path-to-ca-cert>" \
-srvkey "<path-to-server-key>" -port 3000 -certopt 4

8. Follow the steps to creating a webhook sink here, but use the following SQL command to create the changefeed

CREATE CHANGEFEED
    FOR TABLE movr.vehicles
    INTO 'webhook-https://localhost:3000?insecure_tls_skip_verify=true&client_cert=<client-cert>&client_key=<client-key>'
    WITH updated;

Note: The client_cert and client_key must all be passed directly as base64-encoded values

Release notes (enterprise change): Client certificates may now be provided for the webhook changefeed sink.

[cockroach-teamcity]

This change is 

[miretskiy]

Reviewed all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB and @sherman-grewal)

---

-- commits, line 5 at r1:
I think you need to amend your change to include some some additional information.
In particular, we should definitely have a release note here.

---

pkg/ccl/changefeedccl/sink_webhook.go, line 362 at r1 (raw file):

	if err := u.decodeBase64(changefeedbase.SinkParamClientKey, &dialConfig.clientKey); err != nil {
		return nil, err
	}

Can we expand e.g. webhookFeedFactory (in testfeed_test.go) to also support these params?
For example, we could, perhaps add an option to webhookTestWithOptions function to also specify if we want "secure" connection, and have webhookFeedFactory start a secure http server for message ingestion?

[HonoreDB]

Reviewed 6 of 6 files at r2.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB and @sherman-grewal)

[sherman-grewal]

bors r+

[craig]

Build succeeded:

• GitHub CI (Cockroach)