CA file #323

mmatczuk · 2023-07-31T14:53:11Z

Fixes #264

mmatczuk · 2023-07-31T14:53:44Z

I still need to add e2e test for it.

what-the-diff · 2023-07-31T14:54:05Z

PR Summary

Inclusion of a new linter
A linter called 'gochecknoglobals' was added to the configuration file in charge of quality checks (.golangci.yml) to ensure the code is consistent and following best practices.
New option for pac flag
A new functionality has been added in bind/flag.go that enhances the use of the --pac flag allowing the use of data URI scheme.
Improvements in timeout handling
In the http_proxy.go file, timeout management of the HTTP transport configuration has been revamped to provide better control.
Additional fields in TLSClientConfig
TLSClientConfig (found in tls.go) has been enriched with new fields, 'HandshakeTimeout' and 'CAFiles'. These are meant for controlling the maximum time for a TLS handshake and specifying locations of CA certificate files respectively.
New Data-Handling Functions
Two new utility functions ReadURLString and ReadFileOrBase64 have been introduced in the forwarder package. These functions deal with reading various types of data and making them available in appropriate formats.
Adding Unit Tests
New tests have been added for verifying the correct operations of the ReadURLData and ReadFileOrBase64 functions.
New Structure to Represent Server-Side Connections
A new struct TLSServerConfig has been introduced to represent and control TLS configuration for server-side connections.
Efficient Certificate and Key Loading
The loadX509KeyPair function has been added in addition to replacing the CertFile and KeyFile fields in the TLSServerConfig struct with a call to loadCertificate. This is intended to improve and simplify the process of loading a TLS certificate and private key from files.
Broad replacement of ReadURL usage
Several functions in the forwarder package have been updated to use the new ReadURLString function instead of ReadURL, showing our continuous efforts in refining and improving our codebase.

Choraden · 2023-08-01T07:39:47Z

I would squash bind: update PAC with data scheme with readurl: add support for reading data scheme. The bind commit gives no info why, without the context from the other.

Choraden · 2023-08-01T07:42:41Z

How about we also extract TLSServerConfig to separate bind function?

mmatczuk · 2023-08-01T15:00:33Z

I would squash bind: update PAC with data scheme with readurl: add support for reading data scheme. The bind commit gives no info why, without the context from the other.

The changes are unrelated, I'd update the commit msg.

mmatczuk · 2023-08-01T15:02:29Z

I squashed it.

Choraden · 2023-08-02T07:53:13Z

e2e/forwarder/service.go

+			"./certs/"+s.Name+".key:/etc/forwarder/private/"+s.Name+".key:ro",
+		)
+	}
+
 	return s
 }

 func (s *Service) WithUpstream(name, protocol string) *Service {
 	s.Environment["FORWARDER_PROXY"] = protocol + "://" + name + ":3128"
 	if protocol == "https" {


Let's preserve the same condition as in WithProtocol.

if protocol != "http" {

it's https and h2 but sure I can change it.

Choraden · 2023-08-02T07:56:36Z

Great job removingInsecure option from e2e test. How about we mention the certificates generation in e2e/README?

…ed ReadURLString

Reading data scheme is restricted to base64 only.

…ded string

The field order is changed to reflect the field order in http.Transport.

mmatczuk · 2023-08-02T10:19:41Z

I updated the generated certs

e2e/certs: CA signed certificate generation

The script generates CA and certificate key pair for each host name used in tests.
We generate X509v3 certificates based on ecdsa-with-SHA256 signing alg.

For developer convenience the certificates also work with localhost DNS name.

Sample generated key

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            b1:b4:50:5f:59:1c:7b:9a
    Signature Algorithm: ecdsa-with-SHA256
        Issuer: C=US, O=Sauce Labs Inc.
        Validity
            Not Before: Aug  2 10:00:27 2023 GMT
            Not After : Aug  1 10:00:27 2024 GMT
        Subject: C=US, O=Sauce Labs Inc., CN=httpbin
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:b4:80:60:3a:eb:00:e7:e0:ba:59:80:bf:c7:b8:
                    9c:f3:48:2b:ab:39:64:e0:8f:a3:ac:79:58:cf:2b:
                    ec:59:71:95:c0:11:a8:ca:b6:6b:a4:91:2f:fc:b0:
                    0c:ba:92:f8:f9:7e:d5:ba:cf:32:aa:c9:80:fd:7b:
                    5e:ad:1e:4b:7a
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                keyid:10:A7:85:A5:DD:41:80:3F:E5:FE:A3:F8:E9:F1:05:81:BE:B0:16:99

            X509v3 Subject Alternative Name:
                DNS:localhost, DNS:httpbin
            X509v3 Key Usage:
                Digital Signature, Key Encipherment
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:21:00:d2:05:be:ea:10:20:62:3d:13:94:cd:cc:c8:
         37:93:45:1d:73:6a:45:e1:54:20:2a:7c:d3:66:27:93:f9:83:
         3b:02:20:05:e8:a9:94:a0:2f:6a:43:d8:f0:5e:de:33:e5:6b:
         15:26:f7:8e:93:3c:25:81:42:74:8f:e6:db:38:a2:83:b0

The script generates CA and certificate key pair for each host name used in tests. We generate X509v3 certificates based on ecdsa-with-SHA256 signing alg. For developer convenience the certificates also work with localhost DNS name. Sample generated key Certificate: Data: Version: 3 (0x2) Serial Number: b1:b4:50:5f:59:1c:7b:9a Signature Algorithm: ecdsa-with-SHA256 Issuer: C=US, O=Sauce Labs Inc. Validity Not Before: Aug 2 10:00:27 2023 GMT Not After : Aug 1 10:00:27 2024 GMT Subject: C=US, O=Sauce Labs Inc., CN=httpbin Subject Public Key Info: Public Key Algorithm: id-ecPublicKey Public-Key: (256 bit) pub: 04:b4:80:60:3a:eb:00:e7:e0:ba:59:80:bf:c7:b8: 9c:f3:48:2b:ab:39:64:e0:8f:a3:ac:79:58:cf:2b: ec:59:71:95:c0:11:a8:ca:b6:6b:a4:91:2f:fc:b0: 0c:ba:92:f8:f9:7e:d5:ba:cf:32:aa:c9:80:fd:7b: 5e:ad:1e:4b:7a ASN1 OID: prime256v1 NIST CURVE: P-256 X509v3 extensions: X509v3 Basic Constraints: critical CA:FALSE X509v3 Authority Key Identifier: keyid:10:A7:85:A5:DD:41:80:3F:E5:FE:A3:F8:E9:F1:05:81:BE:B0:16:99 X509v3 Subject Alternative Name: DNS:localhost, DNS:httpbin X509v3 Key Usage: Digital Signature, Key Encipherment Signature Algorithm: ecdsa-with-SHA256 30:45:02:21:00:d2:05:be:ea:10:20:62:3d:13:94:cd:cc:c8: 37:93:45:1d:73:6a:45:e1:54:20:2a:7c:d3:66:27:93:f9:83: 3b:02:20:05:e8:a9:94:a0:2f:6a:43:d8:f0:5e:de:33:e5:6b: 15:26:f7:8e:93:3c:25:81:42:74:8f:e6:db:38:a2:83:b0

The curl command has --cacert <file> and --capath <dir> flags. Forwarder implement the file based approach but supports specifying it multiple times. With this patch the flag is --cacert-file <path or base64> (env FORWARDER_CACERT_FILE). We preserve the file suffix for automatic file marking and compatibility with other Forwarder flags.

mmatczuk force-pushed the mmt/url_data branch 5 times, most recently from 8f78354 to 9559fc1 Compare August 1, 2023 15:47

Choraden reviewed Aug 2, 2023

View reviewed changes

mmatczuk added 14 commits August 2, 2023 10:45

golangci: disable gochecknoglobals in tests

1866909

fileurl: add test case for data scheme

bce2ff6

readurl: change signature of ReadURL to return []byte and add dedicat…

9413bfb

…ed ReadURLString

readurl: add support for reading data scheme

5d946f2

Reading data scheme is restricted to base64 only.

readurl: add ReadFileOrBase64 as a base 64 enabled os.ReadFile wrapper

4694023

tls: add support for loading TLS certificate and key from base64 enco…

ae2079b

…ded string

tls: split TLSConfig to TLSClientConfig and TLSServerConfig

90cf3e2

tls: refactor LoadCertificateFromTLSConfig to ConfigureTLSConfig method

bff3ecc

tls: add CAFiles to TLSClientConfig

29a3f16

http_transport: add TLSHandshakeTimeout to TLSClientConfig

98f9849

The field order is changed to reflect the field order in http.Transport.

bind: extract TLSClientConfig from HTTPTransportConfig

50c920e

bind: add ca-file flag

047a37b

bind: extract TLSServerConfig from HTTPServerConfig

7ab4b07

bind: remove tls prefix from tls-[cert,key]-file

18a2dfa

mmatczuk force-pushed the mmt/url_data branch from 9559fc1 to 4e8b33d Compare August 2, 2023 10:18

mmatczuk force-pushed the mmt/url_data branch from acd6095 to 63d12d5 Compare August 2, 2023 10:33

mmatczuk force-pushed the mmt/url_data branch 4 times, most recently from f9cf201 to 9abb5b8 Compare August 2, 2023 11:01

mmatczuk added 5 commits August 2, 2023 13:09

e2e: use generated certificates instead of insecure

0da0f79

gh: update go action to generate certificates

64d70dc

gh: on CI failure dump docker-compose.yaml

fad0324

mmatczuk force-pushed the mmt/url_data branch from 9abb5b8 to fad0324 Compare August 2, 2023 11:19

http_proxy: log when using custom root CA certificates

9ab4444

mmatczuk force-pushed the mmt/url_data branch from 40ab843 to 9ab4444 Compare August 2, 2023 12:01

http_proxy: log when using custom root CA certificates

2ec2171

mmatczuk merged commit b5fabbf into main Aug 2, 2023
3 checks passed

mmatczuk deleted the mmt/url_data branch August 2, 2023 12:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CA file #323

CA file #323

mmatczuk commented Jul 31, 2023

mmatczuk commented Jul 31, 2023

what-the-diff bot commented Jul 31, 2023

Choraden commented Aug 1, 2023

Choraden commented Aug 1, 2023

mmatczuk commented Aug 1, 2023

mmatczuk commented Aug 1, 2023

Choraden Aug 2, 2023

Choraden Aug 2, 2023

mmatczuk Aug 2, 2023

Choraden commented Aug 2, 2023

mmatczuk commented Aug 2, 2023

CA file #323

CA file #323

Conversation

mmatczuk commented Jul 31, 2023

mmatczuk commented Jul 31, 2023

what-the-diff bot commented Jul 31, 2023

PR Summary

Choraden commented Aug 1, 2023

Choraden commented Aug 1, 2023

mmatczuk commented Aug 1, 2023

mmatczuk commented Aug 1, 2023

Choraden Aug 2, 2023

Choose a reason for hiding this comment

Choraden Aug 2, 2023

Choose a reason for hiding this comment

mmatczuk Aug 2, 2023

Choose a reason for hiding this comment

Choraden commented Aug 2, 2023

mmatczuk commented Aug 2, 2023