Add new http transport params to azure, gcs, s3 storage drivers

This enables the azure, gcs, and s3 storage driver users to disable keep alives `"disablekeepalives"`, disable idle connections `"disableidleconnections"`, and enable tracing for http requests `"enableotelhttp"`.

Fixes distribution#4159

Signed-off-by: T Van Doren <108194999+tvdfly@users.noreply.github.com>

docs/content/storage-drivers/azure.md

@@ -16,6 +16,9 @@ An implementation of the `storagedriver.StorageDriver` interface which uses [Mic
 | `realm`                            | no       | Domain name suffix for the Storage Service API endpoint. For example realm for "Azure in China" would be `core.chinacloudapi.cn` and realm for "Azure Government" would be `core.usgovcloudapi.net`. By default, this is `core.windows.net`.                        |
 | `copy_status_poll_max_retry`       | no       | Max retry number for polling of copy operation status. Retries use a simple backoff algorithm where each retry number is multiplied by `copy_status_poll_delay`, and this number is used as the delay. Set to -1 to disable retries and abort if the copy does not complete immediately. Defaults to 5.                |
 | `copy_status_poll_delay`            | no       | Time to wait between retries for polling of copy operation status. This time is multiplied by N on each retry, where N is the retry number. Defaults to 100ms |
+| `disablekeepalives` | no | Disables HTTP keep-alives on the HTTP tranport when `true`. Each connection to the server will be used for a single HTTP request. The default is `false`. |
+| `disableidleconnections` | no | Disables idle HTTP connections on the HTTP transport when `true`. The default is `false`. |
+| `enableotelhttp` | no | Wraps the HTTP transport with [otelhttp](https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp) when `true`. This will generate spans for each HTTP request. The default is `false`. |
 
 
 ## Related information

docs/content/storage-drivers/gcs.md

@@ -14,6 +14,9 @@ An implementation of the `storagedriver.StorageDriver` interface which uses Goog
 | `keyfile`  | no | A private service account key file in JSON format used for [Service Account Authentication](https://cloud.google.com/storage/docs/authentication#service_accounts). |
 | `rootdirectory`  | no | The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root). If a prefix is used, the path `bucketname/<prefix>` has to be pre-created before starting the registry. The prefix is applied to all Google Cloud Storage keys to allow you to segment data in your bucket if necessary.|
 | `chunksize`  | no (default 5242880) | This is the chunk size used for uploading large blobs, must be a multiple of 256*1024. |
+| `disablekeepalives` | no | Disables HTTP keep-alives on the HTTP tranport when `true`. Each connection to the server will be used for a single HTTP request. The default is `false`. |
+| `disableidleconnections` | no | Disables idle HTTP connections on the HTTP transport when `true`. The default is `false`. |
+| `enableotelhttp` | no | Wraps the HTTP transport with [otelhttp](https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp) when `true`. This will generate spans for each HTTP request. The default is `false`. |
 
 {{< hint type=note >}}
 Instead of a key file you can use [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials).

docs/content/storage-drivers/s3.md

@@ -21,11 +21,17 @@ Amazon S3 or S3 compatible services for object storage.
 | `keyid`  | no | Optional KMS key ID to use for encryption (encrypt must be true, or this parameter is ignored). The default is `none`. |
 | `secure`  | no | Indicates whether to use HTTPS instead of HTTP. A boolean value. The default is `true`. |
 | `skipverify`  | no  | Skips TLS verification when the value is set to `true`. The default is `false`. |
+| `disablekeepalives` | no | Disables HTTP keep-alives on the HTTP tranport when `true`. Each connection to the server will be used for a single HTTP request. The default is `false`. |
+| `disableidleconnections` | no | Disables idle HTTP connections on the HTTP transport when `true`. The default is `false`. |
 | `v4auth`  | no | Indicates whether the registry uses Version 4 of AWS's authentication. The default is `true`. |
 | `chunksize`  | no | The S3 API requires multipart upload chunks to be at least 5MB. This value should be a number that is larger than 5 * 1024 * 1024.|
 | `rootdirectory`  | no | This is a prefix that is applied to all S3 keys to allow you to segment data in your bucket if necessary. |
 | `storageclass`  | no | The S3 storage class applied to each registry file. The default is `STANDARD`. |
 | `objectacl`  | no | The S3 Canned ACL for objects. The default value is "private". |
+| `usedualstack` | no | Use AWS S3 dual-stack endpoints, which support both IPv6 and IPv4, when set to `true`. The default is `false`. |
+| `multipartcombinesmallpart` | no | Combines small pending uploads with the ready part when `true`. The default is `true`. |
+| `accelerate` | no | Use AWS S3 Transfer Acceleration endpoints when set to `true`. The default is `false`. |
+| `enableotelhttp` | no | Wraps the HTTP transport with [otelhttp](https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp) when `true`. This will generate spans for each HTTP request. The default is `false`. |
 | `loglevel`  | no | The log level for the S3 client. The default value is `off`. |
 
 > **Note** You can provide empty strings for your access and secret keys to run the driver

go.mod

@@ -27,6 +27,7 @@ require (
 	github.com/redis/go-redis/v9 v9.1.0
 	github.com/sirupsen/logrus v1.8.1
 	github.com/spf13/cobra v1.6.1
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1
 	golang.org/x/crypto v0.14.0
 	golang.org/x/oauth2 v0.7.0
 	google.golang.org/api v0.114.0
@@ -45,13 +46,13 @@ require (
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/cyphar/filepath-securejoin v0.2.4 // indirect
 	github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
-	github.com/felixge/httpsnoop v1.0.1 // indirect
-	github.com/go-logr/logr v1.2.4 // indirect
+	github.com/felixge/httpsnoop v1.0.4 // indirect
+	github.com/go-logr/logr v1.3.0 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
 	github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
 	github.com/golang/protobuf v1.5.3 // indirect
-	github.com/google/go-cmp v0.5.9 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
 	github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
 	github.com/googleapis/gax-go/v2 v2.7.1 // indirect
 	github.com/hashicorp/golang-lru/v2 v2.0.5 // indirect
@@ -69,9 +70,9 @@ require (
 	github.com/redis/go-redis/extra/rediscmd/v9 v9.0.5 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	go.opencensus.io v0.24.0 // indirect
-	go.opentelemetry.io/otel v1.16.0 // indirect
-	go.opentelemetry.io/otel/metric v1.16.0 // indirect
-	go.opentelemetry.io/otel/trace v1.16.0 // indirect
+	go.opentelemetry.io/otel v1.21.0 // indirect
+	go.opentelemetry.io/otel/metric v1.21.0 // indirect
+	go.opentelemetry.io/otel/trace v1.21.0 // indirect
 	golang.org/x/net v0.17.0 // indirect
 	golang.org/x/sys v0.13.0 // indirect
 	golang.org/x/text v0.13.0 // indirect

go.sum

@@ -106,8 +106,9 @@ github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymF
 github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
 github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
-github.com/felixge/httpsnoop v1.0.1 h1:lvB5Jl89CsZtGIWuTcDM1E/vkVs49/Ml7JJe07l8SPQ=
 github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
+github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
+github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
 github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
 github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
@@ -120,8 +121,8 @@ github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9
 github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
 github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
 github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
-github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ=
-github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY=
+github.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
@@ -171,8 +172,8 @@ github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
-github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
 github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no=
 github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
@@ -314,7 +315,7 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
-github.com/stretchr/testify v1.8.3 h1:RP3t2pwF7cMEbC1dqtB6poj3niw/9gnV4Cjg5oW5gtY=
+github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
 github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -326,13 +327,15 @@ go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
 go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
 go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
-go.opentelemetry.io/otel v1.16.0 h1:Z7GVAX/UkAXPKsy94IU+i6thsQS4nb7LviLpnaNeW8s=
-go.opentelemetry.io/otel v1.16.0/go.mod h1:vl0h9NUa1D5s1nv3A5vZOYWn8av4K8Ml6JDeHrT/bx4=
-go.opentelemetry.io/otel/metric v1.16.0 h1:RbrpwVG1Hfv85LgnZ7+txXioPDoh6EdbZHo26Q3hqOo=
-go.opentelemetry.io/otel/metric v1.16.0/go.mod h1:QE47cpOmkwipPiefDwo2wDzwJrlfxxNYodqc4xnGCo4=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 h1:aFJWCqJMNjENlcleuuOkGAPH82y0yULBScfXcIEdS24=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1/go.mod h1:sEGXWArGqc3tVa+ekntsN65DmVbVeW+7lTKTjZF3/Fo=
+go.opentelemetry.io/otel v1.21.0 h1:hzLeKBZEL7Okw2mGzZ0cc4k/A7Fta0uoPgaJCr8fsFc=
+go.opentelemetry.io/otel v1.21.0/go.mod h1:QZzNPQPm1zLX4gZK4cMi+71eaorMSGT3A4znnUvNNEo=
+go.opentelemetry.io/otel/metric v1.21.0 h1:tlYWfeo+Bocx5kLEloTjbcDwBuELRrIFxwdQ36PlJu4=
+go.opentelemetry.io/otel/metric v1.21.0/go.mod h1:o1p3CA8nNHW8j5yuQLdc1eeqEaPfzug24uvsyIEJRWM=
 go.opentelemetry.io/otel/sdk v1.16.0 h1:Z1Ok1YsijYL0CSJpHt4cS3wDDh7p572grzNrBMiMWgE=
-go.opentelemetry.io/otel/trace v1.16.0 h1:8JRpaObFoW0pxuVPapkgH8UhHQj+bJW8jJsCZEu5MQs=
-go.opentelemetry.io/otel/trace v1.16.0/go.mod h1:Yt9vYq1SdNz3xdjZZK7wcXv1qv2pwLkqr2QVwea0ef0=
+go.opentelemetry.io/otel/trace v1.21.0 h1:WD9i5gzvoUPuXIXH24ZNBudiarZDKuekPqi/E8fpfLc=
+go.opentelemetry.io/otel/trace v1.21.0/go.mod h1:LGbsEB0f9LGjN+OZaQQ26sohbOmiMR+BaslueVtS/qQ=
 golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=

registry/storage/driver/azure/azure_auth.go

@@ -2,6 +2,7 @@ package azure
 
 import (
 	"context"
+	"net/http"
 	"sync"
 	"time"
 
@@ -12,6 +13,7 @@ import (
 	"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/container"
 	"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/sas"
 	"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/service"
+	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 )
 
 const (
@@ -45,6 +47,14 @@ type azureClient struct {
 	signer    signer
 }
 
+type azureTransporter struct {
+	roundTripper http.RoundTripper
+}
+
+func (a *azureTransporter) Do(req *http.Request) (*http.Response, error) {
+	return a.roundTripper.RoundTrip(req)
+}
+
 func newAzureClient(params *Parameters) (*azureClient, error) {
 	if params.AccountKey != "" {
 		cred, err := azblob.NewSharedKeyCredential(params.AccountName, params.AccountKey)
@@ -76,7 +86,39 @@ func newAzureClient(params *Parameters) (*azureClient, error) {
 		return nil, err
 	}
 
-	client, err := azblob.NewClient(params.ServiceURL, cred, nil)
+	httpTransportModified := false
+	httpTransport := &http.Transport{}
+
+	if params.DisableKeepAlives {
+		httpTransport.DisableKeepAlives = true
+		httpTransportModified = true
+	}
+
+	if params.DisableIdleConnections {
+		httpTransport.MaxIdleConnsPerHost = -1
+		httpTransportModified = true
+	}
+
+	var httpRoundTripper http.RoundTripper = httpTransport
+
+	if params.EnableOtelHttp {
+		httpRoundTripper = otelhttp.NewTransport(httpTransport)
+		httpTransportModified = true
+	}
+
+	var azClientOpts *azblob.ClientOptions
+
+	if httpTransportModified {
+		azClientOpts = &azblob.ClientOptions{
+			ClientOptions: azcore.ClientOptions{
+				Transport: &azureTransporter{
+					roundTripper: httpRoundTripper,
+				},
+			},
+		}
+	}
+
+	client, err := azblob.NewClient(params.ServiceURL, cred, azClientOpts)
 	if err != nil {
 		return nil, err
 	}

registry/storage/driver/azure/parser.go

@@ -31,6 +31,9 @@ type Parameters struct {
 	ServiceURL             string      `mapstructure:"serviceurl"`
 	CopyStatusPollMaxRetry int         `mapstructure:"copy_status_poll_max_retry"`
 	CopyStatusPollDelay    string      `mapstructure:"copy_status_poll_delay"`
+	DisableKeepAlives      bool        `mapstructure:"disablekeepalives"`
+	DisableIdleConnections bool        `mapstructure:"disableidleconnections"`
+	EnableOtelHttp         bool        `mapstructure:"enableotelhttp"`
 }
 
 func NewParameters(parameters map[string]interface{}) (*Parameters, error) {

registry/storage/driver/gcs/gcs.go

@@ -36,6 +36,7 @@ import (
 	"github.com/distribution/distribution/v3/registry/storage/driver/base"
 	"github.com/distribution/distribution/v3/registry/storage/driver/factory"
 	"github.com/sirupsen/logrus"
+	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 	"golang.org/x/oauth2"
 	"golang.org/x/oauth2/google"
 	"golang.org/x/oauth2/jwt"
@@ -154,6 +155,85 @@ func FromParameters(ctx context.Context, parameters map[string]interface{}) (sto
 		}
 	}
 
+	enableOtelHttpParam := false
+	enableOtelHttp := parameters["enableotelhttp"]
+	switch enableOtelHttp := enableOtelHttp.(type) {
+	case string:
+		b, err := strconv.ParseBool(enableOtelHttp)
+		if err != nil {
+			return nil, fmt.Errorf("the enableotelhttp parameter should be a boolean")
+		}
+		enableOtelHttpParam = b
+	case bool:
+		enableOtelHttpParam = enableOtelHttp
+	case nil:
+		// do nothing
+	default:
+		return nil, fmt.Errorf("the enableotelhttp parameter should be a boolean")
+	}
+
+	disableKeepAlivesParam := false
+	disableKeepAlives := parameters["disablekeepalives"]
+	switch disableKeepAlives := disableKeepAlives.(type) {
+	case string:
+		b, err := strconv.ParseBool(disableKeepAlives)
+		if err != nil {
+			return nil, fmt.Errorf("the disablekeepalives parameter should be a boolean")
+		}
+		disableKeepAlivesParam = b
+	case bool:
+		disableKeepAlivesParam = disableKeepAlives
+	case nil:
+		// do nothing
+	default:
+		return nil, fmt.Errorf("the disablekeepalives parameter should be a boolean")
+	}
+
+	disableIdleConnectionsParam := false
+	disableIdleConnections := parameters["disableidleconnections"]
+	switch disableIdleConnections := disableIdleConnections.(type) {
+	case string:
+		b, err := strconv.ParseBool(disableIdleConnections)
+		if err != nil {
+			return nil, fmt.Errorf("the disableidleconnections parameter should be a boolean")
+		}
+		disableIdleConnectionsParam = b
+	case bool:
+		disableIdleConnectionsParam = disableIdleConnections
+	case nil:
+		// do nothing
+	default:
+		return nil, fmt.Errorf("the disableidleconnections parameter should be a boolean")
+	}
+
+	httpTransportModified := false
+	httpTransport := &http.Transport{}
+
+	if disableKeepAlivesParam {
+		httpTransport.DisableKeepAlives = true
+		httpTransportModified = true
+	}
+
+	if disableIdleConnectionsParam {
+		httpTransport.MaxIdleConnsPerHost = -1
+		httpTransportModified = true
+	}
+
+	var httpRoundTripper http.RoundTripper = httpTransport
+
+	if enableOtelHttpParam {
+		httpRoundTripper = otelhttp.NewTransport(httpTransport)
+		httpTransportModified = true
+	}
+
+	var clientOpts []option.ClientOption
+
+	if httpTransportModified {
+		clientOpts = append(clientOpts, option.WithHTTPClient(&http.Client{
+			Transport: httpRoundTripper,
+		}))
+	}
+
 	var ts oauth2.TokenSource
 	jwtConf := new(jwt.Config)
 	var err error
@@ -168,7 +248,7 @@ func FromParameters(ctx context.Context, parameters map[string]interface{}) (sto
 			return nil, err
 		}
 		ts = jwtConf.TokenSource(ctx)
-		gcs, err = storage.NewClient(ctx, option.WithCredentialsFile(fmt.Sprint(keyfile)))
+		gcs, err = storage.NewClient(ctx, append(clientOpts, option.WithCredentialsFile(fmt.Sprint(keyfile)))...)
 		if err != nil {
 			return nil, err
 		}
@@ -197,7 +277,7 @@ func FromParameters(ctx context.Context, parameters map[string]interface{}) (sto
 			return nil, err
 		}
 		ts = jwtConf.TokenSource(ctx)
-		gcs, err = storage.NewClient(ctx, option.WithCredentialsJSON(data))
+		gcs, err = storage.NewClient(ctx, append(clientOpts, option.WithCredentialsJSON(data))...)
 		if err != nil {
 			return nil, err
 		}
@@ -210,7 +290,7 @@ func FromParameters(ctx context.Context, parameters map[string]interface{}) (sto
 		if err != nil {
 			return nil, err
 		}
-		gcs, err = storage.NewClient(ctx)
+		gcs, err = storage.NewClient(ctx, clientOpts...)
 		if err != nil {
 			return nil, err
 		}
@@ -221,12 +301,18 @@ func FromParameters(ctx context.Context, parameters map[string]interface{}) (sto
 		return nil, fmt.Errorf("maxconcurrency config error: %s", err)
 	}
 
+	oauthClient := oauth2.NewClient(ctx, ts)
+
+	if enableOtelHttpParam {
+		oauthClient.Transport = otelhttp.NewTransport(oauthClient.Transport)
+	}
+
 	params := driverParameters{
 		bucket:         fmt.Sprint(bucket),
 		rootDirectory:  fmt.Sprint(rootDirectory),
 		email:          jwtConf.Email,
 		privateKey:     jwtConf.PrivateKey,
-		client:         oauth2.NewClient(ctx, ts),
+		client:         oauthClient,
 		chunkSize:      chunkSize,
 		maxConcurrency: maxConcurrency,
 		gcs:            gcs,