Revisit configuration defaults for object storage

[09jvilla]

No description provided.

[09jvilla]

I see this issue as 'checked off' in #367 which implies that it is done, yet this github issue is still open and I don't see a linked PR.
@Logiraptor @pracucci is this still waiting to be worked on, or can we close it (and if so can we link the right PR?).

[pracucci]

I see this issue as 'checked off' in #367

How is #367 related? I think I'm missing what you mean by "checked off".

is this still waiting to be worked on, or can we close it (and if so can we link the right PR?).

I think we still need to check the default configuration for object storages.

[09jvilla]

@pracucci -- Sorry I linked the wrong 367 hehe:
https://github.com/grafana/mimir-squad/issues/367

I was referring to the fact that I see a checkmark next to the line item for this issue in our parent tracker, but that the underlying issue is still open. Sounds like from your comment, the checkmark should be removed and the work still needs to be done.

See screenshot - generally I would only expect a check mark next to an issue that has been closed.

@pracucci since this issue as is is quite broad, can you provide a little more description? What should an engineer be looking for when revisiting the defaults? Are we trying to enforce certain specific values? If so, which ones?

[pracucci]

@pracucci since this issue as is is quite broad, can you provide a little more description? What should an engineer be looking for when revisiting the defaults? Are we trying to enforce certain specific values? If so, which ones?

I have no idea what we wanna do here :) It wasn't added by me. I think it was by @simonswine .

[simonswine]

When I added this to the epic issue list, I mostly meant this to be covering the switch from -storage.backend=aws to -storage.backend=filesystem which has happened in #833. Sorry for the confusion.

I am otherwise not too much aware what changes we should make to those flags. I have never used swift and the other defaults seem sensible to me:

  -blocks-storage.azure.account-key value
    	Azure storage account key
  -blocks-storage.azure.account-name string
    	Azure storage account name
  -blocks-storage.azure.container-name string
    	Azure storage container name
  -blocks-storage.azure.endpoint-suffix string
    	Azure storage endpoint suffix without schema. The account name will be prefixed to this value to create the FQDN. If set to empty string, default endpoint suffix is used.
  -blocks-storage.azure.max-retries int
    	Number of retries for recoverable errors (default 20)
  -blocks-storage.azure.msi-resource string
    	If set, this URL is used instead of https://<storage-account-name>.<endpoint-suffix> for obtaining ServicePrincipalToken from MSI.
  -blocks-storage.azure.user-assigned-id string
    	User assigned identity. If empty, then System assigned identity is used.
  -blocks-storage.gcs.bucket-name string
    	GCS bucket name
  -blocks-storage.gcs.service-account value
    	JSON representing either a Google Developers Console client_credentials.json file or a Google Developers service account key file. If empty, fallback to Google default logic.
  -blocks-storage.s3.access-key-id string
    	S3 access key ID
  -blocks-storage.s3.bucket-name string
    	S3 bucket name
  -blocks-storage.s3.endpoint string
    	The S3 bucket endpoint. It could be an AWS S3 endpoint listed at https://docs.aws.amazon.com/general/latest/gr/s3.html or the address of an S3-compatible service in hostname:port format.
  -blocks-storage.s3.expect-continue-timeout duration
    	The time to wait for a server's first response headers after fully writing the request headers if the request has an Expect header. 0 to send the request body immediately. (default 1s)
  -blocks-storage.s3.http.idle-conn-timeout duration
    	The time an idle connection will remain idle before closing. (default 1m30s)
  -blocks-storage.s3.http.insecure-skip-verify
    	If the client connects to S3 via HTTPS and this option is enabled, the client will accept any certificate and hostname.
  -blocks-storage.s3.http.response-header-timeout duration
    	The amount of time the client will wait for a servers response headers. (default 2m0s)
  -blocks-storage.s3.insecure
    	If enabled, use http:// for the S3 endpoint instead of https://. This could be useful in local dev/test environments while using an S3-compatible backend storage, like Minio.
  -blocks-storage.s3.max-connections-per-host int
    	Maximum number of connections per host. 0 means no limit.
  -blocks-storage.s3.max-idle-connections int
    	Maximum number of idle (keep-alive) connections across all hosts. 0 means no limit. (default 100)
  -blocks-storage.s3.max-idle-connections-per-host int
    	Maximum number of idle (keep-alive) connections to keep per-host. If 0, a built-in default value is used. (default 100)
  -blocks-storage.s3.region string
    	S3 region. If unset, the client will issue a S3 GetBucketLocation API call to autodetect it.
  -blocks-storage.s3.secret-access-key value
    	S3 secret access key
  -blocks-storage.s3.signature-version string
    	The signature version to use for authenticating against S3. Supported values are: v4, v2. (default "v4")
  -blocks-storage.s3.sse.kms-encryption-context string
    	KMS Encryption Context used for object encryption. It expects JSON formatted string.
  -blocks-storage.s3.sse.kms-key-id string
    	KMS Key ID used to encrypt objects in S3
  -blocks-storage.s3.sse.type string
    	Enable AWS Server Side Encryption. Supported values: SSE-KMS, SSE-S3.
  -blocks-storage.s3.tls-handshake-timeout duration
    	Maximum time to wait for a TLS handshake. 0 means no limit. (default 10s)
  -blocks-storage.swift.auth-url string
    	OpenStack Swift authentication URL
  -blocks-storage.swift.auth-version int
    	OpenStack Swift authentication API version. 0 to autodetect.
  -blocks-storage.swift.connect-timeout duration
    	Time after which a connection attempt is aborted. (default 10s)
  -blocks-storage.swift.container-name string
    	Name of the OpenStack Swift container to put chunks in.
  -blocks-storage.swift.domain-id string
    	OpenStack Swift user's domain ID.
  -blocks-storage.swift.domain-name string
    	OpenStack Swift user's domain name.
  -blocks-storage.swift.max-retries int
    	Max retries on requests error. (default 3)
  -blocks-storage.swift.password string
    	OpenStack Swift API key.
  -blocks-storage.swift.project-domain-id string
    	ID of the OpenStack Swift project's domain (v3 auth only), only needed if it differs the from user domain.
  -blocks-storage.swift.project-domain-name string
    	Name of the OpenStack Swift project's domain (v3 auth only), only needed if it differs from the user domain.
  -blocks-storage.swift.project-id string
    	OpenStack Swift project ID (v2,v3 auth only).
  -blocks-storage.swift.project-name string
    	OpenStack Swift project name (v2,v3 auth only).
  -blocks-storage.swift.region-name string
    	OpenStack Swift Region to use (v2,v3 auth only).
  -blocks-storage.swift.request-timeout duration
    	Time after which an idle request is aborted. The timeout watchdog is reset each time some data is received, so the timeout triggers after X time no data is received on a request. (default 5s)
  -blocks-storage.swift.user-domain-id string
    	OpenStack Swift user's domain ID.
  -blocks-storage.swift.user-domain-name string
    	OpenStack Swift user's domain name.
  -blocks-storage.swift.user-id string
    	OpenStack Swift user ID.
  -blocks-storage.swift.username string
    	OpenStack Swift username.

[pracucci]

In the Cortex project we had some people using Swift, and it was working for them, so I wouldn't touch it :)

[simonswine]

I think from my point of view we can close this then.

[Logiraptor]

Agreed.