feat(minio): support for upload to proxied minio

[erikmartino]

Motivation:

In Kubernetes, if Minio is proxied, the generated presigned upload address could look something like

http://minio:9000/{bucket}/{key}?{parameters}

However that address is not accesssible outside the cluster, outside the cluster it should be for example

https://minio.localtest.me/{bucket}/{key}?{parameters}

Read URLs translation is already supported using the MINIO_READ_URL_PREFIX environment variable.

Modifications:

Introduce an MINIO_UPLOAD_URL_PREFIX as well. Given MINIO_UPLOAD_URL_PREFIX=https://minio.localtest.me/{bucket} it will translate the presigned URL http://minio:9000/{bucket}/{key}?{parameters} to the externally accessible https://minio.localtest.me/{bucket}/{key}?{parameters}

Results:

If the Ingress controller proxy sets the Host header to minio:9000. Minio presigned url signature validation will accept the upload.

References

• I have updated the documentation. PR feat(minio): document the MINIO_UPLOAD_URL_PREFIX configuration gitbook#32
• I have added included automated tests or evidence that it's working
• I acknowledge that I have tested that the change is backwards-compatible
• Original issue / feature request / discussion: See reasons for doing this below

Use case

It is difficult to create a Sorry Cypress / Minio Kubernetes setup where all the services are hosted behind a single hostname. The Minio API endpoint is inside the cluster but the upload endpoint is outside. Also it is difficult to run
a local setup without messing with port numbers and /etc/hosts.

Example

If the director is configured

MINIO_READ_URL_PREFIX=http://minio.localtest.me/sorry-cypress
MINIO_UPLOAD_URL_PREFIX=http://minio.localtest.me/sorry-cypress

then the readUrl and presigned uploadUrl URL are at the same location

{"level":30,"time":1700033523811,"requestId":"74902818-6763-4cb7-bec8-1e8821f67302","instanceId":"71248ef7-b83d-49af-a8a8-7320aef6e7d7","screenshotUploadUrls":[],"videoUploadInstructions":{"readUrl":"http://minio.localtest.me/sorry-cypress/71248ef7-b83d-49af-a8a8-7320aef6e7d7.mp4","uploadUrl":"http://minio.localtest.me/sorry-cypress/71248ef7-b83d-49af-a8a8-7320aef6e7d7.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio-user%2F20231115%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231115T073203Z&X-Amz-Expires=90&X-Amz-SignedHeaders=host&X-Amz-Signature=4addbc79f267cbb2966658d1d8ff028cc18b9fbb6fd4e54a99da5f2963dc0934"},"msg":"Sending assets upload URLs"}

The translated uploadUrl is not directly valid, the host needs to be changed back to the original uploadUrl generated by the minio api. For example if the uploadUrl was generated inside a cluster using the minio endpoint at port 9000 and translated to an external address using the MINIO_UPLOAD_URL_PREFIX. Then an ingress controller at the external address will need to translate it back again for the signature of the presigned URL to be valid.

Ingress controllers are usually configured by annotations on an Ingress definition. For example the NGINX ingress controller will do this translation by adding the annotation

nginx.ingress.kubernetes.io/upstream-vhost: "minio:9000"

Other Ingress controllers and api gateways handle it slightly differently.

Reason for doing this

I find it hard to configure Sorry-cypress screenshot uploads with Minio in kubernetes, this makes it easier.

[agoldis]

@all-contributors please add @erikmartino for code

[allcontributors]

@agoldis

I've put up a pull request to add @erikmartino! 🎉