BUCKET_ENDPOINT_URL must contain region otherwise URL signing process fails

[offchan42]

Describe the bug
I currently set my BUCKET_ENDPOINT_URL environment variable like this: https://BUCKET_NAME.s3.amazonaws.com/
Notice that I omit the region name. This is fine when uploading images but the URL signing process becomes broken.

To Reproduce
Steps to reproduce the behavior:

1. Set BUCKET_ENDPOINT_URL to not have region
2. Generate an image using HTTP request and open the signed image URL, you will see the following error: Error parsing the X-Amz-Credential parameter; the region 'amazonaws' is wrong; expecting 'us-east-1'

The following is the code that initializes boto client with incorrect region, leading to broken signing.

runpod-python/runpod/serverless/utils/rp_upload.py

Lines 66 to 86 in bd68176

	if bucket_creds:
	endpoint_url = bucket_creds['endpointUrl']
	access_key_id = bucket_creds['accessId']
	secret_access_key = bucket_creds['accessSecret']
	else:
	endpoint_url = os.environ.get('BUCKET_ENDPOINT_URL', None)
	access_key_id = os.environ.get('BUCKET_ACCESS_KEY_ID', None)
	secret_access_key = os.environ.get('BUCKET_SECRET_ACCESS_KEY', None)
	if endpoint_url and access_key_id and secret_access_key:
	# Extract region from the endpoint URL
	region = extract_region_from_url(endpoint_url)
	boto_client = bucket_session.client(
	's3',
	endpoint_url=endpoint_url,
	aws_access_key_id=access_key_id,
	aws_secret_access_key=secret_access_key,
	config=boto_config,
	region_name=region
	)

The error comes from this function that extracts region from URL incorrectly by thinking that amazonaws is the region.

runpod-python/runpod/serverless/utils/rp_upload.py

Lines 27 to 40 in bd68176

	def extract_region_from_url(endpoint_url):
	"""
	Extracts the region from the endpoint URL.
	"""
	parsed_url = urlparse(endpoint_url)
	# AWS/backblaze S3-like URL
	if '.s3.' in endpoint_url:
	return endpoint_url.split('.s3.')[1].split('.')[0]
	# DigitalOcean Spaces-like URL
	if parsed_url.netloc.endswith('.digitaloceanspaces.com'):
	return endpoint_url.split('.')[1].split('.digitaloceanspaces.com')[0]
	return None

Expected behavior
URL signing process should work even when there is no region specified in BUCKET_ENDPOINT_URL.
I think that you should not try to set the value of the region when initializing boto client when there's no region from the URL.

Screenshots