aws cli to auto guess content type of .br (brotli) files

[kazazor]

Today AWS cli is trying to guess the content type of the file even if it's compressed (which is a VERY common case when uploading to S3 files instead of cloudfront auto compression). This is being done here:

aws-cli/awscli/customizations/s3/utils.py

Lines 288 to 307 in 5d34910

	def guess_content_type(filename):
	"""Given a filename, guess it's content type.
	If the type cannot be guessed, a value of None is returned.
	"""
	try:
	return mimetypes.guess_type(filename)[0]
	# This catches a bug in the mimetype libary where some MIME types
	# specifically on windows machines cause a UnicodeDecodeError
	# because the MIME type in the Windows registery has an encoding
	# that cannot be properly encoded using the default system encoding.
	# https://bugs.python.org/issue9291
	#
	# So instead of hard failing, just log the issue and fall back to the
	# default guessed content type of None.
	except UnicodeDecodeError:
	LOGGER.debug(
	'Unable to guess content type for %s due to '
	'UnicodeDecodeError: ', filename, exc_info=True
	)

Which internally uses a python library with hardcoded options: https://github.com/python/cpython/blob/cedc9b74202d8c1ae39bca261cbb45d42ed54d45/Lib/mimetypes.py#L402-L530

All of that is described here in a closed issue: #3303

However, .br extensions are becoming more and more popular and are being sent by default by all major browsers (excluding IE 11). Because of that problem .br files content-type are not being guessed at all and we're required to do it manually.

I suggest either adding it to the python library or embedding the hard-coded list to aws-cli so you will have full control.

[JordonPhillips]

I'm -1 on maintaining a list in the CLI. The mimetypes module allows extension via an array of common files, so if you want something that isn't built in then you should edit one of those files or otherwise provide the value on the command line as you're uploading the files.

[kazazor]

I can understand why not to embed it here, even that you're not controlling your own fate on this one..

or otherwise provide the value on the command line as you're uploading the files.
I believe the auto detect content type was entered there for a reason. Currently, the cli does not support it anymore for one of the popular compressions out there which is the default by all major browsers.. I do believe something should be done to support it.. I have already done it manually for our repo but I do believe it's needed from the same reason it was added there to begin with.

@JordonPhillips - what other options do you see here? Just updating the python repo with this code? Seems like it will never happen to be honest :/

[JordonPhillips]

Yeah, your options are to either get it added into python or add it to your own configuration files. Have you tried opening a PR on cpython? It looks like it would be a pretty simple change.

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

[iAnomaly]

For anyone else looking for this, support for Brotli (.br) extension was added to the upstream Python library by this PR. Sadly, unless they backport to Python 3.8 we will have to wait for 3.9 to use it.