Use different encoding for non-English filenames in content-disposition header

[mabagoury]

If an attachment file is returned in the response body, the content-disposition is set to Content-Disposition: attachment; filename="filename". Now, the browser auto-downloads the file when the response is returned and the file is named filename.

This works fine for English characters. The problem appears when you have a filename that contains, for example, Arabic letters. The header now can't be correctly encoded using the latin-1 format.

This article further discusses the problem and its possible solutions. The best one mentioned is to add the filename* field in the header that contains the filename encoded in utf-8. Now it looks like this:

content-disposition` is set to `Content-Disposition: attachment; filename="filename";  filename*=utf-8"%D0%91%91.txt

This process is described in RFC6266. filename* includes the encoding format then " then the encoded name while filename can have a translated name that only contains English characters.

The only downside of this solution is that some ancient browsers will take filename over filename*. But almost all browsers will take the filename from filename*.

Starlette throws an error like the following when this problem happens

UnicodeEncodeError: 'latin-1' codec can't encode characters in position 21-25: ordinal not in range(256)

The FileResponse class automatically puts the filename in the filename field in the Content-Disposition header. It assumes the file can be encoded by the latin-1 format.

While this can be handled by the app passing a custom Content-Dispostion header, the FileResponse can be a little adjusted to address this common issue. It should test if the filename contains non-English characters. If not, just operate as normal. If yes, follow the above method.