msilib.CAB doesnot support non-ASCII files

[serhiy-storchaka]

BPO	41142
Nosy	@pfmoore, @tjguk, @zware, @serhiy-storchaka, @zooba
PRs	bpo-41142: Add support of non-ASCII paths for CAB files. #21195

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-06-30.08:56:22.843>
created_at = <Date 2020-06-27.15:01:56.046>
labels = ['type-feature', '3.10', 'OS-windows']
title = 'msilib.CAB doesnot support non-ASCII files'
updated_at = <Date 2020-06-30.08:56:22.842>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2020-06-30.08:56:22.842>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-06-30.08:56:22.843>
closer = 'serhiy.storchaka'
components = ['Windows']
creation = <Date 2020-06-27.15:01:56.046>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 41142
keywords = ['patch']
message_count = 2.0
messages = ['372466', '372669']
nosy_count = 5.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'serhiy.storchaka', 'steve.dower']
pr_nums = ['21195']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue41142'
versions = ['Python 3.10']

[serhiy-storchaka]

There are several problems with _msi.FCICreate() used to create the CAB file. It encodes the CAB names and added file names to UTF-8 and then use them as encoded with the local encoding. So you cannot create a CAB file in a directory with non-ASCII name or add files from a directory with non-ASCII name. This is a Python 3 regression.

Two possible solutions:

1. Encode paths with the locale encoding. It will add support for paths encodable with the locale encoding.

2. Encode them to UTF-8, but use a wrapper for opening file which converts the path from UTF-8 to UTF-16 and uses the Unicode variant of _open(). It will support arbitrary names.

[serhiy-storchaka]

New changeset ba67d73 by Serhiy Storchaka in branch 'master':
bpo-41142: Add support of non-ASCII paths for CAB files. (GH-21195)
ba67d73