incompatible_windows_escape_python_args: on Windows, enables correct escaping of py_binary.args and py_test.args

[laszlocsomor]

Description

Add --incompatible_windows_escape_python_args flag (default: false).

This is a no-op on Linux/macOS.

On Linux/macOS/non-Windows: the flag is a no-op.

On Windows: This flag affects how py_binary and py_test targets are built, how their launcher escapes command line flags. When this flag is true, the launcher escapes command line flags using Windows-style escaping (correct behavior). When the flag is false, the launcher uses Bash-style escaping (buggy behavior).

Background

The output of a py_binary or py_test is an .exe file of two parts: a stub (launcher) and a payload (zip file). The payload is the actual Python program with all its runfiles..

To build py_binary/py_test rules, Bazel appends the zip file and some metadata to the launcher. The launcher reads the data from its tail, then sets up the environment and runs Python, passing the command line arguments along.

However the launcher escapes arguments badly. This is the same bug for Python as #7486 was for Java. The solution is similar: use WindowsEscapeArg2 in the launcher:

bazel/src/tools/launcher/python_launcher.cc

Line 52 in 20a78a8

args[i] = WindowsEscapeArg(args[i]);

bazel/src/tools/launcher/util/launcher_util.cc

Lines 173 to 182 in 20a78a8

	// Escape arguments for CreateProcessW.
	//
	// This algorithm is based on information found in
	// http://daviddeley.com/autohotkey/parameters/parameters.htm
	//
	// The following source specifies a similar algorithm:
	// https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/
	// unfortunately I found this algorithm only after creating the one below, but
	// fortunately they seem to do the same.
	std::wstring WindowsEscapeArg2(const std::wstring& s) {

Related bug: #7958

The new flag

The --incompatible_windows_escape_python_args flag affects the metadata Bazel embeds in the launcher, which governs whether the launcher uses bad or good escaping.

Migration recipe

None, as of 2019-04-08.

We don't expect any breakages when this flag is enabled. However if it breaks your build or if it breaks your py_binaries or py_tests, please let us know so we can help fixing it and provide a migration recipe.

Rollout plan

• Bazel 0.25.0 will not support this flag.
• Bazel 0.26.0 is expected to support this flag, with default value being false.
• Bazel 0.27.0 is expected to flip this flag to true, but keep the flag.