Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_windows_escape_jvm_flags: on Windows, enables tokenization and escaping of jvm_flags #7486

Closed
laszlocsomor opened this Issue Feb 21, 2019 · 3 comments

Comments

Projects
None yet
2 participants
@laszlocsomor
Copy link
Contributor

commented Feb 21, 2019

Description

Add --incompatible_windows_escape_jvm_flags flag (default: false). This flag only affects builds on Windows.

This flag has no effect on Linux/macOS/non-Windows.

Background

When you build a java_binary or java_test on Windows, one of the outputs is an .exe file. This is called the launcher, and this is what you run with "bazel run" and "bazel test". The launcher's purpose is to set up the environment, compute the JVM command line (classpath, JVM flags, etc.) and launch the JVM.

The target-specific data (such as the classpath, main class name, but also the jvm_flags attribute from the BUILD-file) are embedded into the launcher as binary data. When you run the launcher, it will compute the command line to run the JVM as a subprocess (using the CreateProcessW system call), and that involves escaping the JVM flags so they can safely be passed to the subprcoess.

The new flag

The --incompatible_windows_escape_jvm_flags flag affects how Bazel builds the launcher:

  • whether or not Bazel will Bash-tokenize the jvm_flags before embedding them in the launcher, and
  • whether the launcher will escape these flags using the correct escaping logic or the buggy one.

When the flag is enabled:

  • Bazel will Bash-tokenize java_binary.jvm_flags and java_test.jvm_flags (as documented by the Build Encyclopedia) before embedding them into the launcher.

  • The launcher will properly escape these flags when it runs the JVM as a subprocess (using launcher_util::WindowsEscapeArg2).

  • The result is that the jvm_flags declared in the BUILD file will arrive to the Java program as intended.

When the flag is disabled:

  • Bazel does not Bash-tokenize the jvm_flags before embedding them in the launcher.

  • The launcher escapes the flags with a Bash-like escaping logic (launcher_util::WindowsEscapeArg)
    which cannot properly quote and escape everything.

  • The result is that the jvm_flags declared in the BUILD file might get messed up as they are passed to the JVM, or the launcher may not even be able to run the JVM.

Related bug: #7072

Example

BUILD file:

java_binary(
    name = "x",
    srcs = ["A.java"],
    main_class = "A",
    jvm_flags = [
        "-Darg0='hello world'",
        r"-Darg1=\"C:\\Program\ Files\\\"",
    ],
)

A.java:

public class A {
  public static void main(String[] args) {
    System.out.printf(
        "arg0=(%s)%narg1=(%s)%n",
        System.getProperty("arg0"),
        System.getProperty("arg1"));
  }
}

The expected output of running the binary is:

arg0=(hello world)
arg1=("C:\Program Files\")

Currently the output is something like:

Error: Could not find or load main class

Migration recipe

None, as of 2019-02-21.

We don't expect any breakages when this flag is enabled. However if it breaks your build or if it breaks your java_binaries or java_tests, please let us know so we can help fixing it and provide a migration recipe.

Rollout plan

  • Bazel 0.23.0 will not support this flag.
  • Bazel 0.24.0 is expected to support this flag, with default value being false.
  • Bazel 0.25.0 is expected to flip this flag to true.

@laszlocsomor laszlocsomor self-assigned this Feb 21, 2019

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Feb 21, 2019

Windows: add --incompatible_windows_escape_jvm_flags
Add --incompatible_windows_escape_jvm_flags flag
(default: false). This flag has no effect on
platforms other than Windows.

This flag affects how Bazel builds the launcher of
java_binary and java_test targets. (The launcher
is the .exe file that sets up the environment for
the Java program and launches the JVM.)

In particular, the flag controls whether or not
Bazel will Bash-tokenize the jvm_flags that were
declared in the BUILD file, and whether or not
these jvm_flags are escaped by the launcher when
it invokes the JVM.

When the flag is enabled:

- Bazel will Bash-tokenize java_binary.jvm_flags
  and java_test.jvm_flags (as documented by the
  Build Encyclopedia) before embedding them into
  the launcher.

- The launcher will properly escape these flags
  when it runs the JVM as a subprocess (using
  launcher_util::WindowsEscapeArg2).

- The result is that the jvm_flags declared in the
  BUILD file will arrive to the Java program as
  intended.

When the flag is disabled:

- Bazel does not Bash-tokenize the jvm_flags
  before embedding them in the launcher.

- The launcher escapes the flags with a Bash-like
  escaping logic (launcher_util::WindowsEscapeArg)
  which cannot properly quote and escape
  everything.

- The result is that the jvm_flags declared in the
  BUILD file might get messed up as they are
  passed to the JVM, or the launcher may not even
  be able to run the JVM.

Incompatible flag: bazelbuild#7486

Related: bazelbuild#7072

RELNOTES[NEW]: Added --incompatible_windows_escape_jvm_flags flag: enables correct java_binary.jvm_flags and java_test.jvm_flags tokenization and escaping on Windows. (No-op on other platforms.)

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Feb 21, 2019

Windows: add --incompatible_windows_escape_jvm_flags
Add --incompatible_windows_escape_jvm_flags flag
(default: false). This flag has no effect on
platforms other than Windows.

This flag affects how Bazel builds the launcher of
java_binary and java_test targets. (The launcher
is the .exe file that sets up the environment for
the Java program and launches the JVM.)

In particular, the flag controls whether or not
Bazel will Bash-tokenize the jvm_flags that were
declared in the BUILD file, and whether or not
these jvm_flags are escaped by the launcher when
it invokes the JVM.

When the flag is enabled:

- Bazel will Bash-tokenize java_binary.jvm_flags
  and java_test.jvm_flags (as documented by the
  Build Encyclopedia) before embedding them into
  the launcher.

- The launcher will properly escape these flags
  when it runs the JVM as a subprocess (using
  launcher_util::WindowsEscapeArg2).

- The result is that the jvm_flags declared in the
  BUILD file will arrive to the Java program as
  intended.

When the flag is disabled:

- Bazel does not Bash-tokenize the jvm_flags
  before embedding them in the launcher.

- The launcher escapes the flags with a Bash-like
  escaping logic (launcher_util::WindowsEscapeArg)
  which cannot properly quote and escape
  everything.

- The result is that the jvm_flags declared in the
  BUILD file might get messed up as they are
  passed to the JVM, or the launcher may not even
  be able to run the JVM.

Incompatible flag: bazelbuild#7486

Related: bazelbuild#7072

RELNOTES[NEW]: Added --incompatible_windows_escape_jvm_flags flag: enables correct java_binary.jvm_flags and java_test.jvm_flags tokenization and escaping on Windows. (No-op on other platforms.)

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Feb 21, 2019

Windows: add --incompatible_windows_escape_jvm_flags
Add --incompatible_windows_escape_jvm_flags flag
(default: false). This flag has no effect on
platforms other than Windows.

This flag affects how Bazel builds the launcher of
java_binary and java_test targets. (The launcher
is the .exe file that sets up the environment for
the Java program and launches the JVM.)

In particular, the flag controls whether or not
Bazel will Bash-tokenize the jvm_flags that were
declared in the BUILD file, and whether or not
these jvm_flags are escaped by the launcher when
it invokes the JVM.

When the flag is enabled:

- Bazel will Bash-tokenize java_binary.jvm_flags
  and java_test.jvm_flags (as documented by the
  Build Encyclopedia) before embedding them into
  the launcher.

- The launcher will properly escape these flags
  when it runs the JVM as a subprocess (using
  launcher_util::WindowsEscapeArg2).

- The result is that the jvm_flags declared in the
  BUILD file will arrive to the Java program as
  intended.

When the flag is disabled:

- Bazel does not Bash-tokenize the jvm_flags
  before embedding them in the launcher.

- The launcher escapes the flags with a Bash-like
  escaping logic (launcher_util::WindowsEscapeArg)
  which cannot properly quote and escape
  everything.

- The result is that the jvm_flags declared in the
  BUILD file might get messed up as they are
  passed to the JVM, or the launcher may not even
  be able to run the JVM.

Incompatible flag: bazelbuild#7486

Related: bazelbuild#7072

RELNOTES[NEW]: Added --incompatible_windows_escape_jvm_flags flag: enables correct java_binary.jvm_flags and java_test.jvm_flags tokenization and escaping on Windows. (No-op on other platforms.)

Change-Id: I531cc63cdfeccbe4b6d48876cb82870c1726a723

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Feb 21, 2019

Windows: add --incompatible_windows_escape_jvm_flags
Add --incompatible_windows_escape_jvm_flags flag
(default: false). This flag has no effect on
platforms other than Windows.

This flag affects how Bazel builds the launcher of
java_binary and java_test targets. (The launcher
is the .exe file that sets up the environment for
the Java program and launches the JVM.)

In particular, the flag controls whether or not
Bazel will Bash-tokenize the jvm_flags that were
declared in the BUILD file, and whether or not
these jvm_flags are escaped by the launcher when
it invokes the JVM.

When the flag is enabled:

- Bazel will Bash-tokenize java_binary.jvm_flags
  and java_test.jvm_flags (as documented by the
  Build Encyclopedia) before embedding them into
  the launcher. The build fails if an entry cannot
  be Bash-tokenized, e.g. if it has unterminated
  quotes.

- The launcher will properly escape these flags
  when it runs the JVM as a subprocess (using
  launcher_util::WindowsEscapeArg2).

- The result is that the jvm_flags declared in the
  BUILD file will arrive to the Java program as
  intended. However, due to the extra
  Bash-tokenization step, some ill-formed flags
  are no longer accepted, e.g.
  `jvm_flags=["-Dfoo='a"]` now results in a build
  error.

When the flag is disabled:

- Bazel does not Bash-tokenize the jvm_flags
  before embedding them in the launcher. This
  preserves quoting meant to be stripped away, and
  it also means Bazel won't check whether the
  argument is properly quoted.

- The launcher escapes the flags with a Bash-like
  escaping logic (launcher_util::WindowsEscapeArg)
  which cannot properly quote and escape
  everything.

- The result: the jvm_flags declared in the BUILD
  file might get messed up as they are passed to
  the JVM, or the launcher may not even be able to
  run the JVM. However, due to the lack of
  Bash-tokenization, Bazel propagates some flags
  to the Java binary that it would no longer
  accept if the new
  `--incompatible_windows_escape_jvm_flags` were
  enabled, e.g. `jvm_flags=["'a"]` is fine.

Incompatible flag: bazelbuild#7486

Related: bazelbuild#7072

RELNOTES[NEW]: Added --incompatible_windows_escape_jvm_flags flag: enables correct java_binary.jvm_flags and java_test.jvm_flags tokenization and escaping on Windows. (No-op on other platforms.)

Change-Id: I531cc63cdfeccbe4b6d48876cb82870c1726a723

bazel-io pushed a commit that referenced this issue Feb 26, 2019

Windows: add --incompatible_windows_escape_jvm_flags
Add --incompatible_windows_escape_jvm_flags flag
(default: false). This flag has no effect on
platforms other than Windows.

This flag affects how Bazel builds the launcher of
java_binary and java_test targets. (The launcher
is the .exe file that sets up the environment for
the Java program and launches the JVM.)

In particular, the flag controls whether or not
Bazel will Bash-tokenize the jvm_flags that were
declared in the BUILD file, and whether or not
these jvm_flags are escaped by the launcher when
it invokes the JVM.

When the flag is enabled:

- Bazel will Bash-tokenize java_binary.jvm_flags
  and java_test.jvm_flags (as documented by the
  Build Encyclopedia) before embedding them into
  the launcher. The build fails if an entry cannot
  be Bash-tokenized, e.g. if it has unterminated
  quotes.

- The launcher will properly escape these flags
  when it runs the JVM as a subprocess (using
  launcher_util::WindowsEscapeArg2).

- Result: the jvm_flags declared in the BUILD file
  will arrive to the Java program as intended.
  However, due to the extra Bash-tokenization
  step, some ill-formed flags are no longer
  accepted, e.g.  `jvm_flags=["-Dfoo='a"]` now
  results in a build error.

When the flag is disabled:

- Bazel does not Bash-tokenize the jvm_flags
  before embedding them in the launcher. This
  preserves quoting meant to be stripped away, and
  it also means Bazel won't check whether the
  argument is properly quoted.

- The launcher escapes the flags with a Bash-like
  escaping logic (launcher_util::WindowsEscapeArg)
  which cannot properly quote and escape
  everything.

- Result: the jvm_flags declared in the BUILD file
  might get messed up as they are passed to the
  JVM, or the launcher may not even be able to run
  the JVM. However, due to the lack of
  Bash-tokenization, Bazel propagates some flags
  to the Java binary that it would no longer
  accept if the new
  `--incompatible_windows_escape_jvm_flags` were
  enabled, e.g. `jvm_flags=["'a"]` is fine.

Incompatible flag: #7486

Related: #7072

RELNOTES[NEW]: Added --incompatible_windows_escape_jvm_flags flag: enables correct java_binary.jvm_flags and java_test.jvm_flags tokenization and escaping on Windows. (No-op on other platforms.)

Change-Id: I531cc63cdfeccbe4b6d48876cb82870c1726a723

Closes #7490.

PiperOrigin-RevId: 235700143

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Mar 6, 2019

Flip --incompatible_windows_escape_jvm_flags
Fixes: bazelbuild#7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See bazelbuild#7486

laszlocsomor added a commit that referenced this issue Mar 7, 2019

Flip --incompatible_windows_escape_jvm_flags
Fixes: #7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See #7486
@dkelmer

This comment has been minimized.

Copy link
Contributor

commented Apr 3, 2019

Seems like the flag has been flipped to true in 2029b1a. I'm closing the issue.

@dkelmer dkelmer closed this Apr 3, 2019

@laszlocsomor

This comment has been minimized.

Copy link
Contributor Author

commented Apr 5, 2019

The flag is not yet flipped:

2029b1a was merged a testing branch, not master, to run the "Bazel / Bazel@HEAD + Downstream" pipeline on CI.

@dkelmer

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2019

Ah, I did think the PR looked weird :)
Thanks for clarifying

bazel-io pushed a commit that referenced this issue Apr 8, 2019

Windows, java launcher: clean up after flag flip
Remove --incompatible_windows_escape_jvm_flags.

The flag is now flipped to true, and the
false-branch logic is removed.

See #7486

RELNOTES[INC]: --incompatible_windows_escape_jvm_flags is enabled by default, and the flag no longer exists

Change-Id: I4ba9a626c28ce9690c0850fa052c302f80bccc1c

Closes #7972.

Change-Id: I4ba9a626c28ce9690c0850fa052c302f80bccc1c
PiperOrigin-RevId: 242482659

dkelmer added a commit that referenced this issue Apr 11, 2019

Flip --incompatible_windows_escape_jvm_flags
All downstream projects are green:
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/903

Fixes: #7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See #7486

Closes #7646.

PiperOrigin-RevId: 242107771

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Windows: add --incompatible_windows_escape_jvm_flags
Add --incompatible_windows_escape_jvm_flags flag
(default: false). This flag has no effect on
platforms other than Windows.

This flag affects how Bazel builds the launcher of
java_binary and java_test targets. (The launcher
is the .exe file that sets up the environment for
the Java program and launches the JVM.)

In particular, the flag controls whether or not
Bazel will Bash-tokenize the jvm_flags that were
declared in the BUILD file, and whether or not
these jvm_flags are escaped by the launcher when
it invokes the JVM.

When the flag is enabled:

- Bazel will Bash-tokenize java_binary.jvm_flags
  and java_test.jvm_flags (as documented by the
  Build Encyclopedia) before embedding them into
  the launcher. The build fails if an entry cannot
  be Bash-tokenized, e.g. if it has unterminated
  quotes.

- The launcher will properly escape these flags
  when it runs the JVM as a subprocess (using
  launcher_util::WindowsEscapeArg2).

- Result: the jvm_flags declared in the BUILD file
  will arrive to the Java program as intended.
  However, due to the extra Bash-tokenization
  step, some ill-formed flags are no longer
  accepted, e.g.  `jvm_flags=["-Dfoo='a"]` now
  results in a build error.

When the flag is disabled:

- Bazel does not Bash-tokenize the jvm_flags
  before embedding them in the launcher. This
  preserves quoting meant to be stripped away, and
  it also means Bazel won't check whether the
  argument is properly quoted.

- The launcher escapes the flags with a Bash-like
  escaping logic (launcher_util::WindowsEscapeArg)
  which cannot properly quote and escape
  everything.

- Result: the jvm_flags declared in the BUILD file
  might get messed up as they are passed to the
  JVM, or the launcher may not even be able to run
  the JVM. However, due to the lack of
  Bash-tokenization, Bazel propagates some flags
  to the Java binary that it would no longer
  accept if the new
  `--incompatible_windows_escape_jvm_flags` were
  enabled, e.g. `jvm_flags=["'a"]` is fine.

Incompatible flag: bazelbuild#7486

Related: bazelbuild#7072

RELNOTES[NEW]: Added --incompatible_windows_escape_jvm_flags flag: enables correct java_binary.jvm_flags and java_test.jvm_flags tokenization and escaping on Windows. (No-op on other platforms.)

Change-Id: I531cc63cdfeccbe4b6d48876cb82870c1726a723

Closes bazelbuild#7490.

PiperOrigin-RevId: 235700143

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Flip --incompatible_windows_escape_jvm_flags
All downstream projects are green:
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/903

Fixes: bazelbuild#7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See bazelbuild#7486

Closes bazelbuild#7646.

PiperOrigin-RevId: 242107771

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Windows, java launcher: clean up after flag flip
Remove --incompatible_windows_escape_jvm_flags.

The flag is now flipped to true, and the
false-branch logic is removed.

See bazelbuild#7486

RELNOTES[INC]: --incompatible_windows_escape_jvm_flags is enabled by default, and the flag no longer exists

Change-Id: I4ba9a626c28ce9690c0850fa052c302f80bccc1c

Closes bazelbuild#7972.

Change-Id: I4ba9a626c28ce9690c0850fa052c302f80bccc1c
PiperOrigin-RevId: 242482659

dkelmer added a commit that referenced this issue Apr 16, 2019

Flip --incompatible_windows_escape_jvm_flags
All downstream projects are green:
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/903

Fixes: #7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See #7486

Closes #7646.

PiperOrigin-RevId: 242107771

dkelmer added a commit that referenced this issue Apr 25, 2019

Flip --incompatible_windows_escape_jvm_flags
All downstream projects are green:
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/903

Fixes: #7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See #7486

Closes #7646.

PiperOrigin-RevId: 242107771

dkelmer added a commit that referenced this issue Apr 25, 2019

Flip --incompatible_windows_escape_jvm_flags
All downstream projects are green:
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/903

Fixes: #7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See #7486

Closes #7646.

PiperOrigin-RevId: 242107771

dkelmer added a commit that referenced this issue Apr 25, 2019

Flip --incompatible_windows_escape_jvm_flags
All downstream projects are green:
https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/903

Fixes: #7486

RELNOTES[INC]: Flip --incompatible_windows_escape_jvm_flags to true. See #7486

Closes #7646.

PiperOrigin-RevId: 242107771
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.