Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_list_based_execution_strategy_selection: List-based explicit spawn strategy selection. #7480

Open
philwo opened this Issue Feb 20, 2019 · 3 comments

Comments

Projects
None yet
4 participants
@philwo
Copy link
Member

philwo commented Feb 20, 2019

πŸ™…β€β™‚οΈ Current situation:

  • The user can set the execution strategy in general and for individual action mnemonics using the flags --spawn_strategy=, --strategy=Mnemonic= and --strategy_regexp=Regex=. The flags take a single strategy name as value. Example: --spawn_strategy=linux-sandbox.
  • Bazel has a hardcoded and undocumented list of mnemonics that are known to work well with certain strategies and uses these unless the user overrides them individually using --strategy=Mnemonic= flags. Example: Bazel will use some kind of sandboxing by default if its available on your system, otherwise non-sandboxed execution (e.g. on Windows). However, it will run Javac actions via the persistent worker strategy.
  • Each strategy has custom code to deal with the situation that it can't execute a given action - for example, the remote strategy silently falls back to sandboxed execution if an action can't run remotely. The sandbox strategy silently falls back to non-sandboxed local execution if an action doesn't support sandboxing. There's no good way to configure this behavior.

πŸ†• The option --incompatible_list_based_execution_strategy_selection:

  • Allows the user to pass comma-separated lists of strategies to the above mentioned flags: --spawn_strategy=remote,worker,linux-sandbox.
  • Each strategy now knows whether it can execute a given action. For any action that it wants to execute, Bazel just picks the first strategy from the given list that claims to be able to execute the action. If no strategy remains, the execution will fail.
  • This means you can now have your build fail if any non-sandboxable or non-remotable action sneaks in! More reproducibility and safety for your builds! πŸŽ‰
  • The strategies no longer do their own custom fallback, simplifying the code and unifying the behavior.

πŸ’‘ Some ideas how to use this:

  • Don't set any flags and Bazel will try to do the best automatically: Use remote execution if it's available, otherwise persistent workers, otherwise sandboxed execution, otherwise non-sandboxed execution.
  • I want the best sandboxed build on my Linux machine and no automatic fallback to symlink-only sandboxing, non-sandboxed execution and no persistent workers: --spawn_strategy=linux-sandbox.
  • I want persistent workers for actions that support it, but otherwise only sandboxed execution: --spawn_strategy=worker,sandboxed.

🚧 Migration needed:

  • If you currently use the strategy selection flags, you'll want to revisit the values you set them to.
  • If you currently patch your Bazel version to remove fallback to non-sandboxed execution, this should no longer be necessary then (πŸ‘‹ @AustinSchuh).
@jin

This comment has been minimized.

Copy link
Member

jin commented Feb 20, 2019

/sub

@AustinSchuh

This comment has been minimized.

Copy link
Contributor

AustinSchuh commented Feb 20, 2019

Thanks @philwo ! This looks helpful.

We moved over to just using --spawn_strategy=linux-sandbox based on your advice in late 2017. That means we no longer have a patch for that.

Does this replace --remote_local_fallback_strategy=linux-sandbox ?

@agoulti

This comment has been minimized.

Copy link

agoulti commented Feb 20, 2019

Yes.
Under incompatible_list_based_execution_strategy_selection, remote_local_fallback_strategy will no longer be respected. A target that is marked no-remote will not be eligible for remote execution, so the next available strategy from the list will be applied.

We plan to deprecate remote_local_fallback_strategy by the time incompatible_list_based_execution_strategy_selection becomes the default.

We think that the new strategy list should serve all the fallback needs (and make things more explicit and cleaner). Do let us know if for some reason your case is not being served!

bazel-io pushed a commit that referenced this issue Feb 20, 2019

Add a "canExec" method to SpawnActionContext and SpawnRunner.
This is used by the ProxySpawnActionContext to determine at runtime
whether a spawn strategy can execute a given spawn.

Adds the flag --incompatible_list_based_execution_strategy_selection,
which is used to make this an opt-in change that will be managed by the
incompatible flags process.

The flag's GitHub issue is here:
#7480

RELNOTES[INC]: The flag --incompatible_list_based_execution_strategy_selection
was added and is used to ease the migration to the new style of specifying
execution strategy selection and fallback behavior. The documentation for
this flag is here: #7480

PiperOrigin-RevId: 234877574

irengrig added a commit to irengrig/bazel that referenced this issue Feb 26, 2019

Add a "canExec" method to SpawnActionContext and SpawnRunner.
This is used by the ProxySpawnActionContext to determine at runtime
whether a spawn strategy can execute a given spawn.

Adds the flag --incompatible_list_based_execution_strategy_selection,
which is used to make this an opt-in change that will be managed by the
incompatible flags process.

The flag's GitHub issue is here:
bazelbuild#7480

RELNOTES[INC]: The flag --incompatible_list_based_execution_strategy_selection
was added and is used to ease the migration to the new style of specifying
execution strategy selection and fallback behavior. The documentation for
this flag is here: bazelbuild#7480

PiperOrigin-RevId: 234877574

bazel-io pushed a commit that referenced this issue Mar 26, 2019

Release 0.24.0 (2019-03-26)
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in #7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes #7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See #7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: #7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [#7593](#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.

katre added a commit that referenced this issue Mar 29, 2019

Release 0.24.0 (2019-03-26)
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in #7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes #7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See #7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: #7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [#7593](#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Add a "canExec" method to SpawnActionContext and SpawnRunner.
This is used by the ProxySpawnActionContext to determine at runtime
whether a spawn strategy can execute a given spawn.

Adds the flag --incompatible_list_based_execution_strategy_selection,
which is used to make this an opt-in change that will be managed by the
incompatible flags process.

The flag's GitHub issue is here:
bazelbuild#7480

RELNOTES[INC]: The flag --incompatible_list_based_execution_strategy_selection
was added and is used to ease the migration to the new style of specifying
execution strategy selection and fallback behavior. The documentation for
this flag is here: bazelbuild#7480

PiperOrigin-RevId: 234877574

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Release 0.24.0 (2019-03-26)
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in bazelbuild#7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes bazelbuild#7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See bazelbuild#7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: bazelbuild#7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [bazelbuild#7593](bazelbuild#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (bazelbuild#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (bazelbuild#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (bazelbuild#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (bazelbuild#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com/bazelbuild/issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com/bazelbuild/issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.