Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_py3_is_default: Bazel configures targets for Python 3 by default #7359

Closed
brandjon opened this Issue Feb 5, 2019 · 13 comments

Comments

Projects
None yet
5 participants
@brandjon
Copy link
Member

brandjon commented Feb 5, 2019

Flag: --incompatible_py3_is_default
Available since: 0.24
Will be flipped in: 0.25
Tracking issue: #6647

It is recommended to enable this flag at the same time as --incompatible_py2_outputs_are_suffixed.

Motivation

Bazel currently defaults to Python 2 due to historic reasons, but the community has largely moved on to Python 3. We should fix this before 1.0.

Note: This incompatible change affects the mechanism that Bazel uses to decide whether a target should be built for Python 2 or 3. However, due to #4815, even when a target is built for Python 3 it may still execute under a Python 2 interpreter. See the workaround using py_runtime / --python_top in #4815 (comment).

Change

As of 0.24, py_binary and py_test targets that do not specify a python_version attribute (formerly called default_python_version) will use PY3 instead of PY2 when --incompatible_py3_is_default is enabled. In addition, --incompatible_py3_is_default makes the host configuration use PY3 instead of PY2, unless --host_force_python=PY2 is given to override this behavior.

--incompatible_py3_is_default requires that --incompatible_allow_python_version_transitions is set true, or else an error is raised. This means that if you opt out of --incompatible_allow_python_version_transitions during the migration window, you should also opt out of --incompatible_py3_is_default.

Migration

Any py_binary or py_test targets that cannot tolerate Python 3 should have an attribute python_version = "PY2" added. You can do this with buildozer using a command like the following:

buildozer -k -types 'py_binary,py_test' 'set_if_absent python_version PY2' //...:*

For targets that get built in the host configuration, this command can modify their definitions but it will not actually affect the version that they use. If your build has tools in the host configuration that cannot tolerate Python 3, then you can keep the old behavior of using Python 2 for all host configuration targets (across your entire build) by adding --host_force_python=PY2 to your build invocations. We do not currently have a story for incremental migration of targets in the host configuration to PY3. This is a consequence of the technical limitations of the host configuration, and should be addressed by future work on toolchains.

@brandjon brandjon self-assigned this Feb 5, 2019

bazel-io pushed a commit that referenced this issue Feb 7, 2019

Add `--incompatible_py3_is_default` to change `py_binary` and `py_tes…
…t`'s default version

When this flag is enabled, `py_binary` and `py_test` targets that do not specify a version (by setting the `python_version` attribute, or its deprecated alias `default_python_version`) will default to `PY3` instead of `PY2`. In addition, the host configuration will use `PY3`, unless `--host_force_python=PY2` is set to override it.

The flag requires the new version semantics (`--incompatible_allow_python_version_transitions=true`), and will fail the build if it is not set.

The flag is implemented by eliminating PythonVersion#DEFAULT_TARGET_VALUE and replacing it with an accessor PythonOptions#getDefaultPythonVersion. For the case of rule transition factories in rule definitions, which previously relied on DEFAULT_TARGET_VALUE and which do not have access to a configuration, they are updated to instead pass a callback function that retrieves the default version from the configuration at the time of the transition.

Work toward #6647, #7359.

RELNOTES[INC]: Added --incompatible_py3_is_default to test switching the default Python version to PY3 for py_binary/py_test targets that do not specify a version. See #7359.

PiperOrigin-RevId: 232939284
@limdor

This comment has been minimized.

Copy link

limdor commented Feb 8, 2019

Nice to see this issue. A really good motivation for this issue that is not mentioned could be that Python 2 will reach End of Life in less than 11 months. https://pythonclock.org/

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 12, 2019

Ran a downstream presubmit with this flag and --incompatible_py2_outputs_are_suffixed both flipped. Here's the results. We have failures in:

For TensorFlow it didn't actually run the test because it failed a hardcoded version check, and it's not clear that it uses the bazel under test. Filed bazelbuild/continuous-integration#569 for assistance.

For the other failures I have no idea whether they're legit or not, and will probably rerun the presubmit to deflake.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 13, 2019

All four of these failures are broken at head.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 13, 2019

It helps if the branch I'm using actually has the flag flips I'm testing. Sigh. Let's do this over again.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 13, 2019

New results here. [Edit: fixed link to show all results]

  • TensorFlow and gmaven_rules are already failing, so no information there.

  • rules_k8s fails due to --incompatible_remove_old_python_version_api, so that needs to be fixed first

  • Not sure about the Gerrit failure, could be a flake. It's only on one platform (mac).

As of this post rules_typescript and Bazel itself still have a mac test running.

In any case, this looks good enough that I'm willing to flip both --incompatible_py3_is_default and --incompatible_py2_outputs_are_suffixed for Bazel 0.25.

@greggdonovan

This comment has been minimized.

Copy link
Member

greggdonovan commented Mar 13, 2019

@brandjon This is really promising -- thanks! Does this test exercise rules_docker and, transitively, containerregistry?

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 13, 2019

Well I'm not so sure. There is indeed a rules_docker project in the set of downstream tests that get run. But when I tested another flag --incompatible_remove_old_python_version_api, I noticed failures in rules_k8s but not rules_docker, even though they both have dependencies on subpar that needed updating.

You can test yourself by grabbing Bazel at head, or a 0.24 RC, and passing (at the same time) --incompatible_py3_is_default and --incompatible_py2_outputs_are_suffixed. You can also check the bazel invocation used in my presubmit and see if any test targets that would've identified problems were missing.

@greggdonovan

This comment has been minimized.

Copy link
Member

greggdonovan commented Mar 13, 2019

@brandjon Ahh, right, it looks like google/containerregistry needs a new release with an updated subpar to enable rules_docker and rules_k8s to be updated.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 21, 2019

In another run today, the only failure is in IntelliJ. They're using an output-path-based method for determining whether a Python target is PY2 or PY3. I'm in communication with them to fix.

bazel-io pushed a commit that referenced this issue Mar 26, 2019

Release 0.24.0 (2019-03-26)
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in #7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes #7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See #7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: #7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [#7593](#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.
@meteorcloudy

This comment has been minimized.

Copy link
Member

meteorcloudy commented Mar 27, 2019

This flag is not actually tested on Bazel CI:
https://buildkite.com/bazel/bazelisk-plus-incompatible-flags/builds/58

Because it relies on --incompatible_allow_python_version_transitions

ERROR: cannot enable `--incompatible_py3_is_default` without also enabling `--incompatible_allow_python_version_transitions`

Can we automatically enable --incompatible_allow_python_version_transitions when --incompatible_py3_is_default is specified in code?

@meteorcloudy

This comment has been minimized.

Copy link
Member

meteorcloudy commented Mar 27, 2019

I manually tested this flag with Bazel 0.24.0 and got the following error

pcloudy@pcloudy:~/workspace/tensorflow
$ bazel build --config=opt tensorflow/tools/pip_package:build_pip_package --incompatible_py3_is_default --incompatible_allow_python_version_transitions
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/python/BUILD:3243:1: in py_library rule //tensorflow/python:standard_ops: target '//tensorflow/python:standard_ops' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/python/BUILD:102:1: in py_library rule //tensorflow/python:no_contrib: target '//tensorflow/python:no_contrib' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/metrics/BUILD:16:1: in py_library rule //tensorflow/contrib/metrics:metrics_py: target '//tensorflow/contrib/metrics:metrics_py' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/bayesflow/BUILD:17:1: in py_library rule //tensorflow/contrib/bayesflow:bayesflow_py: target '//tensorflow/contrib/bayesflow:bayesflow_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/BUILD:13:1: in py_library rule //tensorflow/contrib:contrib_py: target '//tensorflow/contrib:contrib_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
ERROR: file 'external/icu/_objs/icuuc/uprops.pic.o' is generated by these conflicting actions:
Label: @icu//:icuuc
RuleClass: cc_library rule
Configuration: 98c6e8c3d595f000f2cc3fbc5194178d, cd31cd26593c7ea5d91cb6602ad66fc4
Mnemonic: CppCompile
Action key: ec9d8825160200c6b2b395858d5a2e86
Progress message: Compiling external/icu/icu4c/source/common/uprops.cpp
PrimaryInput: File:[/usr/local/google/home/pcloudy/.cache/bazel/_bazel_pcloudy/d2f15366e3b15100d3593f2d595721a3[source]]external/icu/icu4c/source/common/uprops.cpp
PrimaryOutput: File:[[<execution_root>]bazel-out/k8-py3-opt/bin]external/icu/_objs/icuuc/uprops.pic.o
Primary outputs are different: 972537493, 1038679363
Owner information: @icu//:icuuc BuildConfigurationValue.Key[98c6e8c3d595f000f2cc3fbc5194178d] false, @icu//:icuuc BuildConfigurationValue.Key[cd31cd26593c7ea5d91cb6602ad66fc4] false
MandatoryInputs: are equal
Outputs: Attempted action contains artifacts not in previous action (first 5): 
        external/icu/_objs/icuuc/uprops.pic.o
        external/icu/_objs/icuuc/uprops.pic.d
Previous action contains artifacts not in attempted action (first 5): 
        external/icu/_objs/icuuc/uprops.pic.o
        external/icu/_objs/icuuc/uprops.pic.d
ERROR: com.google.devtools.build.lib.actions.MutableActionGraph$ActionConflictException: for external/icu/_objs/icuuc/uprops.pic.o, previous action: action 'Compiling external/icu/icu4c/source/common/uprops.cpp', attempted action: action 'Compiling external/icu/icu4c/source/common/uprops.cpp'
INFO: Elapsed time: 11.820s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (396 packages loaded, 27672 targets configured)

This looks like a bug that has already been fixed at HEAD, but it also means we cannot flip this flag for 0.25.0, because users cannot use 0.24.0 for migration.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 28, 2019

See the link in my last post -- I ran the Bazel CI using the downstream w/ bazel@head pipeline, on a custom build that flipped all four python-version-related flags (2 introduced in 0.23 and 2 in 0.24, all to be flipped in 0.25) and it was green. So I don't think migration is actually an issue for this change.

The action conflict issue is partially mitigated by turning on --incompatible_remove_old_python_version_api, which is one of those four flags to be flipped in 0.25. The only remaining action conflicts occur in the rarer case that a target is built both at the top-level configuration and at a config reached by transitioning away from the default Python version and then back again.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 29, 2019

Ran another rerun of downstream at head + flag flips here. It's clean except for Tulsi which is broken at Bazel head, and Bazel's own presubmits are still running as I'm writing this. So I'm pretty comfortable with proceeding with these flips.

Note that I still have to update Bazels own unit tests for the flag flips. (This isn't reflected by the above presubmit since it modified the host blaze, not the blaze under test.)

katre added a commit that referenced this issue Mar 29, 2019

Release 0.24.0 (2019-03-26)
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in #7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes #7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See #7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: #7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [#7593](#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.

@bazel-io bazel-io closed this in 5f5e6ad Mar 30, 2019

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Release 0.24.0 (2019-03-26)
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in bazelbuild#7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes bazelbuild#7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See bazelbuild#7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: bazelbuild#7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [bazelbuild#7593](bazelbuild#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (bazelbuild#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (bazelbuild#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (bazelbuild#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (bazelbuild#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com/bazelbuild/issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com/bazelbuild/issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.

emusand added a commit to emusand/bazel that referenced this issue Apr 16, 2019

Flip default Python version to PY3
This flips --incompatible_py3_is_default and --incompatible_py2_outputs_are_suffixed. We opt out of this change for our own analysis and shell integration tests to avoid migrating them at the moment.

Fixes bazelbuild#7359, fixes bazelbuild#7593, fixes bazelbuild#6647.

RELNOTES[INC]: Python 3 is now the default Python version (for `py_binary` and `py_test` targets that don't specify the `python_version` attribute). Targets that are built for Python 3 will no longer have their output put in a separate `-py3` directory; instead there is now a separate `-py2` directory for Python 2 targets. See bazelbuild#7359 and bazelbuild#7593 for more information.

PiperOrigin-RevId: 241142275
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.