Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

incompatible_py3_is_default: Bazel configures targets for Python 3 by default #7359

Closed
brandjon opened this issue Feb 5, 2019 · 13 comments
Closed

incompatible_py3_is_default: Bazel configures targets for Python 3 by default #7359

brandjon opened this issue Feb 5, 2019 · 13 comments
Assignees
@brandjon
Labels
incompatible-change Incompatible/breaking change team-Rules-Python Native rules for Python

Comments

@brandjon
Copy link
Member

brandjon commented Feb 5, 2019 •
edited
Loading

Flag: --incompatible_py3_is_default
Available since: 0.24
Will be flipped in: 0.25
Tracking issue: #6647

It is recommended to enable this flag at the same time as --incompatible_py2_outputs_are_suffixed.

Motivation

Bazel currently defaults to Python 2 due to historic reasons, but the community has largely moved on to Python 3. We should fix this before 1.0.

Note: This incompatible change affects the mechanism that Bazel uses to decide whether a target should be built for Python 2 or 3. However, due to #4815, even when a target is built for Python 3 it may still execute under a Python 2 interpreter. See the workaround using py_runtime / --python_top in #4815 (comment).

Change

As of 0.24, py_binary and py_test targets that do not specify a python_version attribute (formerly called default_python_version) will use PY3 instead of PY2 when --incompatible_py3_is_default is enabled. In addition, --incompatible_py3_is_default makes the host configuration use PY3 instead of PY2, unless --host_force_python=PY2 is given to override this behavior.

--incompatible_py3_is_default requires that --incompatible_allow_python_version_transitions is set true, or else an error is raised. This means that if you opt out of --incompatible_allow_python_version_transitions during the migration window, you should also opt out of --incompatible_py3_is_default.

Migration

Any py_binary or py_test targets that cannot tolerate Python 3 should have an attribute python_version = "PY2" added. You can do this with buildozer using a command like the following:

buildozer -k -types 'py_binary,py_test' 'set_if_absent python_version PY2' //...:*

For targets that get built in the host configuration, this command can modify their definitions but it will not actually affect the version that they use. If your build has tools in the host configuration that cannot tolerate Python 3, then you can keep the old behavior of using Python 2 for all host configuration targets (across your entire build) by adding --host_force_python=PY2 to your build invocations. We do not currently have a story for incremental migration of targets in the host configuration to PY3. This is a consequence of the technical limitations of the host configuration, and should be addressed by future work on toolchains.
@brandjon brandjon added incompatible-change Incompatible/breaking change team-Rules-Python Native rules for Python labels Feb 5, 2019
@brandjon brandjon self-assigned this Feb 5, 2019
bazel-io pushed a commit that referenced this issue Feb 7, 2019
@brandjon
Add --incompatible_py3_is_default to change py_binary and `py_tes…
ea956ae 
…t`'s default version

When this flag is enabled, `py_binary` and `py_test` targets that do not specify a version (by setting the `python_version` attribute, or its deprecated alias `default_python_version`) will default to `PY3` instead of `PY2`. In addition, the host configuration will use `PY3`, unless `--host_force_python=PY2` is set to override it.

The flag requires the new version semantics (`--incompatible_allow_python_version_transitions=true`), and will fail the build if it is not set.

The flag is implemented by eliminating PythonVersion#DEFAULT_TARGET_VALUE and replacing it with an accessor PythonOptions#getDefaultPythonVersion. For the case of rule transition factories in rule definitions, which previously relied on DEFAULT_TARGET_VALUE and which do not have access to a configuration, they are updated to instead pass a callback function that retrieves the default version from the configuration at the time of the transition.

Work toward #6647, #7359.

RELNOTES[INC]: Added --incompatible_py3_is_default to test switching the default Python version to PY3 for py_binary/py_test targets that do not specify a version. See #7359.

PiperOrigin-RevId: 232939284
@limdor
Copy link
Contributor

limdor commented Feb 8, 2019

Nice to see this issue. A really good motivation for this issue that is not mentioned could be that Python 2 will reach End of Life in less than 11 months. https://pythonclock.org/

@brandjon brandjon mentioned this issue Feb 28, 2019
Closed
@brandjon
Copy link
Member Author

Ran a downstream presubmit with this flag and --incompatible_py2_outputs_are_suffixed both flipped. Here's the results. We have failures in:

For TensorFlow it didn't actually run the test because it failed a hardcoded version check, and it's not clear that it uses the bazel under test. Filed bazelbuild/continuous-integration#569 for assistance.

For the other failures I have no idea whether they're legit or not, and will probably rerun the presubmit to deflake.

@brandjon
Copy link
Member Author

All four of these failures are broken at head.

@brandjon
Copy link
Member Author

It helps if the branch I'm using actually has the flag flips I'm testing. Sigh. Let's do this over again.

@brandjon
Copy link
Member Author

brandjon commented Mar 13, 2019 •
edited
Loading

New results here. [Edit: fixed link to show all results]

  • TensorFlow and gmaven_rules are already failing, so no information there.

  • rules_k8s fails due to --incompatible_remove_old_python_version_api, so that needs to be fixed first

  • Not sure about the Gerrit failure, could be a flake. It's only on one platform (mac).

As of this post rules_typescript and Bazel itself still have a mac test running.

In any case, this looks good enough that I'm willing to flip both --incompatible_py3_is_default and --incompatible_py2_outputs_are_suffixed for Bazel 0.25.

@brandjon brandjon mentioned this issue Mar 13, 2019
Closed
@greggdonovan
Copy link
Member

greggdonovan commented Mar 13, 2019 •
edited
Loading

@brandjon This is really promising -- thanks! Does this test exercise rules_docker and, transitively, containerregistry?

@brandjon
Copy link
Member Author

Well I'm not so sure. There is indeed a rules_docker project in the set of downstream tests that get run. But when I tested another flag --incompatible_remove_old_python_version_api, I noticed failures in rules_k8s but not rules_docker, even though they both have dependencies on subpar that needed updating.

You can test yourself by grabbing Bazel at head, or a 0.24 RC, and passing (at the same time) --incompatible_py3_is_default and --incompatible_py2_outputs_are_suffixed. You can also check the bazel invocation used in my presubmit and see if any test targets that would've identified problems were missing.

@greggdonovan
Copy link
Member

@brandjon Ahh, right, it looks like google/containerregistry needs a new release with an updated subpar to enable rules_docker and rules_k8s to be updated.

@brandjon
Copy link
Member Author

In another run today, the only failure is in IntelliJ. They're using an output-path-based method for determining whether a Python target is PY2 or PY3. I'm in communication with them to fix.

bazel-io pushed a commit that referenced this issue Mar 26, 2019
@katre
Release 0.24.0 (2019-03-26)
4f5c17c 
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in #7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes #7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See #7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: #7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [#7593](#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.
@meteorcloudy
Copy link
Member

This flag is not actually tested on Bazel CI:
https://buildkite.com/bazel/bazelisk-plus-incompatible-flags/builds/58

Because it relies on --incompatible_allow_python_version_transitions

ERROR: cannot enable `--incompatible_py3_is_default` without also enabling `--incompatible_allow_python_version_transitions`

Can we automatically enable --incompatible_allow_python_version_transitions when --incompatible_py3_is_default is specified in code?

@meteorcloudy
Copy link
Member

I manually tested this flag with Bazel 0.24.0 and got the following error

pcloudy@pcloudy:~/workspace/tensorflow
$ bazel build --config=opt tensorflow/tools/pip_package:build_pip_package --incompatible_py3_is_default --incompatible_allow_python_version_transitions
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/python/BUILD:3243:1: in py_library rule //tensorflow/python:standard_ops: target '//tensorflow/python:standard_ops' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/python/BUILD:102:1: in py_library rule //tensorflow/python:no_contrib: target '//tensorflow/python:no_contrib' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/metrics/BUILD:16:1: in py_library rule //tensorflow/contrib/metrics:metrics_py: target '//tensorflow/contrib/metrics:metrics_py' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/bayesflow/BUILD:17:1: in py_library rule //tensorflow/contrib/bayesflow:bayesflow_py: target '//tensorflow/contrib/bayesflow:bayesflow_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /usr/local/google/home/pcloudy/workspace/tensorflow/tensorflow/contrib/BUILD:13:1: in py_library rule //tensorflow/contrib:contrib_py: target '//tensorflow/contrib:contrib_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
ERROR: file 'external/icu/_objs/icuuc/uprops.pic.o' is generated by these conflicting actions:
Label: @icu//:icuuc
RuleClass: cc_library rule
Configuration: 98c6e8c3d595f000f2cc3fbc5194178d, cd31cd26593c7ea5d91cb6602ad66fc4
Mnemonic: CppCompile
Action key: ec9d8825160200c6b2b395858d5a2e86
Progress message: Compiling external/icu/icu4c/source/common/uprops.cpp
PrimaryInput: File:[/usr/local/google/home/pcloudy/.cache/bazel/_bazel_pcloudy/d2f15366e3b15100d3593f2d595721a3[source]]external/icu/icu4c/source/common/uprops.cpp
PrimaryOutput: File:[[<execution_root>]bazel-out/k8-py3-opt/bin]external/icu/_objs/icuuc/uprops.pic.o
Primary outputs are different: 972537493, 1038679363
Owner information: @icu//:icuuc BuildConfigurationValue.Key[98c6e8c3d595f000f2cc3fbc5194178d] false, @icu//:icuuc BuildConfigurationValue.Key[cd31cd26593c7ea5d91cb6602ad66fc4] false
MandatoryInputs: are equal
Outputs: Attempted action contains artifacts not in previous action (first 5): 
        external/icu/_objs/icuuc/uprops.pic.o
        external/icu/_objs/icuuc/uprops.pic.d
Previous action contains artifacts not in attempted action (first 5): 
        external/icu/_objs/icuuc/uprops.pic.o
        external/icu/_objs/icuuc/uprops.pic.d
ERROR: com.google.devtools.build.lib.actions.MutableActionGraph$ActionConflictException: for external/icu/_objs/icuuc/uprops.pic.o, previous action: action 'Compiling external/icu/icu4c/source/common/uprops.cpp', attempted action: action 'Compiling external/icu/icu4c/source/common/uprops.cpp'
INFO: Elapsed time: 11.820s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (396 packages loaded, 27672 targets configured)

This looks like a bug that has already been fixed at HEAD, but it also means we cannot flip this flag for 0.25.0, because users cannot use 0.24.0 for migration.

@brandjon
Copy link
Member Author

See the link in my last post -- I ran the Bazel CI using the downstream w/ bazel@head pipeline, on a custom build that flipped all four python-version-related flags (2 introduced in 0.23 and 2 in 0.24, all to be flipped in 0.25) and it was green. So I don't think migration is actually an issue for this change.

The action conflict issue is partially mitigated by turning on --incompatible_remove_old_python_version_api, which is one of those four flags to be flipped in 0.25. The only remaining action conflicts occur in the rarer case that a target is built both at the top-level configuration and at a config reached by transitioning away from the default Python version and then back again.

@brandjon
Copy link
Member Author

Ran another rerun of downstream at head + flag flips here. It's clean except for Tulsi which is broken at Bazel head, and Bazel's own presubmits are still running as I'm writing this. So I'm pretty comfortable with proceeding with these flips.

Note that I still have to update Bazels own unit tests for the flag flips. (This isn't reflected by the above presubmit since it modified the host blaze, not the blaze under test.)

katre pushed a commit that referenced this issue Mar 29, 2019
@katre
Release 0.24.0 (2019-03-26)
f092ec3 
Baseline: 235e76b

Cherry picks:

   + badd82e:
     Automated rollback of commit
     1b4c37c.
   + 33e5719:
     Fix the Python version select() mechanism to handle
     PY3-as-default
   + 56366ee:
     Set non-empty values for msvc_env_* when VC not installed
   + 22b3fbf:
     Windows, test wrapper: fix broken integration test
   + f14d447:
     Add whitelist file for starlark transitions
   + d99bc47:
     Update BUILD
   + 3529ad7:
     Rename tools/function_transition_whitelist/BUILD to
     tools/whitelists/function_transition_whitelist/BUILD
   + de0612a:
     Update bazel_toolchains to latest release and add toolchain
     config target for BuildKite CI (rbe_ubuntu1604)
   + 3e660ad:
     Automated rollback of commit
     0877340.
   + 314cf1f:
     Pass -undefined dynamic_lookup to dynamic library linking
     actions on Mac
   + fc586a8:
     Move cc_flags target into @bazel_tools//tools/cpp.
   + ea1703b:
     C++: Fix crash reported in #7721
   + 803801d:
     Pass execution info to xml generating spawn. Fixes #7794

Incompatible changes:

  - Added --incompatible_py3_is_default to test switching the default
    Python version to PY3 for py_binary/py_test targets that do not
    specify a version. See #7359.
  - //tools/cmd_line_differ has been renamed to //tools/aquery_differ
    & can now compare (in addition to command lines) inputs of
    actions given 2 aquery results.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - The flag --incompatible_list_based_execution_strategy_selection
    was added and is used to ease the migration to the new style of
    specifying
    execution strategy selection and fallback behavior. The
    documentation for
    this flag is here: #7480
  - Added --incompatible_py2_outputs_are_suffixed, for switching the
    bazel-bin symlink to point to Python 3 outputs instead of Python
    2 outputs. See
    [#7593](#7593).

New features:

  - Make actions.args() object chainable.
  - Added --incompatible_windows_style_arg_escaping flag: enables
    correct subprocess argument escaping on Windows. (No-op on other
    platforms.)
  - Added --incompatible_windows_escape_jvm_flags flag: enables
    correct java_binary.jvm_flags and java_test.jvm_flags
    tokenization and escaping on Windows. (No-op on other platforms.)

Important changes:

  - Allow running aquery against the current state of Skyframe
  - Added support for the "navigation" resource directory to Android
    resource processing in Bazel. This is used by the Navigation
    Architecture Component.
  - --incompatible_disable_runtimes_filegroups was flipped
    (#6942)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - --incompatible_disallow_filetype is enabled by default.
  - Fixed issue where exceptions and stacktraces were silently
    swallowed in the Android resource processing pipeline.
  - `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_disallow_dict_plus is enabled by default
  - Adds --incompatible_disable_objc_library_resources to disable
    resource attributes in objc_library. Please migrate these
    attributes to `data` instead.
  - --incompatible_disallow_old_style_args_add is enabled by default.
  - Using the `native` module in BUILD files is deprecated. It will
    be forbidden with --incompatible_disallow_native_in_build_file.
  - (Python rules) PyRuntimeInfo is exposed to Starlark, making it
    possible for Starlark rules to depend on or imitate `py_runtime`.
    The `files` attribute of `py_runtime` is no longer mandatory.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - is_using_fission crosstool variable is now exposed in all compile
    actions when fission is active (it used to be exposed only for
    linking actions).
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured
    targetshttps://github.com//issues/7186.
  - `py_runtime` gains a `python_version` attribute for specifying
    whether it represents a Python 2 or 3 interpreter.
  - `--incompatible_java_coverage` is enabled by default.
  - Starlark rules can safely declare attributes named "licenses"
  - When using
    --incompatible_list_based_execution_strategy_selection, Bazel
    will use remote execution by default (if you specify
    --remote_executor), otherwise persistent workers (if the action
    supports it), otherwise sandboxed local execution (if the action
    and platform supports it) and at last unsandboxed local
    execution. The flags --spawn_strategy and --strategy continue to
    work as before - this only sets new defaults for the case where
    you don't specify these flags.
  - Set default value of --incompatible_remap_main_repo to true.
  - Set default value of --incompatible_remap_main_repo to true.

This release contains contributions from many people at Google, as well as Andrew Suffield, Brandon Lico, Chris Eason, Clint Harrison, Ed Schouten, Garrett Hopper, George Gensure, Greg, John Millikin, Julie, Keith Smiley, Laurent Le Brun, Ryan Beasley, Shmuel H, Travis Cline, Vladimir Chebotarev.
bazel-io pushed a commit that referenced this issue May 1, 2019
@dkelmer
Release 0.25.0 (2019-05-01)
2ab3866 
Baseline: 0366246

Cherry picks:

   + 3f7f255:
     Windows: fix native test wrapper's arg. escaping
   + afeb8d0:
     Flip --incompatible_windows_escape_jvm_flags
   + 4299b65:
     Sort DirectoryNode children to ensure validity.
   + 231270c:
     Conditionally use deprecated signature for initWithContentsOfURL
   + 75a3a53:
     Add http_archive entries for testing with various JDK versions.
   + 4a6354a:
     Now that ubuntu1804 uses JDK 11, remove explicit
     ubuntu1804_java11 tests.
   + ae102fb:
     Fix wrong name of ubuntu1804_javabase9 task.
   + 0020a97:
     Remove @executable_path/Frameworks from rpaths
   + 130f86d:
     Download stderr/stdout to a temporary FileOutErr

Incompatible changes:

  - (Starlark rules) The legacy "py" provider can no longer be passed
    to or produced by native Python rules; use
    [PyInfo](https://docs.bazel.build/versions/master/skylark/lib/PyIn
    fo.html) instead. See
    [#7298](#7298) for more
    information.
  - (Python rules) The `default_python_version` attribute of the
    `py_binary` and `py_test` rules has been renamed to
    `python_version`. Also, the `--force_python` flag has been
    renamed to `--python_version`. See
    [#7308](#7308) for more
    information.
  - (Python rules) The python version now changes to whatever version
    is specified in a `py_binary` or `py_test`'s `python_version`
    attribute, instead of being forced to the value set by a command
    line flag. You can temporarily revert this change with
    `--incompatible_allow_python_version_transitions=false`. See
    [#7307](#7307) for more
    information.
  - --incompatible_disable_third_party_license_checking` is enabled
    by default
  - Introduced --incompatible_use_python_toolchains, which supersedes
    --python_top/--python_path. See #7899 and #7375 for more
    information.
  - Python 3 is now the default Python version (for `py_binary` and
    `py_test` targets that don't specify the `python_version`
    attribute). Targets that are built for Python 3 will no longer
    have their output put in a separate `-py3` directory; instead
    there is now a separate `-py2` directory for Python 2 targets.
    See #7359 and #7593 for more information.
  - objc_library resource attributes are now disabled by default.
    Please migrate them to data instead. See
    #7594 for more info.
  - Flip --incompatible_windows_escape_jvm_flags to true. See
    #7486

New features:

  - genrules now support a $(RULEDIR) variable that resolves to the
    directory where the outputs of the rule are put.
  - Added --incompatible_windows_native_test_wrapper flag: enables
    using the Bash-less test wrapper on Windows. (No-op on other
    platforms.)

Important changes:

  - incompatible_use_jdk11_as_host_javabase: makes JDK 11 the default
    --host_javabase for remote jdk
    (#7219)
  - Makes genquery somepath output deterministic.
  - Tristate attributes of native rules now reject True/False (use
    1/0)
  - Rollback of "Tristate attributes of native rules now reject
    True/False (use 1/0)"
  - Tristate attributes of native rules now reject True/False (use
    1/0)
  - Added -incompatible_do_not_split_linking_cmdline flag. See #7670
  - Tristate attributes of native rules now temporarily accept
    True/False again
  - `--incompatible_disable_legacy_crosstool_fields` has been flipped
    (#6861)
    `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - `--incompatible_disable_legacy_crosstool_fields` has been flipped
    (#6861)
    `--incompatible_disable_expand_if_all_available_in_flag_set...
    RELNOTES: None.
  - --incompatible_no_transitive_loads is enabled by default.
  - Makes TreeArtifact deterministic.
  - --incompatible_no_transitive_loads is enabled by default.
  - Android NDK C++ toolchain is now configured in Starlark. This
    should be a backwards compatible change, but in case of bugs
    blame unknown commit.
  - `--incompatible_disable_legacy_crosstool_fields` has been flipped
    (#6861)
    `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_no_transitive_loads is enabled by default.
  - --incompatible_bzl_disallow_load_after_statement is enabled
  - Added `--incompatible_require_ctx_in_configure_features`, see
    #7793 for details.
  - Flag --incompatible_merge_genfiles_directory is flipped. This
    removes the directory `bazel-genfiles` in favor of `bazel-bin`.
  - previously deprecated flag --experimental_remote_spawn_cache was
    removed
  - `--incompatible_disallow_load_labels_to_cross_package_boundaries`
    is enabled by default
  - Fix an issue where the Android resource processor did not surface
    errors from aapt2 compile and link actions.
  - --incompatible_no_attr_license is enabled by default
  - `--incompatible_disable_crosstool_file` has been flipped
    (#7320)
  - A new flag `--incompatible_string_join_requires_strings` is
    introduced. The sequence argument of `string.join` must contain
    only string elements.
  - --incompatible_symlinked_sandbox_expands_tree_artifacts_in_runfile
    s_tree has been flipped
  - Incompatible flag `--incompatible_disable_legacy_cc_provider` has
    been flipped (see #7036
    for details).
  - Don't drop the analysis cache when the same --define flag is set
    multiple times and the last value is the same (e.g. if the
    current invocation was run with "--define foo=bar" and the
    previous one was run with "--define foo=baz --define foo=bar").
  - The --incompatible_disable_genrule_cc_toolchain_dependency flag
    has been flipped (see
    #6867 for details).
  - Incompatible change
    `--incompatible_remove_cpu_and_compiler_attributes_from_cc_toolcha
    in` has been flipped (see
    #7075 for details).
  - --noexperimental_java_coverage is a no-op flag.
  - --experimental_java_coverage/--incompatible_java_coverage flag was
    removed. See #7425.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured targets
    (#7186.)
  - --incompatible_remote_symlinks has been flipped. The remote
    caching and execution protocol will now represent symlinks in
    outputs as such. See
    #7917 for more details.
  - Bazel is now ~20MiB smaller, from unbundling the Android rules'
    runtime dependencies.

This release contains contributions from many people at Google, as well as Andreas Herrmann, Andrew Suffield, Andy Scott, Benjamin Peterson, Ed Baunton, George Gensure, Ian McGinnis, Ity Kaul, Jingwen Chen, John Millikin, Keith Smiley, Marwan Tammam, Mike Fourie, Oscar Bonilla, perwestling, petros, Robert Sayre, Ryan Beasley, silvergasp, Stanimir Mladenov, Travis Cline, Vladimir Chebotarev, ??.
dkelmer pushed a commit that referenced this issue May 6, 2019
@dkelmer
Release 0.25.0 (2019-05-01)
f44bc47 
Baseline: 0366246

Cherry picks:

   + 3f7f255:
     Windows: fix native test wrapper's arg. escaping
   + afeb8d0:
     Flip --incompatible_windows_escape_jvm_flags
   + 4299b65:
     Sort DirectoryNode children to ensure validity.
   + 231270c:
     Conditionally use deprecated signature for initWithContentsOfURL
   + 75a3a53:
     Add http_archive entries for testing with various JDK versions.
   + 4a6354a:
     Now that ubuntu1804 uses JDK 11, remove explicit
     ubuntu1804_java11 tests.
   + ae102fb:
     Fix wrong name of ubuntu1804_javabase9 task.
   + 0020a97:
     Remove @executable_path/Frameworks from rpaths
   + 130f86d:
     Download stderr/stdout to a temporary FileOutErr

Incompatible changes:

  - (Starlark rules) The legacy "py" provider can no longer be passed
    to or produced by native Python rules; use
    [PyInfo](https://docs.bazel.build/versions/master/skylark/lib/PyIn
    fo.html) instead. See
    [#7298](#7298) for more
    information.
  - (Python rules) The `default_python_version` attribute of the
    `py_binary` and `py_test` rules has been renamed to
    `python_version`. Also, the `--force_python` flag has been
    renamed to `--python_version`. See
    [#7308](#7308) for more
    information.
  - (Python rules) The python version now changes to whatever version
    is specified in a `py_binary` or `py_test`'s `python_version`
    attribute, instead of being forced to the value set by a command
    line flag. You can temporarily revert this change with
    `--incompatible_allow_python_version_transitions=false`. See
    [#7307](#7307) for more
    information.
  - --incompatible_disable_third_party_license_checking` is enabled
    by default
  - Introduced --incompatible_use_python_toolchains, which supersedes
    --python_top/--python_path. See #7899 and #7375 for more
    information.
  - Python 3 is now the default Python version (for `py_binary` and
    `py_test` targets that don't specify the `python_version`
    attribute). Targets that are built for Python 3 will no longer
    have their output put in a separate `-py3` directory; instead
    there is now a separate `-py2` directory for Python 2 targets.
    See #7359 and #7593 for more information.
  - objc_library resource attributes are now disabled by default.
    Please migrate them to data instead. See
    #7594 for more info.
  - Flip --incompatible_windows_escape_jvm_flags to true. See
    #7486

New features:

  - genrules now support a $(RULEDIR) variable that resolves to the
    directory where the outputs of the rule are put.
  - Added --incompatible_windows_native_test_wrapper flag: enables
    using the Bash-less test wrapper on Windows. (No-op on other
    platforms.)

Important changes:

  - incompatible_use_jdk11_as_host_javabase: makes JDK 11 the default
    --host_javabase for remote jdk
    (#7219)
  - Makes genquery somepath output deterministic.
  - Tristate attributes of native rules now reject True/False (use
    1/0)
  - Rollback of "Tristate attributes of native rules now reject
    True/False (use 1/0)"
  - Tristate attributes of native rules now reject True/False (use
    1/0)
  - Added -incompatible_do_not_split_linking_cmdline flag. See #7670
  - Tristate attributes of native rules now temporarily accept
    True/False again
  - `--incompatible_disable_legacy_crosstool_fields` has been flipped
    (#6861)
    `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - `--incompatible_disable_legacy_crosstool_fields` has been flipped
    (#6861)
    `--incompatible_disable_expand_if_all_available_in_flag_set...
    RELNOTES: None.
  - --incompatible_no_transitive_loads is enabled by default.
  - Makes TreeArtifact deterministic.
  - --incompatible_no_transitive_loads is enabled by default.
  - Android NDK C++ toolchain is now configured in Starlark. This
    should be a backwards compatible change, but in case of bugs
    blame unknown commit.
  - `--incompatible_disable_legacy_crosstool_fields` has been flipped
    (#6861)
    `--incompatible_disable_expand_if_all_available_in_flag_set` has
    been flipped (#7008)
  - --incompatible_no_transitive_loads is enabled by default.
  - --incompatible_bzl_disallow_load_after_statement is enabled
  - Added `--incompatible_require_ctx_in_configure_features`, see
    #7793 for details.
  - Flag --incompatible_merge_genfiles_directory is flipped. This
    removes the directory `bazel-genfiles` in favor of `bazel-bin`.
  - previously deprecated flag --experimental_remote_spawn_cache was
    removed
  - `--incompatible_disallow_load_labels_to_cross_package_boundaries`
    is enabled by default
  - Fix an issue where the Android resource processor did not surface
    errors from aapt2 compile and link actions.
  - --incompatible_no_attr_license is enabled by default
  - `--incompatible_disable_crosstool_file` has been flipped
    (#7320)
  - A new flag `--incompatible_string_join_requires_strings` is
    introduced. The sequence argument of `string.join` must contain
    only string elements.
  - --incompatible_symlinked_sandbox_expands_tree_artifacts_in_runfile
    s_tree has been flipped
  - Incompatible flag `--incompatible_disable_legacy_cc_provider` has
    been flipped (see #7036
    for details).
  - Don't drop the analysis cache when the same --define flag is set
    multiple times and the last value is the same (e.g. if the
    current invocation was run with "--define foo=bar" and the
    previous one was run with "--define foo=baz --define foo=bar").
  - The --incompatible_disable_genrule_cc_toolchain_dependency flag
    has been flipped (see
    #6867 for details).
  - Incompatible change
    `--incompatible_remove_cpu_and_compiler_attributes_from_cc_toolcha
    in` has been flipped (see
    #7075 for details).
  - --noexperimental_java_coverage is a no-op flag.
  - --experimental_java_coverage/--incompatible_java_coverage flag was
    removed. See #7425.
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured targets
    (#7186.)
  - --incompatible_remote_symlinks has been flipped. The remote
    caching and execution protocol will now represent symlinks in
    outputs as such. See
    #7917 for more details.
  - Bazel is now ~20MiB smaller, from unbundling the Android rules'
    runtime dependencies.

This release contains contributions from many people at Google, as well as Andreas Herrmann, Andrew Suffield, Andy Scott, Benjamin Peterson, Ed Baunton, George Gensure, Ian McGinnis, Ity Kaul, Jingwen Chen, John Millikin, Keith Smiley, Marwan Tammam, Mike Fourie, Oscar Bonilla, perwestling, petros, Robert Sayre, Ryan Beasley, silvergasp, Stanimir Mladenov, Travis Cline, Vladimir Chebotarev, ??.
@EricBurnett EricBurnett mentioned this issue May 28, 2019
Closed
@cryslith cryslith mentioned this issue Jul 21, 2019
Closed
10 tasks
@clarketm clarketm mentioned this issue Aug 5, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brandjon brandjon

Labels
incompatible-change Incompatible/breaking change team-Rules-Python Native rules for Python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@greggdonovan @katre @brandjon @limdor @meteorcloudy