Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify the best set of default configuration for Android app full local build performance #8427

Open
jin opened this issue May 21, 2019 · 2 comments

Comments

Projects
None yet
1 participant
@jin
Copy link
Member

commented May 21, 2019

Using https://github.com/jin/android-projects/tree/master/big_connected as a reference: 30 Java-only and 30 Android modules.

No configuration

build --experimental_generate_json_trace_profile
build --profile profile.json.gz
build --experimental_json_trace_compression
INFO: Found 1 target...
Target //androidAppModule0:androidAppModule0 up-to-date:
  bazel-bin/androidAppModule0/androidAppModule0_deploy.jar
  bazel-bin/androidAppModule0/androidAppModule0_unsigned.apk
  bazel-bin/androidAppModule0/androidAppModule0.apk
INFO: Elapsed time: 73.765s, Critical Path: 44.46s
INFO: 652 processes: 417 linux-sandbox, 235 worker.
INFO: Build completed successfully, 691 total actions

profile.json.gz

Optimal configuration

With #8342 on HEAD:

build --experimental_strict_action_env

build --experimental_generate_json_trace_profile
build --profile profile.json.gz
build --experimental_json_trace_compression

build --spawn_strategy=standalone # disable sandboxing
build --genrule_strategy=standalone # disable sandboxing for genrules
build --strategy=DexBuilder=worker # enable persistent workers for dexing
build --worker_max_instances=DexBuilder=2 # limit 2 workers for dexing to exploit dexbuilder's cache locality
build --strategy=Javac=worker # enable persistent workers for java compilation
build --strategy=Desugar=worker # enable persistent workers for desugaring
build --persistent_android_resource_processor # enable persistent workers for android resource processing
build --android_aapt=aapt2
build --experimental_multi_threaded_digest # great for SSDs
build --nojava_header_compilation # this will definitely affect incremental build performance, this is here only for a proof of concept. Ideally we can turn the Java header compiler into a persistent worker for lower execution cost.
INFO: Found 1 target...
Target //androidAppModule0:androidAppModule0 up-to-date:
  bazel-bin/androidAppModule0/androidAppModule0_deploy.jar
  bazel-bin/androidAppModule0/androidAppModule0_unsigned.apk
  bazel-bin/androidAppModule0/androidAppModule0.apk
INFO: Elapsed time: 37.342s, Critical Path: 20.35s
INFO: 715 processes: 167 local, 548 worker.
INFO: Build completed successfully, 754 total actions

Using the right set of flags, we can halve the build time (73s -> 37s) on the same machine.

profile.json.gz

@jin

This comment has been minimized.

Copy link
Member Author

commented May 22, 2019

@jin jin self-assigned this May 23, 2019

bazel-io pushed a commit that referenced this issue May 29, 2019

Enable persistent workers for Desugar by default for Bazel
Default Desugar to use persistent workers as a sane default. Measured to reduce local build times by up to 20%.

RELNOTES: Android desugaring actions now use persistent workers by default. This has been measured to provide up to 20% reduction in build times. To disable it, use the `--strategy=Desugar=sandboxed` flag. See #8342 and #8427 for more details on local build speed optimization for Android apps.

Closes #8448.

PiperOrigin-RevId: 250531699
@jin

This comment has been minimized.

Copy link
Member Author

commented May 29, 2019

--strategy=Desugar=worker is now enabled by default for a flagless build.

6eda6b5

bazel-io pushed a commit that referenced this issue Jun 17, 2019

Release 0.27.0 (2019-06-17)
Baseline: 5935259

Cherry picks:

   + fe81b49:
     Support of using the absolute path profile for LLVM's Context
     Sensitive FDO
   + ce5e718:
     Demote OptionProcessor from globals
   + 3ed9d36:
     Properly wire up BES half-close. The lack thereof was a simple
     oversight.
   + 4ca768e:
     standardize graph output indentation with 2 spaces
   + aff189a:
     Make sure default Linux artifacts have an associated action,
     even when artifact names are altered.
   + 8c3b3fb:
     Failures early in package loading will now fail all --keep_going
     builds.
   + 123c68d:
     Warn in more cases of possible Python version mismatch in host
     config
   + 052167e:
     Add a non-strict autodetecting Python toolchain
   + 6ef6d87:
     Default java toolchain target
   + 50fa3ec:
     Fix problems with the non-strict Python toolchain
   + e2a626c:
     Automated rollback of commit
     bc6f7cb.
   + 6efc5b7:
     Treat existence of managed directories as a part of repository
     dirtiness.
   + 3a4be3c:
     Add /usr/local/bin to default PATH under strict action env
   + 5c1005c:
     Automated rollback of commit
     536a166.

Incompatible changes:

  - --incompatible_disable_objc_provider_resources no longer has
    effect. Use of deprecated resource fields on the Objc provider is
    now disallowed regardless of this flag.
  - deleted deprecated --experimental-remote-retry* flags, please use
    --remote_retries instead
  - flipped --incompatible_list_based_execution_strategy_selection
    flag to be true by default. See
    #7480 for details.
  - Octal integer literals in Starlark are required to start with
    "Oo".
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - Turn off binary style profile format.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_f...
  - cleanup that affects user provided apple frameworks is
    now enabled by default.  See
    #7944 for more info.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [#7899](#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses
    [#4815](#4815)
    (incorrect interpreter version used) on non-Windows platforms.
    Note however that some builds break due to getting the version
    they asked for -- consider setting `python_version = "PY2"` on
    Python 2 targets and `--host_force_python=PY2` if any Python 2
    targets are used in the host configuration. You can temporarily
    opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Depsets can't be iterated over unless they're converted to lists
    using the .to_list() method. Use
    --incompatible_depset_is_not_iterable=false to
    temporarily restore the previous behaviour.

New features:

  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    To use the new init code, you need Bazel 0.27 or newer. The old
    (longer) init code still works.

Important changes:

  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See #7977 for more info.
  - The --incompatible_disable_objc_library_resources flag is being
    removed. Please remove it from your configs as it otherwise will
    fail the build.
  - Add a generic additional_linker_inputs attribute on cc_binary
    rules.
  - Windows, C++ autoconfigure: BAZEL_VC and BAZEL_VS may now have
    quotes, so if you set these envvars' values in cmd.exe via
    TAB-completion then you no longer need to remove the surrounding
    quotes.
  - pkg_deb has new attributes: `config` and `templates` that can be
    used for integration with debconf
  - Allow cc_import() of a DLL with no interface library on Windows,
    used to document runtime dependencies.
  - All host-configured Python tools that are built for the wrong
    Python version will now emit a warning message when they exit
    with non-zero status. See #7899.
  - deprecated --remote_local_fallback_strategy. Use
    `--strategy=remote,local` instead. See
    #7480.
  - Introduce --incompatible_disable_native_android_rules flag
  - The Android desugaring actions now support a persistent worker
    mode for faster local build performance. Enable it with
    `--strategy=Desugar=worker`.
  - --incompatible_static_name_resolution_in_build_files is now
    enabled by default
  - --incompatible_disable_deprecated_attr_params is now enabled by
    default (#5818)
  - Repository containing autoconfigured C++ toolchain
    `@local_config_cc` has been split in 2 - see
    `local_config_cc_toolchains`.
  - --incompatible_string_join_requires_strings is now enabled by
    default
  - Flag --incompatible_new_actions_api is enabled by dewfault (#5825)
  - New flag `--incompatible_disallow_empty_glob`. See
    #8195
  - --incompatible_no_kwargs_in_build_files is enabled by default
  - Incompatible flag
    `--incompatible_require_ctx_in_configure_features` has been
    flipped. See #7793 for
    more information.
  - `BAZEL_USE_XCODE_TOOLCHAIN=1` tells Bazel not to look for Xcode to
    decide whether to enable toolchains for Apple rules, but to
    assume Xcode is
    available. Can be also used when building on Darwin and no C++ or
    ObjC is being
    built, so there is no need to detect Xcode.
  - Android desugaring actions now use persistent workers by default.
    This has been measured to provide up to 20% reduction in build
    times. To disable it, use the `--strategy=Desugar=sandboxed`
    flag. See #8342 and
    #8427 for more details
    on local build speed optimization for Android apps.
  - Fixed an issue with Android builds where `--fat_apk_cpu` doesn't
    pack all selected shared libraries from `aar_import` targets into
    the APK. See
    [#8283](#8283).
  - Turn on --experimental_starlark_config_transitions by default for
    starlark transitions (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-transitions for more info)
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)
  - Incompatible flag
    `--incompatible_dont_enable_host_nonhost_crosstool_features` has
    been flipped. See #7407
    for more information.
  - Added support for Android NDK 19 and 20.
  - Flip --incompatible_no_support_tools_in_action_inputs
  - --remote_executor, --remote_cache or --bes_backend=someurl.com
    would be treated as grpcs://someurl.com, if the
    --incompatible_tls_enabled_removed flag enabled. See
    #8061 for details.
  - Add new options --cs_fdo_absolute_path= to support the absolute
    path
    profile for LLVM's context-sensitive FDO.
  - When `--incompatible_strict_action_env` is enabled, the default
    `PATH` now includes `/usr/local/bin`.
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)

This release contains contributions from many people at Google, as well as Alex Thompson, Andy Scott, Benjamin Peterson, David McNett, Drew Gassaway, Ira Shikhman, James deBoer, Jay Vercellone, Jingwen Chen, Josh Rosen, Keith Smiley, Laurent Le Brun, Lee Mracek, Marwan Tammam, Matt Passell, Michael Hackner, Michal Majcherski, Patrick Niklaus, Peter Mounce, Ricky Pai, Steeve Morin, szakmary, Takuto Ikuta, Vladimir Chebotarev, Yen-Chi Chen.

irengrig added a commit to irengrig/bazel that referenced this issue Jun 18, 2019

Enable persistent workers for Desugar by default for Bazel
Default Desugar to use persistent workers as a sane default. Measured to reduce local build times by up to 20%.

RELNOTES: Android desugaring actions now use persistent workers by default. This has been measured to provide up to 20% reduction in build times. To disable it, use the `--strategy=Desugar=sandboxed` flag. See bazelbuild#8342 and bazelbuild#8427 for more details on local build speed optimization for Android apps.

Closes bazelbuild#8448.

PiperOrigin-RevId: 250531699

irengrig added a commit to irengrig/bazel that referenced this issue Jun 18, 2019

Release 0.27.0 (2019-06-17)
Baseline: 5935259

Cherry picks:

   + fe81b49:
     Support of using the absolute path profile for LLVM's Context
     Sensitive FDO
   + ce5e718:
     Demote OptionProcessor from globals
   + 3ed9d36:
     Properly wire up BES half-close. The lack thereof was a simple
     oversight.
   + 4ca768e:
     standardize graph output indentation with 2 spaces
   + aff189a:
     Make sure default Linux artifacts have an associated action,
     even when artifact names are altered.
   + 8c3b3fb:
     Failures early in package loading will now fail all --keep_going
     builds.
   + 123c68d:
     Warn in more cases of possible Python version mismatch in host
     config
   + 052167e:
     Add a non-strict autodetecting Python toolchain
   + 6ef6d87:
     Default java toolchain target
   + 50fa3ec:
     Fix problems with the non-strict Python toolchain
   + e2a626c:
     Automated rollback of commit
     bc6f7cb.
   + 6efc5b7:
     Treat existence of managed directories as a part of repository
     dirtiness.
   + 3a4be3c:
     Add /usr/local/bin to default PATH under strict action env
   + 5c1005c:
     Automated rollback of commit
     536a166.

Incompatible changes:

  - --incompatible_disable_objc_provider_resources no longer has
    effect. Use of deprecated resource fields on the Objc provider is
    now disallowed regardless of this flag.
  - deleted deprecated --experimental-remote-retry* flags, please use
    --remote_retries instead
  - flipped --incompatible_list_based_execution_strategy_selection
    flag to be true by default. See
    bazelbuild#7480 for details.
  - Octal integer literals in Starlark are required to start with
    "Oo".
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - Turn off binary style profile format.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_f...
  - cleanup that affects user provided apple frameworks is
    now enabled by default.  See
    bazelbuild#7944 for more info.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses
    [bazelbuild#4815](bazelbuild#4815)
    (incorrect interpreter version used) on non-Windows platforms.
    Note however that some builds break due to getting the version
    they asked for -- consider setting `python_version = "PY2"` on
    Python 2 targets and `--host_force_python=PY2` if any Python 2
    targets are used in the host configuration. You can temporarily
    opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Depsets can't be iterated over unless they're converted to lists
    using the .to_list() method. Use
    --incompatible_depset_is_not_iterable=false to
    temporarily restore the previous behaviour.

New features:

  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    To use the new init code, you need Bazel 0.27 or newer. The old
    (longer) init code still works.

Important changes:

  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - The --incompatible_disable_objc_library_resources flag is being
    removed. Please remove it from your configs as it otherwise will
    fail the build.
  - Add a generic additional_linker_inputs attribute on cc_binary
    rules.
  - Windows, C++ autoconfigure: BAZEL_VC and BAZEL_VS may now have
    quotes, so if you set these envvars' values in cmd.exe via
    TAB-completion then you no longer need to remove the surrounding
    quotes.
  - pkg_deb has new attributes: `config` and `templates` that can be
    used for integration with debconf
  - Allow cc_import() of a DLL with no interface library on Windows,
    used to document runtime dependencies.
  - All host-configured Python tools that are built for the wrong
    Python version will now emit a warning message when they exit
    with non-zero status. See bazelbuild#7899.
  - deprecated --remote_local_fallback_strategy. Use
    `--strategy=remote,local` instead. See
    bazelbuild#7480.
  - Introduce --incompatible_disable_native_android_rules flag
  - The Android desugaring actions now support a persistent worker
    mode for faster local build performance. Enable it with
    `--strategy=Desugar=worker`.
  - --incompatible_static_name_resolution_in_build_files is now
    enabled by default
  - --incompatible_disable_deprecated_attr_params is now enabled by
    default (bazelbuild#5818)
  - Repository containing autoconfigured C++ toolchain
    `@local_config_cc` has been split in 2 - see
    `local_config_cc_toolchains`.
  - --incompatible_string_join_requires_strings is now enabled by
    default
  - Flag --incompatible_new_actions_api is enabled by dewfault (bazelbuild#5825)
  - New flag `--incompatible_disallow_empty_glob`. See
    bazelbuild#8195
  - --incompatible_no_kwargs_in_build_files is enabled by default
  - Incompatible flag
    `--incompatible_require_ctx_in_configure_features` has been
    flipped. See bazelbuild#7793 for
    more information.
  - `BAZEL_USE_XCODE_TOOLCHAIN=1` tells Bazel not to look for Xcode to
    decide whether to enable toolchains for Apple rules, but to
    assume Xcode is
    available. Can be also used when building on Darwin and no C++ or
    ObjC is being
    built, so there is no need to detect Xcode.
  - Android desugaring actions now use persistent workers by default.
    This has been measured to provide up to 20% reduction in build
    times. To disable it, use the `--strategy=Desugar=sandboxed`
    flag. See bazelbuild#8342 and
    bazelbuild#8427 for more details
    on local build speed optimization for Android apps.
  - Fixed an issue with Android builds where `--fat_apk_cpu` doesn't
    pack all selected shared libraries from `aar_import` targets into
    the APK. See
    [bazelbuild#8283](bazelbuild#8283).
  - Turn on --experimental_starlark_config_transitions by default for
    starlark transitions (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-transitions for more info)
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)
  - Incompatible flag
    `--incompatible_dont_enable_host_nonhost_crosstool_features` has
    been flipped. See bazelbuild#7407
    for more information.
  - Added support for Android NDK 19 and 20.
  - Flip --incompatible_no_support_tools_in_action_inputs
  - --remote_executor, --remote_cache or --bes_backend=someurl.com
    would be treated as grpcs://someurl.com, if the
    --incompatible_tls_enabled_removed flag enabled. See
    bazelbuild#8061 for details.
  - Add new options --cs_fdo_absolute_path= to support the absolute
    path
    profile for LLVM's context-sensitive FDO.
  - When `--incompatible_strict_action_env` is enabled, the default
    `PATH` now includes `/usr/local/bin`.
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)

This release contains contributions from many people at Google, as well as Alex Thompson, Andy Scott, Benjamin Peterson, David McNett, Drew Gassaway, Ira Shikhman, James deBoer, Jay Vercellone, Jingwen Chen, Josh Rosen, Keith Smiley, Laurent Le Brun, Lee Mracek, Marwan Tammam, Matt Passell, Michael Hackner, Michal Majcherski, Patrick Niklaus, Peter Mounce, Ricky Pai, Steeve Morin, szakmary, Takuto Ikuta, Vladimir Chebotarev, Yen-Chi Chen.

buchgr pushed a commit to buchgr/bazel that referenced this issue Jun 23, 2019

Release 0.27.0 (2019-06-17)
Baseline: 5935259

Cherry picks:

   + fe81b49:
     Support of using the absolute path profile for LLVM's Context
     Sensitive FDO
   + ce5e718:
     Demote OptionProcessor from globals
   + 3ed9d36:
     Properly wire up BES half-close. The lack thereof was a simple
     oversight.
   + 4ca768e:
     standardize graph output indentation with 2 spaces
   + aff189a:
     Make sure default Linux artifacts have an associated action,
     even when artifact names are altered.
   + 8c3b3fb:
     Failures early in package loading will now fail all --keep_going
     builds.
   + 123c68d:
     Warn in more cases of possible Python version mismatch in host
     config
   + 052167e:
     Add a non-strict autodetecting Python toolchain
   + 6ef6d87:
     Default java toolchain target
   + 50fa3ec:
     Fix problems with the non-strict Python toolchain
   + e2a626c:
     Automated rollback of commit
     bc6f7cb.
   + 6efc5b7:
     Treat existence of managed directories as a part of repository
     dirtiness.
   + 3a4be3c:
     Add /usr/local/bin to default PATH under strict action env
   + 5c1005c:
     Automated rollback of commit
     536a166.

Incompatible changes:

  - --incompatible_disable_objc_provider_resources no longer has
    effect. Use of deprecated resource fields on the Objc provider is
    now disallowed regardless of this flag.
  - deleted deprecated --experimental-remote-retry* flags, please use
    --remote_retries instead
  - flipped --incompatible_list_based_execution_strategy_selection
    flag to be true by default. See
    bazelbuild#7480 for details.
  - Octal integer literals in Starlark are required to start with
    "Oo".
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - Turn off binary style profile format.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_f...
  - cleanup that affects user provided apple frameworks is
    now enabled by default.  See
    bazelbuild#7944 for more info.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses
    [bazelbuild#4815](bazelbuild#4815)
    (incorrect interpreter version used) on non-Windows platforms.
    Note however that some builds break due to getting the version
    they asked for -- consider setting `python_version = "PY2"` on
    Python 2 targets and `--host_force_python=PY2` if any Python 2
    targets are used in the host configuration. You can temporarily
    opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Depsets can't be iterated over unless they're converted to lists
    using the .to_list() method. Use
    --incompatible_depset_is_not_iterable=false to
    temporarily restore the previous behaviour.

New features:

  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    To use the new init code, you need Bazel 0.27 or newer. The old
    (longer) init code still works.

Important changes:

  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - The --incompatible_disable_objc_library_resources flag is being
    removed. Please remove it from your configs as it otherwise will
    fail the build.
  - Add a generic additional_linker_inputs attribute on cc_binary
    rules.
  - Windows, C++ autoconfigure: BAZEL_VC and BAZEL_VS may now have
    quotes, so if you set these envvars' values in cmd.exe via
    TAB-completion then you no longer need to remove the surrounding
    quotes.
  - pkg_deb has new attributes: `config` and `templates` that can be
    used for integration with debconf
  - Allow cc_import() of a DLL with no interface library on Windows,
    used to document runtime dependencies.
  - All host-configured Python tools that are built for the wrong
    Python version will now emit a warning message when they exit
    with non-zero status. See bazelbuild#7899.
  - deprecated --remote_local_fallback_strategy. Use
    `--strategy=remote,local` instead. See
    bazelbuild#7480.
  - Introduce --incompatible_disable_native_android_rules flag
  - The Android desugaring actions now support a persistent worker
    mode for faster local build performance. Enable it with
    `--strategy=Desugar=worker`.
  - --incompatible_static_name_resolution_in_build_files is now
    enabled by default
  - --incompatible_disable_deprecated_attr_params is now enabled by
    default (bazelbuild#5818)
  - Repository containing autoconfigured C++ toolchain
    `@local_config_cc` has been split in 2 - see
    `local_config_cc_toolchains`.
  - --incompatible_string_join_requires_strings is now enabled by
    default
  - Flag --incompatible_new_actions_api is enabled by dewfault (bazelbuild#5825)
  - New flag `--incompatible_disallow_empty_glob`. See
    bazelbuild#8195
  - --incompatible_no_kwargs_in_build_files is enabled by default
  - Incompatible flag
    `--incompatible_require_ctx_in_configure_features` has been
    flipped. See bazelbuild#7793 for
    more information.
  - `BAZEL_USE_XCODE_TOOLCHAIN=1` tells Bazel not to look for Xcode to
    decide whether to enable toolchains for Apple rules, but to
    assume Xcode is
    available. Can be also used when building on Darwin and no C++ or
    ObjC is being
    built, so there is no need to detect Xcode.
  - Android desugaring actions now use persistent workers by default.
    This has been measured to provide up to 20% reduction in build
    times. To disable it, use the `--strategy=Desugar=sandboxed`
    flag. See bazelbuild#8342 and
    bazelbuild#8427 for more details
    on local build speed optimization for Android apps.
  - Fixed an issue with Android builds where `--fat_apk_cpu` doesn't
    pack all selected shared libraries from `aar_import` targets into
    the APK. See
    [bazelbuild#8283](bazelbuild#8283).
  - Turn on --experimental_starlark_config_transitions by default for
    starlark transitions (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-transitions for more info)
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)
  - Incompatible flag
    `--incompatible_dont_enable_host_nonhost_crosstool_features` has
    been flipped. See bazelbuild#7407
    for more information.
  - Added support for Android NDK 19 and 20.
  - Flip --incompatible_no_support_tools_in_action_inputs
  - --remote_executor, --remote_cache or --bes_backend=someurl.com
    would be treated as grpcs://someurl.com, if the
    --incompatible_tls_enabled_removed flag enabled. See
    bazelbuild#8061 for details.
  - Add new options --cs_fdo_absolute_path= to support the absolute
    path
    profile for LLVM's context-sensitive FDO.
  - When `--incompatible_strict_action_env` is enabled, the default
    `PATH` now includes `/usr/local/bin`.
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)

This release contains contributions from many people at Google, as well as Alex Thompson, Andy Scott, Benjamin Peterson, David McNett, Drew Gassaway, Ira Shikhman, James deBoer, Jay Vercellone, Jingwen Chen, Josh Rosen, Keith Smiley, Laurent Le Brun, Lee Mracek, Marwan Tammam, Matt Passell, Michael Hackner, Michal Majcherski, Patrick Niklaus, Peter Mounce, Ricky Pai, Steeve Morin, szakmary, Takuto Ikuta, Vladimir Chebotarev, Yen-Chi Chen.

siberex added a commit to siberex/bazel that referenced this issue Jul 4, 2019

Release 0.27.0 (2019-06-17)
Baseline: 5935259

Cherry picks:

   + fe81b49:
     Support of using the absolute path profile for LLVM's Context
     Sensitive FDO
   + ce5e718:
     Demote OptionProcessor from globals
   + 3ed9d36:
     Properly wire up BES half-close. The lack thereof was a simple
     oversight.
   + 4ca768e:
     standardize graph output indentation with 2 spaces
   + aff189a:
     Make sure default Linux artifacts have an associated action,
     even when artifact names are altered.
   + 8c3b3fb:
     Failures early in package loading will now fail all --keep_going
     builds.
   + 123c68d:
     Warn in more cases of possible Python version mismatch in host
     config
   + 052167e:
     Add a non-strict autodetecting Python toolchain
   + 6ef6d87:
     Default java toolchain target
   + 50fa3ec:
     Fix problems with the non-strict Python toolchain
   + e2a626c:
     Automated rollback of commit
     bc6f7cb.
   + 6efc5b7:
     Treat existence of managed directories as a part of repository
     dirtiness.
   + 3a4be3c:
     Add /usr/local/bin to default PATH under strict action env
   + 5c1005c:
     Automated rollback of commit
     536a166.

Incompatible changes:

  - --incompatible_disable_objc_provider_resources no longer has
    effect. Use of deprecated resource fields on the Objc provider is
    now disallowed regardless of this flag.
  - deleted deprecated --experimental-remote-retry* flags, please use
    --remote_retries instead
  - flipped --incompatible_list_based_execution_strategy_selection
    flag to be true by default. See
    bazelbuild#7480 for details.
  - Octal integer literals in Starlark are required to start with
    "Oo".
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - Turn off binary style profile format.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_f...
  - cleanup that affects user provided apple frameworks is
    now enabled by default.  See
    bazelbuild#7944 for more info.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses
    [bazelbuild#4815](bazelbuild#4815)
    (incorrect interpreter version used) on non-Windows platforms.
    Note however that some builds break due to getting the version
    they asked for -- consider setting `python_version = "PY2"` on
    Python 2 targets and `--host_force_python=PY2` if any Python 2
    targets are used in the host configuration. You can temporarily
    opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Depsets can't be iterated over unless they're converted to lists
    using the .to_list() method. Use
    --incompatible_depset_is_not_iterable=false to
    temporarily restore the previous behaviour.

New features:

  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    To use the new init code, you need Bazel 0.27 or newer. The old
    (longer) init code still works.

Important changes:

  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - The --incompatible_disable_objc_library_resources flag is being
    removed. Please remove it from your configs as it otherwise will
    fail the build.
  - Add a generic additional_linker_inputs attribute on cc_binary
    rules.
  - Windows, C++ autoconfigure: BAZEL_VC and BAZEL_VS may now have
    quotes, so if you set these envvars' values in cmd.exe via
    TAB-completion then you no longer need to remove the surrounding
    quotes.
  - pkg_deb has new attributes: `config` and `templates` that can be
    used for integration with debconf
  - Allow cc_import() of a DLL with no interface library on Windows,
    used to document runtime dependencies.
  - All host-configured Python tools that are built for the wrong
    Python version will now emit a warning message when they exit
    with non-zero status. See bazelbuild#7899.
  - deprecated --remote_local_fallback_strategy. Use
    `--strategy=remote,local` instead. See
    bazelbuild#7480.
  - Introduce --incompatible_disable_native_android_rules flag
  - The Android desugaring actions now support a persistent worker
    mode for faster local build performance. Enable it with
    `--strategy=Desugar=worker`.
  - --incompatible_static_name_resolution_in_build_files is now
    enabled by default
  - --incompatible_disable_deprecated_attr_params is now enabled by
    default (bazelbuild#5818)
  - Repository containing autoconfigured C++ toolchain
    `@local_config_cc` has been split in 2 - see
    `local_config_cc_toolchains`.
  - --incompatible_string_join_requires_strings is now enabled by
    default
  - Flag --incompatible_new_actions_api is enabled by dewfault (bazelbuild#5825)
  - New flag `--incompatible_disallow_empty_glob`. See
    bazelbuild#8195
  - --incompatible_no_kwargs_in_build_files is enabled by default
  - Incompatible flag
    `--incompatible_require_ctx_in_configure_features` has been
    flipped. See bazelbuild#7793 for
    more information.
  - `BAZEL_USE_XCODE_TOOLCHAIN=1` tells Bazel not to look for Xcode to
    decide whether to enable toolchains for Apple rules, but to
    assume Xcode is
    available. Can be also used when building on Darwin and no C++ or
    ObjC is being
    built, so there is no need to detect Xcode.
  - Android desugaring actions now use persistent workers by default.
    This has been measured to provide up to 20% reduction in build
    times. To disable it, use the `--strategy=Desugar=sandboxed`
    flag. See bazelbuild#8342 and
    bazelbuild#8427 for more details
    on local build speed optimization for Android apps.
  - Fixed an issue with Android builds where `--fat_apk_cpu` doesn't
    pack all selected shared libraries from `aar_import` targets into
    the APK. See
    [bazelbuild#8283](bazelbuild#8283).
  - Turn on --experimental_starlark_config_transitions by default for
    starlark transitions (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-transitions for more info)
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)
  - Incompatible flag
    `--incompatible_dont_enable_host_nonhost_crosstool_features` has
    been flipped. See bazelbuild#7407
    for more information.
  - Added support for Android NDK 19 and 20.
  - Flip --incompatible_no_support_tools_in_action_inputs
  - --remote_executor, --remote_cache or --bes_backend=someurl.com
    would be treated as grpcs://someurl.com, if the
    --incompatible_tls_enabled_removed flag enabled. See
    bazelbuild#8061 for details.
  - Add new options --cs_fdo_absolute_path= to support the absolute
    path
    profile for LLVM's context-sensitive FDO.
  - When `--incompatible_strict_action_env` is enabled, the default
    `PATH` now includes `/usr/local/bin`.
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)

This release contains contributions from many people at Google, as well as Alex Thompson, Andy Scott, Benjamin Peterson, David McNett, Drew Gassaway, Ira Shikhman, James deBoer, Jay Vercellone, Jingwen Chen, Josh Rosen, Keith Smiley, Laurent Le Brun, Lee Mracek, Marwan Tammam, Matt Passell, Michael Hackner, Michal Majcherski, Patrick Niklaus, Peter Mounce, Ricky Pai, Steeve Morin, szakmary, Takuto Ikuta, Vladimir Chebotarev, Yen-Chi Chen.

irengrig added a commit to irengrig/bazel that referenced this issue Jul 15, 2019

Enable persistent workers for Desugar by default for Bazel
Default Desugar to use persistent workers as a sane default. Measured to reduce local build times by up to 20%.

RELNOTES: Android desugaring actions now use persistent workers by default. This has been measured to provide up to 20% reduction in build times. To disable it, use the `--strategy=Desugar=sandboxed` flag. See bazelbuild#8342 and bazelbuild#8427 for more details on local build speed optimization for Android apps.

Closes bazelbuild#8448.

PiperOrigin-RevId: 250531699

irengrig added a commit to irengrig/bazel that referenced this issue Jul 15, 2019

Release 0.27.0 (2019-06-17)
Baseline: 5935259

Cherry picks:

   + fe81b49:
     Support of using the absolute path profile for LLVM's Context
     Sensitive FDO
   + ce5e718:
     Demote OptionProcessor from globals
   + 3ed9d36:
     Properly wire up BES half-close. The lack thereof was a simple
     oversight.
   + 4ca768e:
     standardize graph output indentation with 2 spaces
   + aff189a:
     Make sure default Linux artifacts have an associated action,
     even when artifact names are altered.
   + 8c3b3fb:
     Failures early in package loading will now fail all --keep_going
     builds.
   + 123c68d:
     Warn in more cases of possible Python version mismatch in host
     config
   + 052167e:
     Add a non-strict autodetecting Python toolchain
   + 6ef6d87:
     Default java toolchain target
   + 50fa3ec:
     Fix problems with the non-strict Python toolchain
   + e2a626c:
     Automated rollback of commit
     bc6f7cb.
   + 6efc5b7:
     Treat existence of managed directories as a part of repository
     dirtiness.
   + 3a4be3c:
     Add /usr/local/bin to default PATH under strict action env
   + 5c1005c:
     Automated rollback of commit
     536a166.

Incompatible changes:

  - --incompatible_disable_objc_provider_resources no longer has
    effect. Use of deprecated resource fields on the Objc provider is
    now disallowed regardless of this flag.
  - deleted deprecated --experimental-remote-retry* flags, please use
    --remote_retries instead
  - flipped --incompatible_list_based_execution_strategy_selection
    flag to be true by default. See
    bazelbuild#7480 for details.
  - Octal integer literals in Starlark are required to start with
    "Oo".
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_file=false as a flag to
    Blaze.
  - Turn off binary style profile format.
  - The "native" module is no longer available in BUILD files, all
    its members can be accessed directly as global symbols. This can
    be temporarily reverted by providing
    --incompatible_disallow_native_in_build_f...
  - cleanup that affects user provided apple frameworks is
    now enabled by default.  See
    bazelbuild#7944 for more info.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses
    [bazelbuild#4815](bazelbuild#4815)
    (incorrect interpreter version used) on non-Windows platforms.
    Note however that some builds break due to getting the version
    they asked for -- consider setting `python_version = "PY2"` on
    Python 2 targets and `--host_force_python=PY2` if any Python 2
    targets are used in the host configuration. You can temporarily
    opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Depsets can't be iterated over unless they're converted to lists
    using the .to_list() method. Use
    --incompatible_depset_is_not_iterable=false to
    temporarily restore the previous behaviour.

New features:

  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    The rlocation() implementation is the same.
  - Bash, runfiles: the copy-pasted init code of the Bash runfiles
    library is now shorter, see `tools/bash/runfiles/runfiles.bash`.
    To use the new init code, you need Bazel 0.27 or newer. The old
    (longer) init code still works.

Important changes:

  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - The --incompatible_disable_objc_library_resources flag is being
    removed. Please remove it from your configs as it otherwise will
    fail the build.
  - Add a generic additional_linker_inputs attribute on cc_binary
    rules.
  - Windows, C++ autoconfigure: BAZEL_VC and BAZEL_VS may now have
    quotes, so if you set these envvars' values in cmd.exe via
    TAB-completion then you no longer need to remove the surrounding
    quotes.
  - pkg_deb has new attributes: `config` and `templates` that can be
    used for integration with debconf
  - Allow cc_import() of a DLL with no interface library on Windows,
    used to document runtime dependencies.
  - All host-configured Python tools that are built for the wrong
    Python version will now emit a warning message when they exit
    with non-zero status. See bazelbuild#7899.
  - deprecated --remote_local_fallback_strategy. Use
    `--strategy=remote,local` instead. See
    bazelbuild#7480.
  - Introduce --incompatible_disable_native_android_rules flag
  - The Android desugaring actions now support a persistent worker
    mode for faster local build performance. Enable it with
    `--strategy=Desugar=worker`.
  - --incompatible_static_name_resolution_in_build_files is now
    enabled by default
  - --incompatible_disable_deprecated_attr_params is now enabled by
    default (bazelbuild#5818)
  - Repository containing autoconfigured C++ toolchain
    `@local_config_cc` has been split in 2 - see
    `local_config_cc_toolchains`.
  - --incompatible_string_join_requires_strings is now enabled by
    default
  - Flag --incompatible_new_actions_api is enabled by dewfault (bazelbuild#5825)
  - New flag `--incompatible_disallow_empty_glob`. See
    bazelbuild#8195
  - --incompatible_no_kwargs_in_build_files is enabled by default
  - Incompatible flag
    `--incompatible_require_ctx_in_configure_features` has been
    flipped. See bazelbuild#7793 for
    more information.
  - `BAZEL_USE_XCODE_TOOLCHAIN=1` tells Bazel not to look for Xcode to
    decide whether to enable toolchains for Apple rules, but to
    assume Xcode is
    available. Can be also used when building on Darwin and no C++ or
    ObjC is being
    built, so there is no need to detect Xcode.
  - Android desugaring actions now use persistent workers by default.
    This has been measured to provide up to 20% reduction in build
    times. To disable it, use the `--strategy=Desugar=sandboxed`
    flag. See bazelbuild#8342 and
    bazelbuild#8427 for more details
    on local build speed optimization for Android apps.
  - Fixed an issue with Android builds where `--fat_apk_cpu` doesn't
    pack all selected shared libraries from `aar_import` targets into
    the APK. See
    [bazelbuild#8283](bazelbuild#8283).
  - Turn on --experimental_starlark_config_transitions by default for
    starlark transitions (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-transitions for more info)
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)
  - Incompatible flag
    `--incompatible_dont_enable_host_nonhost_crosstool_features` has
    been flipped. See bazelbuild#7407
    for more information.
  - Added support for Android NDK 19 and 20.
  - Flip --incompatible_no_support_tools_in_action_inputs
  - --remote_executor, --remote_cache or --bes_backend=someurl.com
    would be treated as grpcs://someurl.com, if the
    --incompatible_tls_enabled_removed flag enabled. See
    bazelbuild#8061 for details.
  - Add new options --cs_fdo_absolute_path= to support the absolute
    path
    profile for LLVM's context-sensitive FDO.
  - When `--incompatible_strict_action_env` is enabled, the default
    `PATH` now includes `/usr/local/bin`.
  - Turn on --experimental_build_setting_api by default for starlark
    build settings (see
    https://docs.bazel.build/versions/master/skylark/config.html#user-
    defined-build-settings for more info)

This release contains contributions from many people at Google, as well as Alex Thompson, Andy Scott, Benjamin Peterson, David McNett, Drew Gassaway, Ira Shikhman, James deBoer, Jay Vercellone, Jingwen Chen, Josh Rosen, Keith Smiley, Laurent Le Brun, Lee Mracek, Marwan Tammam, Matt Passell, Michael Hackner, Michal Majcherski, Patrick Niklaus, Peter Mounce, Ricky Pai, Steeve Morin, szakmary, Takuto Ikuta, Vladimir Chebotarev, Yen-Chi Chen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.