Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_make_thinlto_command_lines_standalone: Make ThinLTO actions fully configured in the CcToolchainInfo #6791

Open
hlopko opened this issue Nov 28, 2018 · 0 comments

Comments

Projects
None yet
3 participants
@hlopko
Copy link
Contributor

commented Nov 28, 2018

Flag: --incompatible_enable_cc_toolchain_resolution
Available since: 0.26
Will be flipped in: 1.0

Motivation

Before this flag, lto-indexing action reuses flags from plain C++ linking actions. lto-backend uses GCC tool from tool_paths, not action configs as it should. Because of this it's unnecessary hard to configure C++ toolchain that supports ThinLTO. Once this flag is flipped, we'll document how to configure C++ toolchain for ThinLTO, and add support for ThinLTO into Bazel's autoconfigured toolchain as part of #6926.

Migration

If your toolchain doesn't support ThinLTO you don't have to do anything. In case it does:

  1. You have to define action configs for these actions:
    • lto-backend
    • lto-index-executable
    • lto-index-dynamic-library
    • lto-index-nodeps-dynamic-library
  2. You have to mention these actions in all flag sets that you want to be expanded for lto.
  3. Once this flag is flipped, you can remove mentions of lto-indexing in your toolchain, it will no longer be used by Bazel.

@hlopko hlopko changed the title Make thinlto actions standalone Make thinlto Bazel-worthy Nov 28, 2018

@aiuto aiuto added the untriaged label Nov 28, 2018

@hlopko hlopko added P2 and removed untriaged labels Jan 9, 2019

@hlopko hlopko changed the title Make thinlto Bazel-worthy incompatible_make_thinlto_command_lines_standalone: Make ThinLTO actions fully configured in the CcToolchainInfo Apr 17, 2019

bazel-io pushed a commit that referenced this issue Apr 17, 2019

Make thinlto action command lines standalone
This cl introduces an incompatible flag `--incompatible_make_thinlto_command_lines_standalone`, which when flipped will change Bazel's handling of thinlto:

1) lto-backend action_config now has to be defined in the CcToolchainConfigInfo
2) lto-indexing action is split into:
    * lto-index-for-executable
    * lto-index-for-dynamic-library
    * lto-index-for-nodeps-dynamic-library
3) all these need an action_config too
4) lto-indexing action is completely ignored
5) lto indexing command line is now defined independently of c++-link-executable, c++-link-dynamic-library, and c++-link-nodeps-dynamic-library, flags are no longer reused.

#6791

RELNOTES: Incompatible flag `--incompatible_make_thinlto_command_lines_standalone` has been added. See #6791 for details.
PiperOrigin-RevId: 243988905

pull bot pushed a commit to Pandinosaurus/bazel that referenced this issue May 28, 2019

Release 0.26.0 (2019-05-28)
Baseline: daa8ae5

Cherry picks:

   + 61c7ffa:
     Automated rollback of commit
     87388e2.
   + 898d7b6:
     Add test for repository overrides, conflicting with managed
     directories being added when Bazel server is already started.
   + c2001a4:
     Automated rollback of commit
     bbe47a1.
   + e67c961:
     Fix a non-determinism in create_embedded_tools.py.
   + 81aefe7:
     Remove unsupported cpu attribute from cc_toolchains.
   + 597e289:
     remote: made CombinedCache a composition of Disk and Http Cache
   + 942f7cf:
     C++: Fixes bug in C++ API with external repo aspects
   + 85a5a2b:
     Configure @androidsdk//:emulator_x86 and :emulator_arm to point
     to the unified emulator binary
   + 9835cb4:
     Automated rollback of commit
     844e4e2.
   + c963ba2:
     Windows, Python: fix arg. esc. also in host config
   + a1ea487:
     Do not pre-cache changed files under managed directories
   + 7dc78cd:
     Add explicit execution and target constraints for autodiscovered
     cc t?
   + dd9ac13:
     Fix a bug when a relative path is used for the execution log
   + 0ff19c6:
     Fix StandaloneTestStrategy.appendStderr
   + 7f49531:
     Fix the autodetecting Python toolchain on Mac
   + ddce723:
     Avoid exporting PATH unnecessarily
   + 35dd05a:
     Allow Starlark rules to be able to use the `exec_compatible_with`

Incompatible changes:

  - Flip --incompatible_windows_escape_jvm_flags to true. See
    bazelbuild#7486
  - Flip --incompatible_windows_style_arg_escaping to true.  See
    bazelbuild#7454
  - --incompatible_windows_escape_jvm_flags is enabled by default,
    and the flag no longer exists
  - `--incompatible_no_output_attr_default` is enabled by default.
  - --incompatible_depset_union is enabled by default.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses bazelbuild#4815 (incorrect
    interpreter version used) on non-Windows platforms. You can
    temporarily opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See bazelbuild#7899 for information on declaring Python
    toolchains and migrating your code. As a side-benefit, this
    addresses bazelbuild#4815 (incorrect interpreter version used) on
    non-Windows platforms. You can temporarily opt out of this change
    with `--incompatible_use_python_toolchains=false`.

New features:

  - Windows, Python: the --incompatible_windows_escape_python_args
    flag (false by default) builds py_binary and py_test targets with
    correct command line argument escaping.
  - cquery supports --output=build

Important changes:

  - Allow debugging C++ features logic.
  - The --ios_multi_cpus, --watchos_cpus, --macos_cpus and tvos_cpus
    are now additive. This means that you can now split the
    --ios_multi_cpus=arm64,armv7 into --ios_multi_cpus=arm64 and
    --ios_multi_cpus=armv7.
  - Generated Go protobufs now depend on
    //net/proto2/go:proto_gendeps instead of //net/proto2/go:proto
  - Add new options --cs_fdo_instrument and --cs_profile to support
    LLVM's context-sensitive FDO (CSFDO).
  - Bazel C++ compile/link Starlark API. Can be used with
    experimental flag
    --experimental_cc_skylark_api_enabled_packages=<package_path>,<pac
    kage_path2>.
  - `cc_toolchain.static_runtime_lib` and
    `cc_toolchain.dynamic_runtime_lib` are now exposed to Starlark.
  - New flag `--incompatible_no_kwargs_in_build_files`. See
    bazelbuild#8021
  - struct.to_proto() converts dict into proto3 text message (map<,>).
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Flag `--incompatible_static_name_resolution_in_build_files` is
    added. See bazelbuild#8022
  - Add --incompatible_objc_framework_cleanup to control whether to
    enable some objc framework cleanup that changes the API.
    Specifically, the cleanup changes the objc provider API
    pertaining to frameworks.  This change is expected to be
    transparent to most users unless they write their own Starlark
    rules to handle frameworks.  See
    bazelbuild#7594 for details.
  - Added --incompatible_remove_binary_profile to disable the old
    binary
    profiles. Instead use the JSON profile format:
    https://docs.bazel.build/versions/master/skylark/performance.html#
    json-profile
  - Introducing --execution_log_binary_file and
    --execution_log_json_file that output a stable sorted execution
    log. They will offer a stable replacement to
    --experimental_execution_log_file.
  - Flag `--incompatible_disallow_old_octal_notation` is added. See
    //github.com/bazelbuild/issues/8059
  - Removes the
    --incompatible_disable_genrule_cc_toolchain_dependency flag.
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Incompatible flag
    `--incompatible_make_thinlto_command_lines_standalone` has been
    added. See bazelbuild#6791 for
    details.
  - objc_library does not support resource attributes any more.
    Please read bazelbuild#7594 for more info.
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - New platform_mappings ability to allow gradual flag to
    platforms/toolchains migration. See also
    bazelbuild#6426
  - Added support for compiling against fully qualified R classes
    from aar_import dependencies.
  - --tls_enabled flag is deprecated. Please provide 'grpcs' as a
    scheme in the URLs if TLS should be used for a remote connection.
  - Adds
    incompatible_disallow_rule_execution_platform_constraints_allowed,
     which
    disallows the use of the "execution_platform_constraints_allowed"
    attribute when defining new rules.
  - Flag `--incompatible_restrict_named_params` is added. See
    bazelbuild#8147 for details.
  - The glob function has a new argument `allow_empty`. When set to
    False, the glob fails when it doesn't match anything.
  - Adds the "disable_whole_archive_for_static_lib" feature to allow
    turning off legacy_whole_archive for individual targets.
  - C++ Starlark API for compilation and linking is no longer
    whitelisted
  - Update visibility advice in build-style
  - --incompatible_disable_objc_provider_resources is now enabled by
    default.
  - Fixed an issue where some `py_runtime`s were incompatible with
    using `--build_python_zip` (bazelbuild#5104).
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.

This release contains contributions from many people at Google, as well as Benjamin Peterson, Brian Topping, clyang82, Dave Lee, George Gensure, Greg Estren, Greg, Guro Bokum, Keith Smiley, Max Vorobev, Michael Hackner, Robert Brown, Robert Sayre, Ryan Beasley, Yannic.

aehlig added a commit that referenced this issue Jun 3, 2019

Release 0.26.0 (2019-05-28)
Baseline: daa8ae5

Cherry picks:

   + 61c7ffa:
     Automated rollback of commit
     87388e2.
   + 898d7b6:
     Add test for repository overrides, conflicting with managed
     directories being added when Bazel server is already started.
   + c2001a4:
     Automated rollback of commit
     bbe47a1.
   + e67c961:
     Fix a non-determinism in create_embedded_tools.py.
   + 81aefe7:
     Remove unsupported cpu attribute from cc_toolchains.
   + 597e289:
     remote: made CombinedCache a composition of Disk and Http Cache
   + 942f7cf:
     C++: Fixes bug in C++ API with external repo aspects
   + 85a5a2b:
     Configure @androidsdk//:emulator_x86 and :emulator_arm to point
     to the unified emulator binary
   + 9835cb4:
     Automated rollback of commit
     844e4e2.
   + c963ba2:
     Windows, Python: fix arg. esc. also in host config
   + a1ea487:
     Do not pre-cache changed files under managed directories
   + 7dc78cd:
     Add explicit execution and target constraints for autodiscovered
     cc t?
   + dd9ac13:
     Fix a bug when a relative path is used for the execution log
   + 0ff19c6:
     Fix StandaloneTestStrategy.appendStderr
   + 7f49531:
     Fix the autodetecting Python toolchain on Mac
   + ddce723:
     Avoid exporting PATH unnecessarily
   + 35dd05a:
     Allow Starlark rules to be able to use the `exec_compatible_with`

Incompatible changes:

  - Flip --incompatible_windows_escape_jvm_flags to true. See
    #7486
  - Flip --incompatible_windows_style_arg_escaping to true.  See
    #7454
  - --incompatible_windows_escape_jvm_flags is enabled by default,
    and the flag no longer exists
  - `--incompatible_no_output_attr_default` is enabled by default.
  - --incompatible_depset_union is enabled by default.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [#7899](#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses #4815 (incorrect
    interpreter version used) on non-Windows platforms. You can
    temporarily opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See #7899 for information on declaring Python
    toolchains and migrating your code. As a side-benefit, this
    addresses #4815 (incorrect interpreter version used) on
    non-Windows platforms. You can temporarily opt out of this change
    with `--incompatible_use_python_toolchains=false`.

New features:

  - Windows, Python: the --incompatible_windows_escape_python_args
    flag (false by default) builds py_binary and py_test targets with
    correct command line argument escaping.
  - cquery supports --output=build

Important changes:

  - Allow debugging C++ features logic.
  - The --ios_multi_cpus, --watchos_cpus, --macos_cpus and tvos_cpus
    are now additive. This means that you can now split the
    --ios_multi_cpus=arm64,armv7 into --ios_multi_cpus=arm64 and
    --ios_multi_cpus=armv7.
  - Generated Go protobufs now depend on
    //net/proto2/go:proto_gendeps instead of //net/proto2/go:proto
  - Add new options --cs_fdo_instrument and --cs_profile to support
    LLVM's context-sensitive FDO (CSFDO).
  - Bazel C++ compile/link Starlark API. Can be used with
    experimental flag
    --experimental_cc_skylark_api_enabled_packages=<package_path>,<pac
    kage_path2>.
  - `cc_toolchain.static_runtime_lib` and
    `cc_toolchain.dynamic_runtime_lib` are now exposed to Starlark.
  - New flag `--incompatible_no_kwargs_in_build_files`. See
    #8021
  - struct.to_proto() converts dict into proto3 text message (map<,>).
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Flag `--incompatible_static_name_resolution_in_build_files` is
    added. See #8022
  - Add --incompatible_objc_framework_cleanup to control whether to
    enable some objc framework cleanup that changes the API.
    Specifically, the cleanup changes the objc provider API
    pertaining to frameworks.  This change is expected to be
    transparent to most users unless they write their own Starlark
    rules to handle frameworks.  See
    #7594 for details.
  - Added --incompatible_remove_binary_profile to disable the old
    binary
    profiles. Instead use the JSON profile format:
    https://docs.bazel.build/versions/master/skylark/performance.html#
    json-profile
  - Introducing --execution_log_binary_file and
    --execution_log_json_file that output a stable sorted execution
    log. They will offer a stable replacement to
    --experimental_execution_log_file.
  - Flag `--incompatible_disallow_old_octal_notation` is added. See
    //github.com//issues/8059
  - Removes the
    --incompatible_disable_genrule_cc_toolchain_dependency flag.
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Incompatible flag
    `--incompatible_make_thinlto_command_lines_standalone` has been
    added. See #6791 for
    details.
  - objc_library does not support resource attributes any more.
    Please read #7594 for more info.
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See #7977 for more info.
  - New platform_mappings ability to allow gradual flag to
    platforms/toolchains migration. See also
    #6426
  - Added support for compiling against fully qualified R classes
    from aar_import dependencies.
  - --tls_enabled flag is deprecated. Please provide 'grpcs' as a
    scheme in the URLs if TLS should be used for a remote connection.
  - Adds
    incompatible_disallow_rule_execution_platform_constraints_allowed,
     which
    disallows the use of the "execution_platform_constraints_allowed"
    attribute when defining new rules.
  - Flag `--incompatible_restrict_named_params` is added. See
    #8147 for details.
  - The glob function has a new argument `allow_empty`. When set to
    False, the glob fails when it doesn't match anything.
  - Adds the "disable_whole_archive_for_static_lib" feature to allow
    turning off legacy_whole_archive for individual targets.
  - C++ Starlark API for compilation and linking is no longer
    whitelisted
  - Update visibility advice in build-style
  - --incompatible_disable_objc_provider_resources is now enabled by
    default.
  - Fixed an issue where some `py_runtime`s were incompatible with
    using `--build_python_zip` (#5104).
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See #7977 for more info.

This release contains contributions from many people at Google, as well as Benjamin Peterson, Brian Topping, clyang82, Dave Lee, George Gensure, Greg Estren, Greg, Guro Bokum, Keith Smiley, Max Vorobev, Michael Hackner, Robert Brown, Robert Sayre, Ryan Beasley, Yannic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.