Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_disable_target_provider_fields: disallow access of providers on "target" objects by field syntax #9014

Open
c-parsons opened this issue Jul 30, 2019 · 3 comments

Comments

@c-parsons
Copy link
Contributor

commented Jul 30, 2019

Related: #7347 (--incompatible_disallow_struct_provider_syntax)

Flag: --incompatible_disable_target_provider_fields

Motivation

The motivation for this change is described in #6241 and #7347. The affected Starlark syntax has been deprecated for over a year, and this aids #7347 in migrating off of the deprecated syntax.
The deprecated syntax is explicitly disabled with the flag --incompatible_disallow_struct_provider_syntax (see #7347). Namely, one is not allowed to return a struct from a rule implementation function. Instead, the returned object must be a provider-instance object or a list of such objects.

That is,

  foo_info = FooProvider()
  return struct(foo_info = [foo_info])

is disallowed. Instead, use the following:

  foo_info = FooProvider()
  return [foo_info]

However, this is difficult to do in isolation. Namely, if my rule foo_rule uses the above deprecated syntax, and I have a rule bar_rule which depends on a target of foo_rule, then bar_rule's implementation might contain:

  ctx.attr.dep.foo_info

So, when foo_rule changes, bar_rule will break.
This flag, --incompatible_disallow_struct_provider_syntax, explicitly disables the bar_rule's syntax, which facilitates a multi-phase migration. See the proposed migration ordering below.

Migration

Here we outline the migration plan for --incompatible_disable_target_provider_fields and --incompatible_disallow_struct_provider_syntax:

STEP 1: Rule implementation functions which return providers use the old struct syntax must simultaneously support the new syntax and the old syntax. For example:

  foo_info = FooProvider()
  return struct(foo_info = [foo_info])

becomes:

  foo_info = FooProvider()
  return struct(foo_info = [foo_info], providers = [foo_info])

STEP 2: Rules which depend on rules using the deprecated syntax can, after step 1, access the provider using either ctx.attr.dep.foo_info or ctx.attr.dep[FooProvider]. They should migrate to using the ctx.attr.dep[FooProvider] syntax. This can and should be verified by using the --incompatible_disable_target_provider_fields flag.

STEP 3: Rules which needed to be migrated in step 1 can completely migrate to the new syntax. The syntax from step 1 becomes:

  foo_info = FooProvider()
  return [foo_info]

This becomes a safe migration if we're guaranteed that all users of the rule have --incompatible_disable_target_provider_fields enabled from step 2.
We can completely deprecate the old syntax using the flag --incompatible_disallow_struct_provider_syntax (#7347).

@c-parsons c-parsons self-assigned this Jul 30, 2019

bazel-io pushed a commit that referenced this issue Jul 31, 2019
Disallow field syntax to access a target's providers
The provider-key syntax should be used instead.

This is an incompatible change attached to new flag --incompatible_disable_target_provider_fields.

See #9014 for details.

RELNOTES: New incompatible flag --incompatible_disable_target_provider_fields removes the ability (in Starlark) to access a target's providers via the field syntax (for example, `ctx.attr.dep.my_provider`). The provider-key syntax should be used instead (for example, `ctx.attr.dep[MyProvider]`). See #9014 for details.
PiperOrigin-RevId: 260920114
@dslomov

This comment has been minimized.

Copy link
Contributor

commented Aug 5, 2019

Is this realistic for 1.0? Many rules will need to be migrated, and migration is non-trivial and non-automatic.

@Globegitter

This comment has been minimized.

Copy link

commented Aug 6, 2019

Yeah I do agree, especially given this flag has just been introduced unflipped for the upcoming 0.29 as far as I am aware. Maybe it is worth considering to postpone 1.0 a month to allow for smoother migration?

@laurentlb

This comment has been minimized.

Copy link
Member

commented Aug 6, 2019

It's a much simpler version of #7347 (basically, just the first step of it). I'm interested to hear if there's any migration difficulty with this new flag.

bazel-io pushed a commit that referenced this issue Aug 28, 2019
Release 0.29.0 (2019-08-28)
Baseline: 6c5ef53

Cherry picks:

   + 338829f:
     Fix retrying of SocketTimeoutExceptions in HttpConnector
   + 14651cd:
     Fallback to next urls if download fails in HttpDownloader
   + b7d300c:
     Fix incorrect stdout/stderr in remote action cache. Fixes #9072
   + 9602176:
     Automated rollback of commit
     0f0a0d5.
   + da557f9:
     Windows: fix "bazel run" argument quoting
   + ef8b6f6:
     Return JavaInfo from java proto aspects.
   + 209175f:
     Revert back to the old behavior of not creating a proto source
     root for generated .proto files.
   + 644060b:
     Fix PatchUtil for parsing special patch format
   + 067040d:
     Put the removal of the legacy repository-relative proto path
     behind the --incompatible_generated_protos_in_virtual_imports
     flag.
   + 76ed014:
     repository mapping lookup: convert to canonical name first

Important changes:

  - rule_test: fix Bazel 0.27 regression ("tags" attribute was
    ingored, #8723
  - Adds --incompatible_enable_execution_transition, which enables
    incremental migration of host attributes to exec attributes.
  - objc_proto_library rule has been deleted from Bazel.
  - repository_ctx.read is no longer restricted to files
        in the repository contructed.
  - tags 'no-remote', 'no-cache', 'no-remote-cache',
    'no-remote-exec', 'no-sandbox' are propagated now to the actions
    from targets when '--ncompatible_allow_tags_propagation' flag set
    to true. See #8830.
  - Adds flag
    --//tools/build_defs/pkg:incompatible_no_build_defs_pkg. This
    flag turns off the rules //tools/build_defs/pkg:{pkg_deb,
    pkg_rpm, pkg_tar}.
  - The Android NDK is now integrated with toolchains. To use them,
    pass the `--extra_toolchains=@androidndk//:all` flag or register
    them in your WORKSPACE with
    `register_toolchains("@androidndk//:all")`.
  - Stdout and stderr are checked to determine if output is going to a
    terminal. `--is_stderr_atty` is deprecated and `--isatty` is
    undeprecated.
  - --incompatible_load_proto_rules_from_bzl was added to forbid
    loading the native proto rules directly. See more on tracking
    issue #8922
  - Docker Sandbox now respects remote_default_platform_properties
  - pkg_deb, pkg_rpm & pkg_tar deprecation plan announced in the
    documentation.
  - The new java_tools release:
    * fixes #8614
    * exposes a new toolchain `@java_tools//:prebuilt_toolchain`
    which is using all the pre-built tools, including singlejar and
    ijar, even on remote execution. This toolchain should be used
    only when host and execution platform are the same, otherwise the
    binaries will not work on the execution platform.
  - java_common.compile supports specifying
    annotation_processor_additional_inputs and
    annotation_processor_additional_outputs for the Java compilation
    action for supporting annotation processors that consume or
    produce artifacts. Fixes #6415
  - There is now documentation on optimizing Android app build
    performance. Read it at
    https://docs.bazel.build/versions/0.29.0/android-build-performance
    .html
  - Execution log now respects --remote_default_platform_properties
  - Include a link to the relevant documenation on transitive Python
    version errors.
  - New incompatible flag
    --incompatible_disable_target_provider_fields removes the ability
    (in Starlark) to access a target's providers via the field syntax
    (for example, `ctx.attr.dep.my_provider`). The provider-key
    syntax should be used instead (for example,
    `ctx.attr.dep[MyProvider]`). See
    #9014 for details.
  - A new platform exec_properties is added to replace
    remote_execution_properties.
  - Added --incompatible_load_python_rules_from_bzl, which will be
    flipped in Bazel 1.0. See
    #9006.
  - add --break_build_on_parallel_dex2oat_failure to shortcut tests
    on dex2oat errors

This release contains contributions from many people at Google, as well as Alexander Ilyin, Arek Sredzki, Artem Zinnatullin, Benjamin Peterson, Fan Wu, John Millikin, Loo Rong Jie, Marwan Tammam, Oscar Bonilla, Peter Mounce, Sergio Rodriguez Orellana, Takeo Sawada, and Yannic Bonenberger.
buchgr pushed a commit to buchgr/bazel that referenced this issue Aug 30, 2019
Release 0.29.0 (2019-08-28)
Baseline: 6c5ef53

Cherry picks:

   + 338829f:
     Fix retrying of SocketTimeoutExceptions in HttpConnector
   + 14651cd:
     Fallback to next urls if download fails in HttpDownloader
   + b7d300c:
     Fix incorrect stdout/stderr in remote action cache. Fixes bazelbuild#9072
   + 9602176:
     Automated rollback of commit
     0f0a0d5.
   + da557f9:
     Windows: fix "bazel run" argument quoting
   + ef8b6f6:
     Return JavaInfo from java proto aspects.
   + 209175f:
     Revert back to the old behavior of not creating a proto source
     root for generated .proto files.
   + 644060b:
     Fix PatchUtil for parsing special patch format
   + 067040d:
     Put the removal of the legacy repository-relative proto path
     behind the --incompatible_generated_protos_in_virtual_imports
     flag.
   + 76ed014:
     repository mapping lookup: convert to canonical name first

Important changes:

  - rule_test: fix Bazel 0.27 regression ("tags" attribute was
    ingored, bazelbuild#8723
  - Adds --incompatible_enable_execution_transition, which enables
    incremental migration of host attributes to exec attributes.
  - objc_proto_library rule has been deleted from Bazel.
  - repository_ctx.read is no longer restricted to files
        in the repository contructed.
  - tags 'no-remote', 'no-cache', 'no-remote-cache',
    'no-remote-exec', 'no-sandbox' are propagated now to the actions
    from targets when '--ncompatible_allow_tags_propagation' flag set
    to true. See bazelbuild#8830.
  - Adds flag
    --//tools/build_defs/pkg:incompatible_no_build_defs_pkg. This
    flag turns off the rules //tools/build_defs/pkg:{pkg_deb,
    pkg_rpm, pkg_tar}.
  - The Android NDK is now integrated with toolchains. To use them,
    pass the `--extra_toolchains=@androidndk//:all` flag or register
    them in your WORKSPACE with
    `register_toolchains("@androidndk//:all")`.
  - Stdout and stderr are checked to determine if output is going to a
    terminal. `--is_stderr_atty` is deprecated and `--isatty` is
    undeprecated.
  - --incompatible_load_proto_rules_from_bzl was added to forbid
    loading the native proto rules directly. See more on tracking
    issue bazelbuild#8922
  - Docker Sandbox now respects remote_default_platform_properties
  - pkg_deb, pkg_rpm & pkg_tar deprecation plan announced in the
    documentation.
  - The new java_tools release:
    * fixes bazelbuild#8614
    * exposes a new toolchain `@java_tools//:prebuilt_toolchain`
    which is using all the pre-built tools, including singlejar and
    ijar, even on remote execution. This toolchain should be used
    only when host and execution platform are the same, otherwise the
    binaries will not work on the execution platform.
  - java_common.compile supports specifying
    annotation_processor_additional_inputs and
    annotation_processor_additional_outputs for the Java compilation
    action for supporting annotation processors that consume or
    produce artifacts. Fixes bazelbuild#6415
  - There is now documentation on optimizing Android app build
    performance. Read it at
    https://docs.bazel.build/versions/0.29.0/android-build-performance
    .html
  - Execution log now respects --remote_default_platform_properties
  - Include a link to the relevant documenation on transitive Python
    version errors.
  - New incompatible flag
    --incompatible_disable_target_provider_fields removes the ability
    (in Starlark) to access a target's providers via the field syntax
    (for example, `ctx.attr.dep.my_provider`). The provider-key
    syntax should be used instead (for example,
    `ctx.attr.dep[MyProvider]`). See
    bazelbuild#9014 for details.
  - A new platform exec_properties is added to replace
    remote_execution_properties.
  - Added --incompatible_load_python_rules_from_bzl, which will be
    flipped in Bazel 1.0. See
    bazelbuild#9006.
  - add --break_build_on_parallel_dex2oat_failure to shortcut tests
    on dex2oat errors

This release contains contributions from many people at Google, as well as Alexander Ilyin, Arek Sredzki, Artem Zinnatullin, Benjamin Peterson, Fan Wu, John Millikin, Loo Rong Jie, Marwan Tammam, Oscar Bonilla, Peter Mounce, Sergio Rodriguez Orellana, Takeo Sawada, and Yannic Bonenberger.

@dslomov dslomov removed the bazel 1.0 label Sep 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.