Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_disallow_legacy_py_provider: Use PyInfo instead of "py" struct providers #7298

Closed
brandjon opened this Issue Jan 30, 2019 · 3 comments

Comments

Projects
None yet
5 participants
@brandjon
Copy link
Member

brandjon commented Jan 30, 2019

Flag: --incompatible_disallow_legacy_py_provider
Available since: 0.23
Will be flipped in: either 0.24 or 0.25, depending on migration progress
Tracking issue: #7010

Motivation

Legacy struct providers have been deprecated for a long time, yet the native Python rules (py_library, py_binary, py_test) still use them internally and for sandwiching with Starlark-defined rules. As of 0.23, there is now a PyInfo provider that has the same fields as the struct. All rules should migrate to using PyInfo exclusively so we can help remove struct providers from Bazel altogether.

Change

When a dependency of a native Python rule has both the legacy provider and PyInfo provider, the legacy one is ignored.

When the incompatible flag is enabled, it is not even allowed for a dependency to pass the legacy provider to the native Python rules. Doing so produces an error like:

The legacy "py" provider is disallowed. Migrate to the PyInfo provider instead.

(Strictly speaking, we could have allowed this and just continued to ignore the legacy provider, but failing fast whenever we see "py" helps us locate rules that need to be migrated.

Migration

See Migrating from Legacy Providers for general information about supporting both legacy and modern providers within the same ruleset.

The recommended migration steps are to:

  1. Update rules to produce both providers (with identical field values).
  2. Update rules to accept either provider from their dependencies (but prefer PyInfo over "py").
  3. Update rules to no longer produce "py".
  4. Flip the flag in your build and validate that it works.
  5. Update rules to no longer care about "py" in their dependencies.

@brandjon brandjon self-assigned this Jan 30, 2019

bazel-io pushed a commit that referenced this issue Feb 1, 2019

Add PyInfo migration flag to --all_incompatible_changes
This renames --experimental_disallow_legacy_py_provider to --incompatible_disallow_legacy_py_provider and makes it available under --all_incompatible_changes. Migrate legacy "py" struct providers to PyInfo instead.

See #7298 for more information.

Work toward #7298 and #7010.

RELNOTES[INC]: Python rules will soon reject the legacy "py" struct provider (preview by enabling --incompatible_disallow_legacy_py_provider). Upgrade rules to use PyInfo instead. See [#7298](#7298).

PiperOrigin-RevId: 231885505

bazel-io pushed a commit that referenced this issue Feb 26, 2019

Release 0.23.0 (2019-02-26)
Baseline: 441fd75

Cherry picks:

   + 6ca7763:
     Fix a typo
   + 2310b1c:
     Ignore SIGCHLD in test setup script
   + f9eb1b5:
     Complete channel initialization in the event loop

Incompatible changes:

  - //src:bazel uses the minimal embedded JDK, if you want to
    avoid the extra steps of minimizing the JDK, use //src:bazel-dev
    instead.
  - //src:bazel uses the minimal embedded JDK, if you want to
    avoid the extra steps of building bazel with the minimized JDK,
    use //src:bazel-dev
    instead.
  - The default value of --host_platform and --platforms will be
      changed to not be dependent on the configuration. This means
    that setting
      --cpu or --host_cpu will not affect the target or host platform.
  - Toolchain resolution for cc rules is now enabled via an
    incompatible flag, --incompatible_enable_cc_toolchain_resolution.
    The previous
    flag, --enabled_toolchain_types, is deprecated and will be
    removed.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - Python rules will soon reject the legacy "py" struct provider
    (preview by enabling --incompatible_disallow_legacy_py_provider).
    Upgrade rules to use PyInfo instead. See
    [#7298](#7298).
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - Two changes to native Python rules: 1) `default_python_version`
    and `--force_python` are deprecated; use `python_version` and
    `--python_version` respectively instead. You can preview the
    removal of the deprecated names with
    --incompatible_remove_old_python_version_api. See
    [#7308](#7308). 2) The
    version flag will no longer override the declared version of a
    `py_binary` or `py_test` target. You can preview this new
    behavior with --incompatible_allow_python_version_transitions.
    See [#7307](#7307).

Important changes:

  - There is a new flag available
    `--experimental_java_common_create_provider_enabled_packages`
    that acts as a whitelist for usages of
    `java_common.create_provider`. The constructor will be deprecated
    in Bazel 0.23.
  - [#7024] Allow chaining of the same function type in aquery.
  - Introduces --local_{ram,cpu}_resources, which will take the place
    of --local_resources.
  - [#6930] Add documentation for the aquery command.
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (#6825)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (#6826)
  - Flag --incompatible_range_type is removed.
  - Flag --incompatible_disallow_slash_operator is removed.
  - Flag --incompatible_disallow_conflicting_providers is removed.
  - `--incompatible_disallow_data_transition` is now enabled by
    default
  - Allow inclusion of param files in aquery output
  - [#6985] Add test to verify aquery's behavior for Cpp action
    templates.
  - --incompatible_require_feature_configuration_for_pic was flipped
    (#7007).
  - Also ignore module-info.class in multi-version Jars
  - objc_framework has been deleted. Please refer to
    apple_dynamic_framework_import and apple_static_framework_import
    rules available in
    [rules_apple](https://github.com/bazelbuild/rules_apple/blob/maste
    r/doc/rules-general.md)
  - --test_sharding_strategy=experimental_heuristic is no more
  - objc_bundle_library has been removed. Please migrate to
    rules_apple's
    [apple_resource_bundle](https://github.com/bazelbuild/rules_apple/
    blob/master/doc/rules-resources.md#apple_resource_bundle).
  - You can now use the attribute `aapt_version` or the flag
    `--android_aapt` to pick the aapt version for android_local_test
    tests
  - In --keep_going mode, Bazel now correctly returns a non-zero exit
    code when encountering a package loading error during target
    pattern parsing of patterns like "//foo:all" and "//foo/...".
  - The default value for --incompatible_strict_action_env has been
    flipped to 'false' again, as we discovered breakages for local
    execution users. We'll need some more time to figure out the best
    way to make this work for local and remote execution. Follow
    #7026 for more details.
  - Locally-executed spawns tagged "no-cache" no longer upload their
    outputs to the remote cache.
  - Introduces --host_compiler flag to allow setting a compiler for
    host compilation when --host_crosstool_top is specified.
  - --incompatible_expand_directories is enabled by default
  - [aquery] Handle the case of aspect-on-aspect.
  - Fixed a longstanding bug in the http remote cache where the value
    passed to
    --remote_timeout would be interpreted as milliseconds instead of
    seconds.
  - Enable --incompatible_use_jdk10_as_host_javabase by default, see
    #6661
  - Add --incompatible_use_jdk11_as_host_javabase: makes JDK 11 the
    default --host_javabase for remote jdk
    (#7219)
  - Highlight TreeArtifact in aquery text output.
  - Locally-executed spawns tagged "no-cache" no longer upload their
    outputs to the remote cache.
  - java_common APIs now accept JavaToolchainInfo and JavaRuntimeInfo
    instead of configured targets for java_toolchain and java_runtime
  - cc_common.create_cc_toolchain_config_info is stable and available
    for production use
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured targets
    (#7186)
  - --incompatible_strict_argument_ordering is enabled by default.
  - Bazel now supports reading cache hits from a repository cache,
    even if it doesn't have write access to the cache.
  - Adding arm64e to OSX CROSSTOOL.
  - Ignore package-level licenses on config_setting.
  - Add an optional output_source_jar parameter to java_common.compile
  - --incompatible_disable_objc_provider_resources is now enabled by
    default. This disables ObjcProvider's fields related to resource
    processing.
  - Explicitly set https.protocols and exclude TLSv1.3.
  - Bazel now validates that JAVA_HOME points to a valid JDK and
    falls back to auto-detection by looking up the path of `javac`.
  - Upgrade the embedded JDK version to 11.0.2.
  - Added --incompatible_disable_crosstool_file
    (#7320)
  - --incompatible_disable_objc_provider_resources is now enabled by
    default. This disables ObjcProvider's fields related to resource
    processing.
  - --incompatible_disable_tools_defaults_package has been flipped.
  - For tests that do not generate a test.xml, Bazel now uses a
    separate action to generate one; this results in minor
    differences in the generated test.xml, and makes the generation
    more reliable overall.
  - incompatible_generate_javacommon_source_jar: java_common.compile
    now always generates a source jar, see
    #5824.
  - New incompatible flag
    --incompatible_disallow_struct_provider_syntax removes the
    ability for rule implementation functions to return struct. Such
    functions should return a list of providers instead. Migration
    tracking: #7347

This release contains contributions from many people at Google, as well as Benjamin Peterson, Ed Schouten, erenon, George Gensure, Greg Estren, Igal Tabachnik, Ittai Zeidman, Jannis Andrija Schnitzer, John Millikin, Keith Smiley, Kelly Campbell, Max Vorobev, nicolov, Robin Nabel.

excitoon pushed a commit to KasperskyLab/bazel that referenced this issue Feb 27, 2019

Release 0.23.0 (2019-02-26)
Baseline: 441fd75

Cherry picks:

   + 6ca7763:
     Fix a typo
   + 2310b1c:
     Ignore SIGCHLD in test setup script
   + f9eb1b5:
     Complete channel initialization in the event loop

Incompatible changes:

  - //src:bazel uses the minimal embedded JDK, if you want to
    avoid the extra steps of minimizing the JDK, use //src:bazel-dev
    instead.
  - //src:bazel uses the minimal embedded JDK, if you want to
    avoid the extra steps of building bazel with the minimized JDK,
    use //src:bazel-dev
    instead.
  - The default value of --host_platform and --platforms will be
      changed to not be dependent on the configuration. This means
    that setting
      --cpu or --host_cpu will not affect the target or host platform.
  - Toolchain resolution for cc rules is now enabled via an
    incompatible flag, --incompatible_enable_cc_toolchain_resolution.
    The previous
    flag, --enabled_toolchain_types, is deprecated and will be
    removed.
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - Python rules will soon reject the legacy "py" struct provider
    (preview by enabling --incompatible_disallow_legacy_py_provider).
    Upgrade rules to use PyInfo instead. See
    [bazelbuild#7298](bazelbuild#7298).
  - java_(mutable_|)proto_library: removed strict_deps attribute.
  - Two changes to native Python rules: 1) `default_python_version`
    and `--force_python` are deprecated; use `python_version` and
    `--python_version` respectively instead. You can preview the
    removal of the deprecated names with
    --incompatible_remove_old_python_version_api. See
    [bazelbuild#7308](bazelbuild#7308). 2) The
    version flag will no longer override the declared version of a
    `py_binary` or `py_test` target. You can preview this new
    behavior with --incompatible_allow_python_version_transitions.
    See [bazelbuild#7307](bazelbuild#7307).

Important changes:

  - There is a new flag available
    `--experimental_java_common_create_provider_enabled_packages`
    that acts as a whitelist for usages of
    `java_common.create_provider`. The constructor will be deprecated
    in Bazel 0.23.
  - [bazelbuild#7024] Allow chaining of the same function type in aquery.
  - Introduces --local_{ram,cpu}_resources, which will take the place
    of --local_resources.
  - [bazelbuild#6930] Add documentation for the aquery command.
  - Incompatible flag `--incompatible_dont_emit_static_libgcc` has
    been flipped (bazelbuild#6825)
  - Incompatible flag `--incompatible_linkopts_in_user_link_flags`
    has been flipped (bazelbuild#6826)
  - Flag --incompatible_range_type is removed.
  - Flag --incompatible_disallow_slash_operator is removed.
  - Flag --incompatible_disallow_conflicting_providers is removed.
  - `--incompatible_disallow_data_transition` is now enabled by
    default
  - Allow inclusion of param files in aquery output
  - [bazelbuild#6985] Add test to verify aquery's behavior for Cpp action
    templates.
  - --incompatible_require_feature_configuration_for_pic was flipped
    (bazelbuild#7007).
  - Also ignore module-info.class in multi-version Jars
  - objc_framework has been deleted. Please refer to
    apple_dynamic_framework_import and apple_static_framework_import
    rules available in
    [rules_apple](https://github.com/bazelbuild/rules_apple/blob/maste
    r/doc/rules-general.md)
  - --test_sharding_strategy=experimental_heuristic is no more
  - objc_bundle_library has been removed. Please migrate to
    rules_apple's
    [apple_resource_bundle](https://github.com/bazelbuild/rules_apple/
    blob/master/doc/rules-resources.md#apple_resource_bundle).
  - You can now use the attribute `aapt_version` or the flag
    `--android_aapt` to pick the aapt version for android_local_test
    tests
  - In --keep_going mode, Bazel now correctly returns a non-zero exit
    code when encountering a package loading error during target
    pattern parsing of patterns like "//foo:all" and "//foo/...".
  - The default value for --incompatible_strict_action_env has been
    flipped to 'false' again, as we discovered breakages for local
    execution users. We'll need some more time to figure out the best
    way to make this work for local and remote execution. Follow
    bazelbuild#7026 for more details.
  - Locally-executed spawns tagged "no-cache" no longer upload their
    outputs to the remote cache.
  - Introduces --host_compiler flag to allow setting a compiler for
    host compilation when --host_crosstool_top is specified.
  - --incompatible_expand_directories is enabled by default
  - [aquery] Handle the case of aspect-on-aspect.
  - Fixed a longstanding bug in the http remote cache where the value
    passed to
    --remote_timeout would be interpreted as milliseconds instead of
    seconds.
  - Enable --incompatible_use_jdk10_as_host_javabase by default, see
    bazelbuild#6661
  - Add --incompatible_use_jdk11_as_host_javabase: makes JDK 11 the
    default --host_javabase for remote jdk
    (bazelbuild#7219)
  - Highlight TreeArtifact in aquery text output.
  - Locally-executed spawns tagged "no-cache" no longer upload their
    outputs to the remote cache.
  - java_common APIs now accept JavaToolchainInfo and JavaRuntimeInfo
    instead of configured targets for java_toolchain and java_runtime
  - cc_common.create_cc_toolchain_config_info is stable and available
    for production use
  - incompatible_use_toolchain_providers_in_java_common: pass
    JavaToolchainInfo and JavaRuntimeInfo providers to java_common
    APIs instead of configured targets
    (bazelbuild#7186)
  - --incompatible_strict_argument_ordering is enabled by default.
  - Bazel now supports reading cache hits from a repository cache,
    even if it doesn't have write access to the cache.
  - Adding arm64e to OSX CROSSTOOL.
  - Ignore package-level licenses on config_setting.
  - Add an optional output_source_jar parameter to java_common.compile
  - --incompatible_disable_objc_provider_resources is now enabled by
    default. This disables ObjcProvider's fields related to resource
    processing.
  - Explicitly set https.protocols and exclude TLSv1.3.
  - Bazel now validates that JAVA_HOME points to a valid JDK and
    falls back to auto-detection by looking up the path of `javac`.
  - Upgrade the embedded JDK version to 11.0.2.
  - Added --incompatible_disable_crosstool_file
    (bazelbuild#7320)
  - --incompatible_disable_objc_provider_resources is now enabled by
    default. This disables ObjcProvider's fields related to resource
    processing.
  - --incompatible_disable_tools_defaults_package has been flipped.
  - For tests that do not generate a test.xml, Bazel now uses a
    separate action to generate one; this results in minor
    differences in the generated test.xml, and makes the generation
    more reliable overall.
  - incompatible_generate_javacommon_source_jar: java_common.compile
    now always generates a source jar, see
    bazelbuild#5824.
  - New incompatible flag
    --incompatible_disallow_struct_provider_syntax removes the
    ability for rule implementation functions to return struct. Such
    functions should return a list of providers instead. Migration
    tracking: bazelbuild#7347

This release contains contributions from many people at Google, as well as Benjamin Peterson, Ed Schouten, erenon, George Gensure, Greg Estren, Igal Tabachnik, Ittai Zeidman, Jannis Andrija Schnitzer, John Millikin, Keith Smiley, Kelly Campbell, Max Vorobev, nicolov, Robin Nabel.
@katre

This comment has been minimized.

Copy link
Member

katre commented Mar 1, 2019

This issue was tagged as "breaking-change-0.24" but does not appear ready to be flipped in the 0.24.0 release. If this is incorrect please comment on that issue and discuss with me.

@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 1, 2019

Changing target to 0.25.

bazel-io pushed a commit that referenced this issue Mar 4, 2019

Update tests to be compatible with --incompatible_disallow_legacy_py_…
…provider

Work toward #7298.

RELNOTES: None
PiperOrigin-RevId: 236711661
@brandjon

This comment has been minimized.

Copy link
Member Author

brandjon commented Mar 12, 2019

According to a recent run of the bazelisk CI, the only downstream failures due to this flag are fixed by PRs bazelbuild/rules_appengine#99 and bazelbuild/intellij#538.

Edit: Both issues are now closed. Pending a clean run of the bazelisk pipeline, I'll submit the flag flip (yay!).

@bazel-io bazel-io closed this in 575bb48 Mar 15, 2019

@lberki lberki added the bazel 1.0 label Mar 18, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.