Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_default_to_explicit_init_py #10076

Open
brandjon opened this issue Oct 21, 2019 · 1 comment
Open

incompatible_default_to_explicit_init_py #10076

brandjon opened this issue Oct 21, 2019 · 1 comment

Comments

@brandjon
Copy link
Member

@brandjon brandjon commented Oct 21, 2019

Flag: --incompatible_default_to_explicit_init_py
Available since: 1.2
Will be flipped in: ???
Feature tracking issue: #7386

Motivation

For py_binary and py_test targets, Bazel currently automatically creates empty __init__.py files in the runfiles tree wherever it is needed to import Python files. This is done in the ancestor directories of paths that contain Python files, up to but not including the top-level workspace directory, where an explicit __init__.py file must be used. Thus for example, if your py_binary target depends (directly or indirectly) on //pkg/subpkg:foo.py, but your workspace has no //pkg:__init__.py or //pkg/subpkg:__init__.py (or these files were not declared as dependencies), your target will behave as if they exist and are empty.

We want to deprecate this because it is magic at a distance. Python programmers are already used to creating __init__.py files in their source trees, so doing it behind their backs introduces confusion and changes the semantics of imports (since these directories will no longer be considered namespace packages). Eliminating this behavior also should allow us to remove some special runfiles logic.

Change

py_binary and py_test already have a legacy_create_init attribute which effectively defaults to true. With this flag enabled, the effective default becomes false. You can still opt back into true for targets that need it, but in the future we will do another incompatible change to remove the attribute altogether.

I say "effectively" true or false because before this flag was added, the attribute was an actual boolean, and now it is a tristate that defaults to auto, where auto means consult this flag. It is possible some .bzl macro that tries to introspect a py_binary or py_test's attributes dictionary (via native.existing_rules) will observe this change in attribute type, even without enabling the incompatible flag.

Migration

If your build depended on having empty __init__.py files automatically created, you should explicitly create these files in your source tree and add them to the srcs of the relevant py_library targets. If for whatever reason that's not feasible at the moment, you can temporarily opt out of this change on a per-target basis even when the flag itself is enabled, by explicitly adding legacy_create_init = True to your targets.

Timing

It is currently unclear how burdensome migration will be, so we do not yet know when we will flip this flag.

@brandjon

This comment has been minimized.

Copy link
Member Author

@brandjon brandjon commented Oct 24, 2019

Blocked by #10098 (among other things).

bazel-io pushed a commit that referenced this issue Oct 24, 2019
This effectively changes the default value of legacy_create_init to false.

Technically, the attribute type has changed from boolean (default true) to tristate (default auto). Therefore, it is possible to see a semantic change in code that introspects a py_binary's or py_test's attrs dict (via native.existing_rules). We think it is unlikely this will break anyone, and in any case it is not currently possible to gate a rule's definition on an incompatible change flag.

Closes #9271. Work toward #7386 and #10076.

RELNOTES: [Python] Added flag --incomaptible_default_to_explicit_init_py to switch the default value of legacy_create_init to True. With this flag enabled, your py_binary and py_test targets will no longer behave as if empty __init__.py files were implicitly littered in your runfiles tree. See [#10076](#10076).
PiperOrigin-RevId: 276526057
bazel-io pushed a commit that referenced this issue Nov 20, 2019
Baseline: 11deef7

Cherry picks:

   + c76c3e5:
     Replace macOS CC path with relative path
   + 63332eb:
     Hardcode path to dirname on macOS
   + ceadf0a:
     Add tool executables (from FilesToRunProvider) to action inputs.
   + dbe63b0:
     Fix some of the bazel Windows tools code to work with GCC.

Incompatible changes:

  - Tree artifacts and regular artifact paths can no longer overlap.

New features:

  - Added a special "_validation" output group to enable moving
    "validation actions" off the critical path of builds.

Important changes:

  - The query flag "--host_deps" (commonly used as "--nohost_deps")
    has been renamed to "--tool_deps", and now also removes
    dependencies in any execution configuration from being reported
    in the query output. The previous flag name is deprecated and
    will be removed in a future release.
  - The `cc_common.{compile,link}` APIs can now be used without
    passing the `--experimental_cc_skylark_api_enabled_packages` flag.
  - A list of log paths will be provided in build output.
  - Improve runfiles documentation.
  - Improve documentation on rule outputs.
  - BUILD/.bzl execution errors cause execution to stop, even at
    top-level
  - Multiple Starlark validation errors are reported in a single pass.
  - Introduce --experimental_nested_set_as_skykey_threshold
  - Blaze will prevent idle sleep during test and build actions. Note
    that this does not affect screen savers and will not keep a
    laptop awake if the user forces sleep or closes the lid. This is
    purely to avoid idle sleeping when the user is not interacting
    with the device.
  - Improve testing docs.
  - Incompatible flag
    `--incompatible_validate_top_level_header_inclusions` has been
    added. See #10047 for
    details.
  - Fix an aquery bug with handling malformed queries that crashes
    bazel.
  - List fields on CcLinkingOutputs.
  - [Python] Added flag --incomaptible_default_to_explicit_init_py to
    switch the default value of legacy_create_init to True. With this
    flag enabled, your py_binary and py_test targets will no longer
    behave as if empty __init__.py files were implicitly littered in
    your runfiles tree. See
    [#10076](#10076).
  - Fix documentation on allowed target names.
  - --target_platform_fallback now also applies to exec/host
    configurations
  - android_binary and android_libary can now depend on targets
    providing
    CcInfos.
  - Add support for tracking suspensions (sleeps or SIGSTOP) on macOS.
  - d8 dexers (both standalone and incremental) are now available for
    use.
  - Add Desugar support for FreezePeriod#<init>

This release contains contributions from many people at Google, as well as Alex Kirchhoff, Andrew Suffield, Asaf Flescher, Austin Schuh, Benjamin Peterson, Bor Kae Hwang, Brian Richardson, Christy Norman, Clint Harrison, Dan Halperin, Daniel Martn, Dave Lee, David Neil, David Ostrovsky, George Gensure, Greg Estren, Greg, Ira Shikhman, Jacob Parker, Jakub Bujny, John Millikin, John Millikin, Keith Smiley, Laurent Le Brun, marcohu, Marwan Tammam, Mostyn Bramley-Moore, Peter Mounce, Ruben Das, Stepan Koltsov, Thi Don, Thi, Tomasz Strejczek, Walt Panfil, Yannic Bonenberger, Zackary Lowery.
AlessandroPatti pushed a commit to uber-common/bazel that referenced this issue Nov 22, 2019
Baseline: 11deef7

Cherry picks:

   + c76c3e5:
     Replace macOS CC path with relative path
   + 63332eb:
     Hardcode path to dirname on macOS
   + ceadf0a:
     Add tool executables (from FilesToRunProvider) to action inputs.
   + dbe63b0:
     Fix some of the bazel Windows tools code to work with GCC.

Incompatible changes:

  - Tree artifacts and regular artifact paths can no longer overlap.

New features:

  - Added a special "_validation" output group to enable moving
    "validation actions" off the critical path of builds.

Important changes:

  - The query flag "--host_deps" (commonly used as "--nohost_deps")
    has been renamed to "--tool_deps", and now also removes
    dependencies in any execution configuration from being reported
    in the query output. The previous flag name is deprecated and
    will be removed in a future release.
  - The `cc_common.{compile,link}` APIs can now be used without
    passing the `--experimental_cc_skylark_api_enabled_packages` flag.
  - A list of log paths will be provided in build output.
  - Improve runfiles documentation.
  - Improve documentation on rule outputs.
  - BUILD/.bzl execution errors cause execution to stop, even at
    top-level
  - Multiple Starlark validation errors are reported in a single pass.
  - Introduce --experimental_nested_set_as_skykey_threshold
  - Blaze will prevent idle sleep during test and build actions. Note
    that this does not affect screen savers and will not keep a
    laptop awake if the user forces sleep or closes the lid. This is
    purely to avoid idle sleeping when the user is not interacting
    with the device.
  - Improve testing docs.
  - Incompatible flag
    `--incompatible_validate_top_level_header_inclusions` has been
    added. See bazelbuild#10047 for
    details.
  - Fix an aquery bug with handling malformed queries that crashes
    bazel.
  - List fields on CcLinkingOutputs.
  - [Python] Added flag --incomaptible_default_to_explicit_init_py to
    switch the default value of legacy_create_init to True. With this
    flag enabled, your py_binary and py_test targets will no longer
    behave as if empty __init__.py files were implicitly littered in
    your runfiles tree. See
    [bazelbuild#10076](bazelbuild#10076).
  - Fix documentation on allowed target names.
  - --target_platform_fallback now also applies to exec/host
    configurations
  - android_binary and android_libary can now depend on targets
    providing
    CcInfos.
  - Add support for tracking suspensions (sleeps or SIGSTOP) on macOS.
  - d8 dexers (both standalone and incremental) are now available for
    use.
  - Add Desugar support for FreezePeriod#<init>

This release contains contributions from many people at Google, as well as Alex Kirchhoff, Andrew Suffield, Asaf Flescher, Austin Schuh, Benjamin Peterson, Bor Kae Hwang, Brian Richardson, Christy Norman, Clint Harrison, Dan Halperin, Daniel Martn, Dave Lee, David Neil, David Ostrovsky, George Gensure, Greg Estren, Greg, Ira Shikhman, Jacob Parker, Jakub Bujny, John Millikin, John Millikin, Keith Smiley, Laurent Le Brun, marcohu, Marwan Tammam, Mostyn Bramley-Moore, Peter Mounce, Ruben Das, Stepan Koltsov, Thi Don, Thi, Tomasz Strejczek, Walt Panfil, Yannic Bonenberger, Zackary Lowery.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.