Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

py_runtime does not work with --build_python_zip #5104

Closed
siddharthab opened this issue Apr 27, 2018 · 3 comments

Comments

Projects
None yet
4 participants
@siddharthab
Copy link
Contributor

commented Apr 27, 2018

Description of the problem:

The interpreter binary is copied over to the runfiles without executable permissions, only when --build_python_zip is true. In other cases, it works as expected. The checked-in source file does have executable permissions.

$ git clone https://github.com/erain/bazel-python-example
$ cd bazel-python-example
$ bazel run --python_top=//:python-3 :bin  # Works
...
$ bazel run --build_python_zip --python_top=//:python-3 :bin  # Does not work
...
OSError: [Errno 13] Permission denied: '/tmp/Bazel.runfiles_vdNLqf/runfiles/__main__/python-3/bin/python3'

What operating system are you running Bazel on?

macOS and Ubuntu

What's the output of bazel info release?

release 0.12.0-homebrew
release 0.12.0

Have you found anything relevant by searching the web?

No; just this discussion over using --build_python_zip vs using google/subpar.

@siddharthab siddharthab changed the title `py_runtime` does not work with `--build_python_zip` py_runtime does not work with --build_python_zip Apr 27, 2018

@brandjon

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

This is problematic because it means that enabling --incompatible_use_python_toolchains will break anyone using --build_python_zip.

But from what I can tell, the zip file actually does have the execute bit copied over, so something else is causing the failure.

@brandjon brandjon added P1 and removed P3 labels Apr 30, 2019

@brandjon

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

Possibly related to #7947.

@brandjon

This comment has been minimized.

Copy link
Member

commented May 1, 2019

This is due to a bug in Python's zipfile library, which is used to extract the zip file. The actual zip file has the correct permissions.

@bazel-io bazel-io closed this in 1b324c4 May 1, 2019

pull bot pushed a commit to Pandinosaurus/bazel that referenced this issue May 28, 2019

Release 0.26.0 (2019-05-28)
Baseline: daa8ae5

Cherry picks:

   + 61c7ffa:
     Automated rollback of commit
     87388e2.
   + 898d7b6:
     Add test for repository overrides, conflicting with managed
     directories being added when Bazel server is already started.
   + c2001a4:
     Automated rollback of commit
     bbe47a1.
   + e67c961:
     Fix a non-determinism in create_embedded_tools.py.
   + 81aefe7:
     Remove unsupported cpu attribute from cc_toolchains.
   + 597e289:
     remote: made CombinedCache a composition of Disk and Http Cache
   + 942f7cf:
     C++: Fixes bug in C++ API with external repo aspects
   + 85a5a2b:
     Configure @androidsdk//:emulator_x86 and :emulator_arm to point
     to the unified emulator binary
   + 9835cb4:
     Automated rollback of commit
     844e4e2.
   + c963ba2:
     Windows, Python: fix arg. esc. also in host config
   + a1ea487:
     Do not pre-cache changed files under managed directories
   + 7dc78cd:
     Add explicit execution and target constraints for autodiscovered
     cc t?
   + dd9ac13:
     Fix a bug when a relative path is used for the execution log
   + 0ff19c6:
     Fix StandaloneTestStrategy.appendStderr
   + 7f49531:
     Fix the autodetecting Python toolchain on Mac
   + ddce723:
     Avoid exporting PATH unnecessarily
   + 35dd05a:
     Allow Starlark rules to be able to use the `exec_compatible_with`

Incompatible changes:

  - Flip --incompatible_windows_escape_jvm_flags to true. See
    bazelbuild#7486
  - Flip --incompatible_windows_style_arg_escaping to true.  See
    bazelbuild#7454
  - --incompatible_windows_escape_jvm_flags is enabled by default,
    and the flag no longer exists
  - `--incompatible_no_output_attr_default` is enabled by default.
  - --incompatible_depset_union is enabled by default.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses bazelbuild#4815 (incorrect
    interpreter version used) on non-Windows platforms. You can
    temporarily opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See bazelbuild#7899 for information on declaring Python
    toolchains and migrating your code. As a side-benefit, this
    addresses bazelbuild#4815 (incorrect interpreter version used) on
    non-Windows platforms. You can temporarily opt out of this change
    with `--incompatible_use_python_toolchains=false`.

New features:

  - Windows, Python: the --incompatible_windows_escape_python_args
    flag (false by default) builds py_binary and py_test targets with
    correct command line argument escaping.
  - cquery supports --output=build

Important changes:

  - Allow debugging C++ features logic.
  - The --ios_multi_cpus, --watchos_cpus, --macos_cpus and tvos_cpus
    are now additive. This means that you can now split the
    --ios_multi_cpus=arm64,armv7 into --ios_multi_cpus=arm64 and
    --ios_multi_cpus=armv7.
  - Generated Go protobufs now depend on
    //net/proto2/go:proto_gendeps instead of //net/proto2/go:proto
  - Add new options --cs_fdo_instrument and --cs_profile to support
    LLVM's context-sensitive FDO (CSFDO).
  - Bazel C++ compile/link Starlark API. Can be used with
    experimental flag
    --experimental_cc_skylark_api_enabled_packages=<package_path>,<pac
    kage_path2>.
  - `cc_toolchain.static_runtime_lib` and
    `cc_toolchain.dynamic_runtime_lib` are now exposed to Starlark.
  - New flag `--incompatible_no_kwargs_in_build_files`. See
    bazelbuild#8021
  - struct.to_proto() converts dict into proto3 text message (map<,>).
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Flag `--incompatible_static_name_resolution_in_build_files` is
    added. See bazelbuild#8022
  - Add --incompatible_objc_framework_cleanup to control whether to
    enable some objc framework cleanup that changes the API.
    Specifically, the cleanup changes the objc provider API
    pertaining to frameworks.  This change is expected to be
    transparent to most users unless they write their own Starlark
    rules to handle frameworks.  See
    bazelbuild#7594 for details.
  - Added --incompatible_remove_binary_profile to disable the old
    binary
    profiles. Instead use the JSON profile format:
    https://docs.bazel.build/versions/master/skylark/performance.html#
    json-profile
  - Introducing --execution_log_binary_file and
    --execution_log_json_file that output a stable sorted execution
    log. They will offer a stable replacement to
    --experimental_execution_log_file.
  - Flag `--incompatible_disallow_old_octal_notation` is added. See
    //github.com/bazelbuild/issues/8059
  - Removes the
    --incompatible_disable_genrule_cc_toolchain_dependency flag.
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Incompatible flag
    `--incompatible_make_thinlto_command_lines_standalone` has been
    added. See bazelbuild#6791 for
    details.
  - objc_library does not support resource attributes any more.
    Please read bazelbuild#7594 for more info.
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - New platform_mappings ability to allow gradual flag to
    platforms/toolchains migration. See also
    bazelbuild#6426
  - Added support for compiling against fully qualified R classes
    from aar_import dependencies.
  - --tls_enabled flag is deprecated. Please provide 'grpcs' as a
    scheme in the URLs if TLS should be used for a remote connection.
  - Adds
    incompatible_disallow_rule_execution_platform_constraints_allowed,
     which
    disallows the use of the "execution_platform_constraints_allowed"
    attribute when defining new rules.
  - Flag `--incompatible_restrict_named_params` is added. See
    bazelbuild#8147 for details.
  - The glob function has a new argument `allow_empty`. When set to
    False, the glob fails when it doesn't match anything.
  - Adds the "disable_whole_archive_for_static_lib" feature to allow
    turning off legacy_whole_archive for individual targets.
  - C++ Starlark API for compilation and linking is no longer
    whitelisted
  - Update visibility advice in build-style
  - --incompatible_disable_objc_provider_resources is now enabled by
    default.
  - Fixed an issue where some `py_runtime`s were incompatible with
    using `--build_python_zip` (bazelbuild#5104).
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.

This release contains contributions from many people at Google, as well as Benjamin Peterson, Brian Topping, clyang82, Dave Lee, George Gensure, Greg Estren, Greg, Guro Bokum, Keith Smiley, Max Vorobev, Michael Hackner, Robert Brown, Robert Sayre, Ryan Beasley, Yannic.

aehlig added a commit that referenced this issue Jun 3, 2019

Release 0.26.0 (2019-05-28)
Baseline: daa8ae5

Cherry picks:

   + 61c7ffa:
     Automated rollback of commit
     87388e2.
   + 898d7b6:
     Add test for repository overrides, conflicting with managed
     directories being added when Bazel server is already started.
   + c2001a4:
     Automated rollback of commit
     bbe47a1.
   + e67c961:
     Fix a non-determinism in create_embedded_tools.py.
   + 81aefe7:
     Remove unsupported cpu attribute from cc_toolchains.
   + 597e289:
     remote: made CombinedCache a composition of Disk and Http Cache
   + 942f7cf:
     C++: Fixes bug in C++ API with external repo aspects
   + 85a5a2b:
     Configure @androidsdk//:emulator_x86 and :emulator_arm to point
     to the unified emulator binary
   + 9835cb4:
     Automated rollback of commit
     844e4e2.
   + c963ba2:
     Windows, Python: fix arg. esc. also in host config
   + a1ea487:
     Do not pre-cache changed files under managed directories
   + 7dc78cd:
     Add explicit execution and target constraints for autodiscovered
     cc t?
   + dd9ac13:
     Fix a bug when a relative path is used for the execution log
   + 0ff19c6:
     Fix StandaloneTestStrategy.appendStderr
   + 7f49531:
     Fix the autodetecting Python toolchain on Mac
   + ddce723:
     Avoid exporting PATH unnecessarily
   + 35dd05a:
     Allow Starlark rules to be able to use the `exec_compatible_with`

Incompatible changes:

  - Flip --incompatible_windows_escape_jvm_flags to true. See
    #7486
  - Flip --incompatible_windows_style_arg_escaping to true.  See
    #7454
  - --incompatible_windows_escape_jvm_flags is enabled by default,
    and the flag no longer exists
  - `--incompatible_no_output_attr_default` is enabled by default.
  - --incompatible_depset_union is enabled by default.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [#7899](#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses #4815 (incorrect
    interpreter version used) on non-Windows platforms. You can
    temporarily opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See #7899 for information on declaring Python
    toolchains and migrating your code. As a side-benefit, this
    addresses #4815 (incorrect interpreter version used) on
    non-Windows platforms. You can temporarily opt out of this change
    with `--incompatible_use_python_toolchains=false`.

New features:

  - Windows, Python: the --incompatible_windows_escape_python_args
    flag (false by default) builds py_binary and py_test targets with
    correct command line argument escaping.
  - cquery supports --output=build

Important changes:

  - Allow debugging C++ features logic.
  - The --ios_multi_cpus, --watchos_cpus, --macos_cpus and tvos_cpus
    are now additive. This means that you can now split the
    --ios_multi_cpus=arm64,armv7 into --ios_multi_cpus=arm64 and
    --ios_multi_cpus=armv7.
  - Generated Go protobufs now depend on
    //net/proto2/go:proto_gendeps instead of //net/proto2/go:proto
  - Add new options --cs_fdo_instrument and --cs_profile to support
    LLVM's context-sensitive FDO (CSFDO).
  - Bazel C++ compile/link Starlark API. Can be used with
    experimental flag
    --experimental_cc_skylark_api_enabled_packages=<package_path>,<pac
    kage_path2>.
  - `cc_toolchain.static_runtime_lib` and
    `cc_toolchain.dynamic_runtime_lib` are now exposed to Starlark.
  - New flag `--incompatible_no_kwargs_in_build_files`. See
    #8021
  - struct.to_proto() converts dict into proto3 text message (map<,>).
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Flag `--incompatible_static_name_resolution_in_build_files` is
    added. See #8022
  - Add --incompatible_objc_framework_cleanup to control whether to
    enable some objc framework cleanup that changes the API.
    Specifically, the cleanup changes the objc provider API
    pertaining to frameworks.  This change is expected to be
    transparent to most users unless they write their own Starlark
    rules to handle frameworks.  See
    #7594 for details.
  - Added --incompatible_remove_binary_profile to disable the old
    binary
    profiles. Instead use the JSON profile format:
    https://docs.bazel.build/versions/master/skylark/performance.html#
    json-profile
  - Introducing --execution_log_binary_file and
    --execution_log_json_file that output a stable sorted execution
    log. They will offer a stable replacement to
    --experimental_execution_log_file.
  - Flag `--incompatible_disallow_old_octal_notation` is added. See
    //github.com//issues/8059
  - Removes the
    --incompatible_disable_genrule_cc_toolchain_dependency flag.
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Incompatible flag
    `--incompatible_make_thinlto_command_lines_standalone` has been
    added. See #6791 for
    details.
  - objc_library does not support resource attributes any more.
    Please read #7594 for more info.
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See #7977 for more info.
  - New platform_mappings ability to allow gradual flag to
    platforms/toolchains migration. See also
    #6426
  - Added support for compiling against fully qualified R classes
    from aar_import dependencies.
  - --tls_enabled flag is deprecated. Please provide 'grpcs' as a
    scheme in the URLs if TLS should be used for a remote connection.
  - Adds
    incompatible_disallow_rule_execution_platform_constraints_allowed,
     which
    disallows the use of the "execution_platform_constraints_allowed"
    attribute when defining new rules.
  - Flag `--incompatible_restrict_named_params` is added. See
    #8147 for details.
  - The glob function has a new argument `allow_empty`. When set to
    False, the glob fails when it doesn't match anything.
  - Adds the "disable_whole_archive_for_static_lib" feature to allow
    turning off legacy_whole_archive for individual targets.
  - C++ Starlark API for compilation and linking is no longer
    whitelisted
  - Update visibility advice in build-style
  - --incompatible_disable_objc_provider_resources is now enabled by
    default.
  - Fixed an issue where some `py_runtime`s were incompatible with
    using `--build_python_zip` (#5104).
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See #7977 for more info.

This release contains contributions from many people at Google, as well as Benjamin Peterson, Brian Topping, clyang82, Dave Lee, George Gensure, Greg Estren, Greg, Guro Bokum, Keith Smiley, Max Vorobev, Michael Hackner, Robert Brown, Robert Sayre, Ryan Beasley, Yannic.

irengrig added a commit to irengrig/bazel that referenced this issue Jun 18, 2019

Release 0.26.0 (2019-05-28)
Baseline: daa8ae5

Cherry picks:

   + 61c7ffa:
     Automated rollback of commit
     87388e2.
   + 898d7b6:
     Add test for repository overrides, conflicting with managed
     directories being added when Bazel server is already started.
   + c2001a4:
     Automated rollback of commit
     bbe47a1.
   + e67c961:
     Fix a non-determinism in create_embedded_tools.py.
   + 81aefe7:
     Remove unsupported cpu attribute from cc_toolchains.
   + 597e289:
     remote: made CombinedCache a composition of Disk and Http Cache
   + 942f7cf:
     C++: Fixes bug in C++ API with external repo aspects
   + 85a5a2b:
     Configure @androidsdk//:emulator_x86 and :emulator_arm to point
     to the unified emulator binary
   + 9835cb4:
     Automated rollback of commit
     844e4e2.
   + c963ba2:
     Windows, Python: fix arg. esc. also in host config
   + a1ea487:
     Do not pre-cache changed files under managed directories
   + 7dc78cd:
     Add explicit execution and target constraints for autodiscovered
     cc t?
   + dd9ac13:
     Fix a bug when a relative path is used for the execution log
   + 0ff19c6:
     Fix StandaloneTestStrategy.appendStderr
   + 7f49531:
     Fix the autodetecting Python toolchain on Mac
   + ddce723:
     Avoid exporting PATH unnecessarily
   + 35dd05a:
     Allow Starlark rules to be able to use the `exec_compatible_with`

Incompatible changes:

  - Flip --incompatible_windows_escape_jvm_flags to true. See
    bazelbuild#7486
  - Flip --incompatible_windows_style_arg_escaping to true.  See
    bazelbuild#7454
  - --incompatible_windows_escape_jvm_flags is enabled by default,
    and the flag no longer exists
  - `--incompatible_no_output_attr_default` is enabled by default.
  - --incompatible_depset_union is enabled by default.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See
    [bazelbuild#7899](bazelbuild#7899) for
    information on declaring Python toolchains and migrating your
    code. As a side-benefit, this addresses bazelbuild#4815 (incorrect
    interpreter version used) on non-Windows platforms. You can
    temporarily opt out of this change with
    `--incompatible_use_python_toolchains=false`.
  - Python rules now determine the Python runtime using toolchains
    rather than `--python_top` and `--python_path`, which are
    deprecated. See bazelbuild#7899 for information on declaring Python
    toolchains and migrating your code. As a side-benefit, this
    addresses bazelbuild#4815 (incorrect interpreter version used) on
    non-Windows platforms. You can temporarily opt out of this change
    with `--incompatible_use_python_toolchains=false`.

New features:

  - Windows, Python: the --incompatible_windows_escape_python_args
    flag (false by default) builds py_binary and py_test targets with
    correct command line argument escaping.
  - cquery supports --output=build

Important changes:

  - Allow debugging C++ features logic.
  - The --ios_multi_cpus, --watchos_cpus, --macos_cpus and tvos_cpus
    are now additive. This means that you can now split the
    --ios_multi_cpus=arm64,armv7 into --ios_multi_cpus=arm64 and
    --ios_multi_cpus=armv7.
  - Generated Go protobufs now depend on
    //net/proto2/go:proto_gendeps instead of //net/proto2/go:proto
  - Add new options --cs_fdo_instrument and --cs_profile to support
    LLVM's context-sensitive FDO (CSFDO).
  - Bazel C++ compile/link Starlark API. Can be used with
    experimental flag
    --experimental_cc_skylark_api_enabled_packages=<package_path>,<pac
    kage_path2>.
  - `cc_toolchain.static_runtime_lib` and
    `cc_toolchain.dynamic_runtime_lib` are now exposed to Starlark.
  - New flag `--incompatible_no_kwargs_in_build_files`. See
    bazelbuild#8021
  - struct.to_proto() converts dict into proto3 text message (map<,>).
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Flag `--incompatible_static_name_resolution_in_build_files` is
    added. See bazelbuild#8022
  - Add --incompatible_objc_framework_cleanup to control whether to
    enable some objc framework cleanup that changes the API.
    Specifically, the cleanup changes the objc provider API
    pertaining to frameworks.  This change is expected to be
    transparent to most users unless they write their own Starlark
    rules to handle frameworks.  See
    bazelbuild#7594 for details.
  - Added --incompatible_remove_binary_profile to disable the old
    binary
    profiles. Instead use the JSON profile format:
    https://docs.bazel.build/versions/master/skylark/performance.html#
    json-profile
  - Introducing --execution_log_binary_file and
    --execution_log_json_file that output a stable sorted execution
    log. They will offer a stable replacement to
    --experimental_execution_log_file.
  - Flag `--incompatible_disallow_old_octal_notation` is added. See
    //github.com/bazelbuild/issues/8059
  - Removes the
    --incompatible_disable_genrule_cc_toolchain_dependency flag.
  - Android resource conflicts will no longer be reported between a
    strong attr resource and a weak attr resource, if the weak attr
    does not have format specified.
  - Incompatible flag
    `--incompatible_make_thinlto_command_lines_standalone` has been
    added. See bazelbuild#6791 for
    details.
  - objc_library does not support resource attributes any more.
    Please read bazelbuild#7594 for more info.
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.
  - New platform_mappings ability to allow gradual flag to
    platforms/toolchains migration. See also
    bazelbuild#6426
  - Added support for compiling against fully qualified R classes
    from aar_import dependencies.
  - --tls_enabled flag is deprecated. Please provide 'grpcs' as a
    scheme in the URLs if TLS should be used for a remote connection.
  - Adds
    incompatible_disallow_rule_execution_platform_constraints_allowed,
     which
    disallows the use of the "execution_platform_constraints_allowed"
    attribute when defining new rules.
  - Flag `--incompatible_restrict_named_params` is added. See
    bazelbuild#8147 for details.
  - The glob function has a new argument `allow_empty`. When set to
    False, the glob fails when it doesn't match anything.
  - Adds the "disable_whole_archive_for_static_lib" feature to allow
    turning off legacy_whole_archive for individual targets.
  - C++ Starlark API for compilation and linking is no longer
    whitelisted
  - Update visibility advice in build-style
  - --incompatible_disable_objc_provider_resources is now enabled by
    default.
  - Fixed an issue where some `py_runtime`s were incompatible with
    using `--build_python_zip` (bazelbuild#5104).
  - The `outputs` parameter of the `rule()` function is deprecated
    and attached to flag `--incompatible_no_rule_outputs_param`.
    Migrate rules to use `OutputGroupInfo` or `attr.output` instead.
    See bazelbuild#7977 for more info.

This release contains contributions from many people at Google, as well as Benjamin Peterson, Brian Topping, clyang82, Dave Lee, George Gensure, Greg Estren, Greg, Guro Bokum, Keith Smiley, Max Vorobev, Michael Hackner, Robert Brown, Robert Sayre, Ryan Beasley, Yannic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.