fix(local) Add api3 targets and additional defines. #3408
Conversation
Summary of ChangesHello @laramiel, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request resolves critical linking issues encountered in Blaze builds using local Python installations. It achieves this by significantly improving the mechanism for detecting and propagating Python runtime information, including ABI flags, necessary DLLs, and specific compiler defines. These changes are particularly beneficial for Windows environments and free-threaded Python setups, ensuring that the build system correctly identifies and utilizes the appropriate Python libraries and configurations, thereby preventing linking mismatches. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the local Python runtime configuration to better support abi3 targets and handle platform-specific defines and library requirements. The changes in get_local_runtime_info.py and local_runtime_repo_setup.bzl are well-implemented, improving modularity and correctness. However, I've identified a couple of critical errors in local_runtime_repo.bzl where non-existent functions are called from repo_utils, which would lead to failures. I've also suggested a fix for an incorrect docstring. Once these issues are addressed, this PR will be a solid improvement.
|
I'm likely to push a commit which mirrors this in tensorstore within a few hours. Edit: The PR landed google/tensorstore@32caaba |
…l python installs. In get_local_runtime_info.py: * detect abi3 vs. full abi libraries. * Ensure that returned libraries are unique. * Add additional dlls required by pythonXY.dll / pythonX.dll on windows. In local_runtime_repo_setup.bzl * More closely match hermetic_runtime_repo_setup * Add abi3 header targets. In local_runtime_repo.bzl * rework linking to local repository directories to handl abi3 and extra dlls. * Update parameters passed into local_runtime_repo_setup.bzl
| "abi_dynamic_libraries": _unique_basenames(abi_dynamic_libraries), | ||
| "abi_interface_libraries": _unique_basenames(abi_interface_libraries), | ||
| "abi_flags": abi_flags, | ||
| "shlib_suffix": ".dylib" if _IS_DARWIN else "", |
There was a problem hiding this comment.
This line looks weird? Shouldn't shlib_suffix be .so for linux, etc ?
Previously, it was empty for windows only. Now its empty for non-darwin
There was a problem hiding this comment.
It could be. The goal here is to allow bazel to use the python framework libraries, which don't end in the .dylib suffix (think Frameworks/Python), and need to be symlinked with the correct suffix due to the restrictions in cc_import / cc_library allowed file extensions. For linux and windows I think that we never have to change the suffix.
| # runtime configuration | ||
| py_runtime( | ||
| name = "_py3_runtime", | ||
| name = "py3_runtime", |
There was a problem hiding this comment.
is there a particular reason these various targets were made public?
There was a problem hiding this comment.
They were changed to mimic the hermetic toolchain names. I'm not sure whether there is a benefit to one vs. the other.
2 participants
Propagate defines and additional dll requirements for local python installs.
In get_local_runtime_info.py:
In local_runtime_repo_setup.bzl
In local_runtime_repo.bzl
Before these changes, some bazel builds using local Python fail to link properly.
This happens due to a mismatch in the interpreter and the python GIL DISABLED mode, or (on Windows), where both freethreaded and non-freethreaded libraries may attempt to be linked at the same time.