Skip to content

[Build] Replace rules_boost lzma patches with a custom org_lzma_lzma build file#61669

Merged
aslonnie merged 1 commit intomasterfrom
andrew/revup/master/lzma-custom-build-file
Mar 14, 2026
Merged

[Build] Replace rules_boost lzma patches with a custom org_lzma_lzma build file#61669
aslonnie merged 1 commit intomasterfrom
andrew/revup/master/lzma-custom-build-file

Conversation

@andrew-anyscale
Copy link
Contributor

@andrew-anyscale andrew-anyscale commented Mar 11, 2026

Instead of patching rules_boost's BUILD.lzma at dependency-resolution time,
declare org_lzma_lzma before com_github_nelhage_rules_boost so that
boost_deps()'s maybe() skips it. A local build_file label
(//thirdparty/patches:org_lzma_lzma.BUILD.bazel) forces content-based cache
invalidation: changing the file triggers re-setup on all machines, including
Windows CI with a persistent output base where patching an external repo's
BUILD file in-place would not propagate to already-extracted directories.

The custom build file also fixes Windows MSVC D8004 by using -Ipath copts
(no space between -I and the path) and replaces the includes attribute with
strip_include_prefix for more reliable include path handling.

Topic: lzma-custom-build-file
Relative: java-all-tests-bin-deploy-jar
Signed-off-by: andrew andrew@anyscale.com

@andrew-anyscale
Copy link
Contributor Author

andrew-anyscale commented Mar 11, 2026

Reviews in this chain:
#61669 [Build] Replace rules_boost lzma patches with a custom org_lzma_lzma build file
 └#61694 [Build] Patch protobuf for Bazel 7 exec_tools removal
  └#61695 [Build] Upgrade rules_apple/apple_support and set macOS toolchain flags for Bazel 7
   └#61601 [Build] Upgrade Bazel from 6.5.0 to 7.5.0
    └#61725 [Build] Upgrade protobuf 28.2, absl 20240722.0, grpc 1.68.2, boringssl

@andrew-anyscale
Copy link
Contributor Author

andrew-anyscale commented Mar 11, 2026

# head base diff date summary
0 a6fec9ca d63555fd diff Mar 11 16:07 PM 2 files changed, 208 insertions(+)
1 d2029f77 636fcd7b diff Mar 12 11:49 AM 1 file changed, 14 insertions(+), 7 deletions(-)
2 f63c28fd 076937ea diff Mar 12 12:31 PM 0 files changed
3 c8e84390 076937ea diff Mar 12 12:49 PM 1 file changed, 7 insertions(+), 40 deletions(-)
4 4b62149c df80fe60 rebase Mar 12 20:31 PM 0 files changed
5 be1c72d1 23649dd4 rebase Mar 13 11:13 AM 0 files changed
6 5c8b5467 23649dd4 diff Mar 13 14:43 PM 1 file changed, 6 insertions(+), 56 deletions(-)
7 83fffee6 23649dd4 diff Mar 13 17:26 PM 2 files changed, 28 insertions(+), 43 deletions(-)
8 f3190a02 ae7ad50c diff Mar 13 17:27 PM 0 files changed

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the patching of rules_boost's lzma build file with a custom build file for org_lzma_lzma. This is a solid approach to improve build reliability and cacheability, especially on Windows. The changes are well-explained in the pull request description and in the code comments. I have one suggestion for the new BUILD file to improve its maintainability by avoiding a hardcoded repository name.

@andrew-anyscale andrew-anyscale added the go add ONLY when ready to merge, run all tests label Mar 11, 2026
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/lzma-custom-build-file branch from a6fec9c to d2029f7 Compare March 12, 2026 18:49
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/java-all-tests-bin-deploy-jar branch from d63555f to 636fcd7 Compare March 12, 2026 18:49
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/lzma-custom-build-file branch from d2029f7 to f63c28f Compare March 12, 2026 19:31
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/java-all-tests-bin-deploy-jar branch from 636fcd7 to 076937e Compare March 12, 2026 19:31
@andrew-anyscale andrew-anyscale marked this pull request as ready for review March 12, 2026 19:31
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.


cc_library(
name = "lzma",
srcs = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these be using glob?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glob'd what I could. Some cannot be without pulling in files we do not want

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. which files are ones we do not want?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mix of "causes linker errors", others are just hygeine.

  1. src/xz/, src/xzdec/, src/lzmainfo/** + the *_tablegen.c files
    These all define main(). Including them in a cc_library produces a link error (multiple definition of
    main) for any binary that depends on :lzma.

  2. crc32_small.c and crc64_small.c
    Both define the same symbols as crc32_fast.c and crc64_fast.c (lzma_crc32, lzma_crc64). Including both is
    a link error — duplicate symbol at link time.

  3. The five tuklib_*.c CLI utility files (tuklib_exit.c, tuklib_mbstr_fw.c, etc.)
    These don't have main() and don't conflict with anything in the library. Worst case: dead code in the
    archive, and they may pull in CLI-only headers that don't compile cleanly on all platforms. Probably
    harmless in practice, but formally wrong.

@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/lzma-custom-build-file branch from be1c72d to 5c8b546 Compare March 13, 2026 21:43
"src/liblzma/rangecoder/price_table.c",
] + glob(["src/**/*.h"]),
hdrs = [
"src/liblzma/api/config.h",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these also seems glob-able?

"src/liblzma/lzma/lzma_encoder_optimum_normal.c",
"src/liblzma/lzma/lzma_encoder_presets.c",
"src/liblzma/rangecoder/price_table.c",
] + glob(["src/**/*.h"]),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe exclude api headers? those api header files are double included now?


cc_library(
name = "lzma",
srcs = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. which files are ones we do not want?

Base automatically changed from andrew/revup/master/java-all-tests-bin-deploy-jar to master March 13, 2026 21:51
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/lzma-custom-build-file branch from 52fb330 to 83fffee Compare March 14, 2026 00:27
…build file

Instead of patching rules_boost's BUILD.lzma at dependency-resolution time,
declare org_lzma_lzma before com_github_nelhage_rules_boost so that
boost_deps()'s maybe() skips it. A local build_file label
(//thirdparty/patches:org_lzma_lzma.BUILD.bazel) forces content-based cache
invalidation: changing the file triggers re-setup on all machines, including
Windows CI with a persistent output base where patching an external repo's
BUILD file in-place would not propagate to already-extracted directories.

The custom build file also fixes Windows MSVC D8004 by using -Ipath copts
(no space between -I and the path) and replaces the includes attribute with
strip_include_prefix for more reliable include path handling.

Topic: lzma-custom-build-file
Relative: java-all-tests-bin-deploy-jar
Signed-off-by: andrew <andrew@anyscale.com>
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/lzma-custom-build-file branch from 83fffee to f3190a0 Compare March 14, 2026 00:27
@aslonnie aslonnie merged commit af9ae6a into master Mar 14, 2026
6 checks passed
@aslonnie aslonnie deleted the andrew/revup/master/lzma-custom-build-file branch March 14, 2026 04:02
abrarsheikh pushed a commit that referenced this pull request Mar 17, 2026
…build file (#61669)

Instead of patching rules_boost's BUILD.lzma at dependency-resolution
time,
declare org_lzma_lzma before com_github_nelhage_rules_boost so that
boost_deps()'s maybe() skips it. A local build_file label
(//thirdparty/patches:org_lzma_lzma.BUILD.bazel) forces content-based
cache
invalidation: changing the file triggers re-setup on all machines,
including
Windows CI with a persistent output base where patching an external
repo's
BUILD file in-place would not propagate to already-extracted
directories.

The custom build file also fixes Windows MSVC D8004 by using -Ipath
copts
(no space between -I and the path) and replaces the includes attribute
with
strip_include_prefix for more reliable include path handling.

Signed-off-by: andrew <andrew@anyscale.com>
Signed-off-by: abrar <abrar@anyscale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build core Issues that should be addressed in Ray Core go add ONLY when ready to merge, run all tests windows

Development

Successfully merging this pull request may close these issues.

2 participants