GH-33851: [C++] Update bundled boost version #33890
Conversation
|
|
|
@kou mentioned on the issue: "All Apache Arrow committers/PMC members can upload a file to https://apache.jfrog.io/artifactory/arrow/thirdparty/ from https://apache.jfrog.io/ui/repos/tree/General/arrow/thirdparty with their ASF account." |
|
There's also a curl command suggested here: #18890 (comment) |
|
The ui for artifactory is a bit slow but to upload one file it is fine :D (would need to login to get the api key anyway ^^) Once that is uploaded and changed you can test it locally with |
|
@github-actions crossbow submit nightly-* |
|
|
@github-actions crossbow submit --group nightly |
|
Revision: b562f75 Submitted crossbow builds: ursacomputing/crossbow @ actions-989dabe857
|
|
@github-actions crossbow submit r-binary-packages That job is missing from nightly for some reason ^^ |
|
Revision: b562f75 Submitted crossbow builds: ursacomputing/crossbow @ actions-c049af73be
|
|
When I run I get I can dig into this in a bit when I circle back to this PR but maybe one of you has an idea of what this could be/how to go about debugging it in the meantime? |
|
You can test this in your regular build env too without docker. That might make debug easier. You might want to |
paleolimbot
force-pushed
the
update-bundled-boost
branch
from
7eca941
to
01124ba
Compare
|
Hmm...Boost seems fixed now but I managed to make all the other downloads fail the integrity check? |
paleolimbot
force-pushed
the
update-bundled-boost
branch
from
01124ba
to
ddc5c78
Compare
|
Never mind...that error seems to be ubiquitous on Arrow builds right now. |
|
It's the internet and not you ;) #33939 |
|
Why "7.0.0"? Does this version number have a special meaning? |
|
I would guess that 7.0.0 was when we moved these bundles to artifactory and someone at the time thought it would be useful to associate them with an arrow version. But AFAICT it shouldn't matter, the dependency's version is in its filename, so boost_1.75.0 can live alongside boost_1.81.0. |
|
Is Boost the only dependency that gets downloaded in this way? (If it is, I'm happy to change the URL in this PR if somebody tells me where it is). |
No, but it's the only one we alter in this way to reduce the size. The rest are just mirrors of the official sources for redundancy, so that if one site/CDN goes down, we're less likely to get a failed build. I wouldn't just change the URL since we've shipped several versions of the library with it. You could just put the boost bundle in the thirdparty directory (i.e. up one level) and update the URL for it. Then whenever we bump the dependency versions for everything else, we can put them in that directory. (You could make an issue for that.) |
|
Ok, I tested with
It seems this is fixed, at least temporarily?
For now I think that may be even more confusing than 7.0.0? I'll re-run |
|
@github-actions crossbow submit --group nightly |
|
Revision: a934a48 Submitted crossbow builds: ursacomputing/crossbow @ actions-9f1739740a
|
It doesn't have a special meaning. It’s just for historical reason. We changed our thirdparty mirror to Artifactory in #12190. We can remove the tag name from URL in the PR but we keep only "7.0.0" in the PR. We can use different URL such as https://apache.jfrog.io/artifactory/arrow/thirdparty/boost/boost_X_Y_Z.tar.gz for new files. |
|
If anybody is passionate about the slightly confusing URL for bundled third-party libraries, I agree that it's sub-optimal; however, this PR is about updating Boost and we may need to pick it into the CRAN branch. Future me might be a good candidate to do this (after I've interacted with the C++ CMake build a little more), but I don't feel comfortable making that change until I understand it a bit better. Of that huge list of nightlies, these are the errors I can't explain otherwise: test-alpine-linux-cpp: verify-rc-source-cpp-macos-conda-amd64, verify-rc-source-integration-macos-amd64, verify-rc-source-integration-macos-conda-amd64, verify-rc-source-python-macos-conda-amd64, verify-rc-source-ruby-macos-amd64, : |
|
Is there any insight on whether or not these failures are related to the Boost update? Or is it early enough in the release cycle that we can merge this? |
|
Well we can only change the URL for future versions. We have to keep the 7.0.0 folder around so previously released versions also work (or we decide to drop them at some point?) Also agree that that is a separate issue The hash join fail feels familiar but I have nothing specific. |
|
test-alpine-linux-cpp doesn't use bundled Boost. So it's not related to this. |
|
Benchmark runs are scheduled for baseline = 771c37a and contender = d1e5cb5. d1e5cb5 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
This PR updates the bundled version of Boost, as it was suggested by a maintainer of CRAN (R packaging) that the older version of boost might be responsible for an Arrow build failure on Fedora/clang (apache#33819). Closes apache#33851. The resulting tarball also has to be uploaded to https://apache.jfrog.io/ui/native/arrow/thirdparty/7.0.0/ (right?) to kick in and almost certainly needs to be tested through a round of CI. I'm not sure how to do either of those things but perhaps @ assignUser does? * Closes: apache#33851 Authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
This PR updates the bundled version of Boost, as it was suggested by a maintainer of CRAN (R packaging) that the older version of boost might be responsible for an Arrow build failure on Fedora/clang (apache#33819). Closes apache#33851. The resulting tarball also has to be uploaded to https://apache.jfrog.io/ui/native/arrow/thirdparty/7.0.0/ (right?) to kick in and almost certainly needs to be tested through a round of CI. I'm not sure how to do either of those things but perhaps @ assignUser does? * Closes: apache#33851 Authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
This PR updates the bundled version of Boost, as it was suggested by a maintainer of CRAN (R packaging) that the older version of boost might be responsible for an Arrow build failure on Fedora/clang (apache#33819). Closes apache#33851. The resulting tarball also has to be uploaded to https://apache.jfrog.io/ui/native/arrow/thirdparty/7.0.0/ (right?) to kick in and almost certainly needs to be tested through a round of CI. I'm not sure how to do either of those things but perhaps @ assignUser does? * Closes: apache#33851 Authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
This PR updates the bundled version of Boost, as it was suggested by a maintainer of CRAN (R packaging) that the older version of boost might be responsible for an Arrow build failure on Fedora/clang (apache#33819). Closes apache#33851. The resulting tarball also has to be uploaded to https://apache.jfrog.io/ui/native/arrow/thirdparty/7.0.0/ (right?) to kick in and almost certainly needs to be tested through a round of CI. I'm not sure how to do either of those things but perhaps @ assignUser does? * Closes: apache#33851 Authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
This PR updates the bundled version of Boost, as it was suggested by a maintainer of CRAN (R packaging) that the older version of boost might be responsible for an Arrow build failure on Fedora/clang (#33819).
Closes #33851.
The resulting tarball also has to be uploaded to https://apache.jfrog.io/ui/native/arrow/thirdparty/7.0.0/ (right?) to kick in and almost certainly needs to be tested through a round of CI. I'm not sure how to do either of those things but perhaps @assignUser does?