ARROW-5776: [Gandiva][Crossbow] Use commit id instead of fetch head.#4738
ARROW-5776: [Gandiva][Crossbow] Use commit id instead of fetch head.#4738praveenbingo wants to merge 8 commits intoapache:masterfrom
Conversation
|
@ursabot crossbow package -g gandiva |
|
AMD64 Conda Crossbow (#27835) builder has been succeeded. Revision: b6556ffdc678811a367bdd72cb50da7a0b1fce41 Submitted crossbow builds: ursa-labs/crossbow @ ursabot-96
|
|
@kszucs The ursabot failure looks like a random job termination. Please take a look if this looks ok to you. |
|
@ursabot crossbow package -g gandiva |
|
AMD64 Conda Crossbow (#27997) builder has been succeeded. Revision: f91ecb42fe9b718e5607d0266a86fe5bd143cfa4 Submitted crossbow builds: ursa-labs/crossbow @ ursabot-97
|
|
@praveenbingo in order to crossbow work properly with ursabot We need to use FETCH_HEAD. Ursabot builds the magic refs provided by github: Could You give me a little more context why it is not working for you? I'm sure We can find out a solution for both use cases. Please don't merge yet. |
|
@kszucs We need all of the builds - Gandiva Mac, Gandiva Linux and the internal jar packager to work off the same versions in the tree to avoid inconsistencies. So we need this commit id in the cross bow file as an anchor to ensure that the three build jobs work using the same code. Please let me know if this helps. |
|
It seems like this synchronization issue is something that the script that runs the multiple Crossbow builds should take care of. All of the branches that you push to your Crossbow queue should be based off the same commit id |
|
@wesm Yes I agree. And it was achieved earlier using the commit id as part of the trigger to the crossbow branches. |
|
@kszucs Do you think we can push this in until we have an alternative? |
|
We should add a flag to crossbow whether to use fetch head or the commit sha. |
|
@kszucs Sounds good, i will unblock by releasing temporarily from a private fork. Will get to the requested change in a couple days. Thanks! |
kszucs
force-pushed
the
master
branch
2 times, most recently
from
ed180da to
85fe336
Compare
…head - It is important to use the same commit to build the various gandiva artifacts on OsX/Linux etc, so that they are consistent. - We use the commit id to pin all of the builds to the same commit on the tree.
|
@ursabot crossbow package -g gandiva |
|
@kszucs Can you please see if the current change is ok? |
|
AMD64 Conda Crossbow (#43561) builder has been succeeded. Revision: 94c86fa Submitted crossbow builds: ursa-labs/crossbow @ ursabot-130
|
|
sample build where commit id is used https://travis-ci.org/praveenbingo/arrow-build/builds/563952605 |
fix style error
|
@kszucs can you please review this change ? |
Codecov Report
@@ Coverage Diff @@
## master #4738 +/- ##
==========================================
+ Coverage 88.53% 89.11% +0.57%
==========================================
Files 912 721 -191
Lines 116620 101649 -14971
Branches 1418 0 -1418
==========================================
- Hits 103255 90582 -12673
+ Misses 13003 11067 -1936
+ Partials 362 0 -362
Continue to review full report at Codecov.
|
|
@pravindra where is |
|
@kszucs it is set in Travis env variables |
|
@kszucs this travis job picked up the env variable (CROSSBOW_USE_COMMIT_ID) and ran successfully. https://travis-ci.org/praveenbingo/arrow-build/jobs/563952606 Please let's know if you want some change in the fix. Our (dremio) repo has been out-of-sync with arrow master for a while, and is blocked on this change. |
|
@pravindra I'm fine with this change, mostly because it only affects the gandiva build. In the future we should factor out this change to the level of the submit command. |
|
thanks @kszucs |
6 participants
on OsX/Linux etc, so that they are consistent.