Skip to content

ARROW-927: C++/Python: Add manylinux1 builds to Travis matrix#624

Closed
xhochy wants to merge 5 commits intoapache:masterfrom
xhochy:ARROW-927
Closed

ARROW-927: C++/Python: Add manylinux1 builds to Travis matrix#624
xhochy wants to merge 5 commits intoapache:masterfrom
xhochy:ARROW-927

Conversation

@xhochy
Copy link
Copy Markdown
Member

@xhochy xhochy commented May 1, 2017

Also pre-builds flatbuffers and gtest so that they are only build once.

Change-Id: Ibf37e6e7522d06346bb18db0059dfc5ecffbf33f
@wesm
Copy link
Copy Markdown
Member

wesm commented May 1, 2017

The manylinux1 build took 32 minutes, almost 20 minutes more than the next slowest build -- I'm concerned this is going to affect developer productivity. What do you think about setting up a separate repo (and give committers push/admin on it) and periodically updating the build to run in CircleCI? I know that the point of this is to not break the packaging build, but this would help others be proactive about checking that things still work properly.

@xhochy
Copy link
Copy Markdown
Member Author

xhochy commented May 1, 2017

The base problem here that we spent all time building dependencies like boost, openssl and flatbuffers. I'm going to set up a separate build that will package the thirdparties into a docker image that we then can pull in Travis.

xhochy added 4 commits May 1, 2017 19:52
Change-Id: Ief6d9aaa9f682f9df868376529f923c17a2de1a1
Change-Id: If150a7bba078cbd1f07710d2d6bef5c512b4a063
Change-Id: I053faf9bf4be05b4bdce87d94d101c153bd90ee1
Change-Id: I07d55df7b6b08dc5eb6f5ac48f1e478145678757
@wesm
Copy link
Copy Markdown
Member

wesm commented May 1, 2017

The build time is better, but still not great. The OS X build varies in time from 13-25 minutes based on the time of day; one benefit of the build is that it's Linux only so generally starts up right away.

build_times

I think the way to make things faster would be to also include the rest of parquet-cpp's thirdparty toolchain (especially Thrift) in the Docker image. What do you think? Doesn't have to get done in this patch

@xhochy
Copy link
Copy Markdown
Member Author

xhochy commented May 2, 2017

I'll include Thrift & co in the docker imager but first I need to adjust it so that the size of the image itself is a lot smaller, currently pulling the image takes a long time.

Copy link
Copy Markdown
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@asfgit asfgit closed this in 02a121f May 2, 2017
jeffknupp pushed a commit to jeffknupp/arrow that referenced this pull request Jun 3, 2017
Also pre-builds flatbuffers and gtest so that they are only build once.

Author: Uwe L. Korn <uwelk@xhochy.com>

Closes apache#624 from xhochy/ARROW-927 and squashes the following commits:

c901b7a [Uwe L. Korn] Separate scripts folder
0dd4e08 [Uwe L. Korn] Add scripts folder
e705813 [Uwe L. Korn] Move boost and openssl to scripts
1b44878 [Uwe L. Korn] Add base image for thirdparties
f4ff321 [Uwe L. Korn] ARROW-927: C++/Python: Add manylinux1 builds to Travis matrix
pribor pushed a commit to GlobalWebIndex/arrow that referenced this pull request Oct 24, 2025
## What's Changed

* Add a comment when we can remove this workaround.
* Remove a needless workaround for Windows.

Closes apache#623.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants