[Python] Instructions for setting up nightly builds on Linux

[asfimport]

cc @cpcloud

Reporter: Wes McKinney / @wesm
Assignee: Phillip Cloud / @cpcloud

PRs and other links:

• GitHub Pull Request #1489

_{Note: This issue was originally created as ARROW-1580. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
This would be great, but not urgent for 0.8.0

[asfimport]

Tom Augspurger / @TomAugspurger:
The short version is

 

1. We have a build machine running Ubuntu
2. Phil has a build tool https://github.com/cpcloud/scourge that's been setup on the build machine
3. That tool is triggered through an airflow scheduler. Airflow (and some PyData ASVs) are bootstrapped through https://github.com/tomaugspurger/asv-runner. The extra step for the arrow nightlies was to add a file in ~/airflow/dags so that the scheduler picked it up.

[asfimport]

Wes McKinney / @wesm:
Cool, it would be great to have reproducible instructions for running a conda build of pyarrow (using the scourge tool) based on either master or some other branch of the conda-forge recipes, that is not necessarily coupled to the build machine where we are current uploading nightlies to https://anaconda.org/twosigma/pyarrow. This would make it easier for others to set up their own nightly development / testing builds without being beholden to others

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 1489
#1489