Skip to content
Permalink
Browse files

[SPARK-25685][BUILD] Allow running tests in Jenkins in enterprise Git…

… repository

## What changes were proposed in this pull request?

Many companies have their own enterprise GitHub to manage Spark code. To build and test in those repositories with Jenkins need to modify this script.
So I suggest to add some environment variables to allow regression testing in enterprise Jenkins instead of default Spark repository in GitHub.

## How was this patch tested?

Manually test.

Closes #22678 from LantaoJin/SPARK-25685.

Lead-authored-by: lajin <lajin@ebay.com>
Co-authored-by: LantaoJin <jinlantao@gmail.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
  • Loading branch information...
LantaoJin authored and srowen committed Oct 12, 2018
1 parent d47a25f commit 541d7e1e4ba837f69d6f545b669185df91e8f406
Showing with 32 additions and 2 deletions.
  1. +4 −2 dev/run-tests-jenkins.py
  2. +28 −0 docs/building-spark.md
@@ -39,7 +39,8 @@ def print_err(msg):
def post_message_to_github(msg, ghprb_pull_id):
print("Attempting to post to Github...")

url = "https://api.github.com/repos/apache/spark/issues/" + ghprb_pull_id + "/comments"
api_url = os.getenv("GITHUB_API_BASE", "https://api.github.com/repos/apache/spark")
url = api_url + "/issues/" + ghprb_pull_id + "/comments"
github_oauth_key = os.environ["GITHUB_OAUTH_KEY"]

posted_message = json.dumps({"body": msg})
@@ -176,7 +177,8 @@ def main():
build_display_name = os.environ["BUILD_DISPLAY_NAME"]
build_url = os.environ["BUILD_URL"]

commit_url = "https://github.com/apache/spark/commit/" + ghprb_actual_commit
project_url = os.getenv("SPARK_PROJECT_URL", "https://github.com/apache/spark")
commit_url = project_url + "/commit/" + ghprb_actual_commit

# GitHub doesn't auto-link short hashes when submitted via the API, unfortunately. :(
short_commit_hash = ghprb_actual_commit[0:7]
@@ -260,3 +260,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6):
./build/sbt -Dscala.version=2.12.6

Otherwise, the sbt-pom-reader plugin will use the `scala.version` specified in the spark-parent pom.

## Running Jenkins tests with Github Enterprise

To run tests with Jenkins:

./dev/run-tests-jenkins

If use an individual repository or a repository on GitHub Enterprise, export below environment variables before running above command.

### Related environment variables

<table class="table">
<tr><th>Variable Name</th><th>Default</th><th>Meaning</th></tr>
<tr>
<td><code>SPARK_PROJECT_URL</code></td>
<td>https://github.com/apache/spark</td>
<td>
The Spark project URL of GitHub Enterprise.
</td>
</tr>
<tr>
<td><code>GITHUB_API_BASE</code></td>
<td>https://api.github.com/repos/apache/spark</td>
<td>
The Spark project API server URL of GitHub Enterprise.
</td>
</tr>
</table>

0 comments on commit 541d7e1

Please sign in to comment.
You can’t perform that action at this time.