[BEAM-6760] website: test external links in postcommit, not precommit

[ibzib]

Following discussion on the mailing list (and repeated failures of :testWebsite on my PRs), I think it might be wise to disable testing external links for that task by default. If one wishes, external links can optionally be tested with the following command:

./gradlew :beam-website:testWebsite -PdisableExternal=false

Update: I added the full website test as a postcommit. Now precommit does not check external links, while the postcommit does. One can also invoke the full test on a PR by commenting Run Full Website Test.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---	---	---	---	---
Java
Python		---			---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[apilloud]

LGTM!

There was a request on the mailing list to add a website postcommit with this check enabled. The configurations for our Jenkins jobs live here: https://github.com/apache/beam/tree/master/.test-infra/jenkins

[ibzib]

add a website postcommit with this check enabled

@apilloud Seems like there are a number of ways we could do this. We already have a postcommit for publishing the website. Would it make sense to just add the test as a step in that job?

beam/.test-infra/jenkins/job_PostCommit_Website_Publish.groovy

Line 23 in b987283

// This job builds and publishes the website into the asf-site branch of the beam repo.

[apilloud]

The website publish is its own job because we don't want to gate that on flaky tests. I have no strong opinions on adding it to that job vs a new one, but we do need to make sure the website publish doesn't break when external sites are down.

[ibzib]

Okay, I'll just make it its own postcommit then.

On Tue, Apr 16, 2019 at 11:58 AM Andrew Pilloud ***@***.***> wrote: The website publish is its own job because we don't want to gate that on flaky tests. I have no strong opinions on adding it to that job vs a new one, but we do need to make sure the website publish doesn't break when external sites are down. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8318 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APS_NtdNFeG-oO-lPGI3HuPcieU2OLlyks5vhh07gaJpZM4cxIS5> .

-- Kyle Weaver | Software Engineer | github.com/ibzib | kcweaver@google.com | +16502035555

[kennknowles]

run seed job

[aaltay]

Can we merge this?

[apilloud]

We should test it first, looks like the Run Full Website Test never got run.

[apilloud]

run seed job

[apilloud]

run seed job

[apilloud]

Run Full Website Test

[apilloud]

LGTM, passes tests.