[BEAM-8470] Exclude failed ValidatesRunner tests

[aromanenko-dev]

Exclude failed ValidatesRunner tests to prevent SparkStructuredStreaming Runner ValidatesRunner Tests job being always red.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---		---	---
Java
Python		---			---	---
XLang	---	---	---		---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[aromanenko-dev]

Run Spark StructuredStreaming ValidatesRunner

[aromanenko-dev]

@echauchot please, take a look

[echauchot]

IMHO, I would prefer to keep it red until they all pass because it is the essence of the tests. As they are not UTests, they don't break the build.

[aromanenko-dev]

If they are always red, it's difficult to say if we have a regression or not with every new PR coming. You need to compare test results all the time, it's not convenient (this is what I already did several times).

[aromanenko-dev]

IMHO, I would prefer to keep it red until they all pass because it is the essence of the tests.

We have disabled a bunch of other tests (that are failing) as well. Do we need to enable them in this case?

[echauchot]

We have disabled a bunch of other tests (that are failing) as well. Do we need to enable them in this case?

well, you have a point.

If they are always red, it's difficult to say if we have a regression or not with every new PR coming. You need to compare test results all the time, it's not convenient (this is what I already did several times).

you're right here also.

What is the correct way to avoid forgetting fixing the non-passing tests ? Any suggestion better than listing content of build.gradle exclusions ?

[aromanenko-dev]

@echauchot I'd prefer to keep ValidatesRunner Tests green and create a Jira(s) about failing tests since, in the end, there are the issues with transforms implementation (even if it's just corner cases). Wdyt?

[echauchot]

I agree.
please create the jiras.
LGTM, merging

[aromanenko-dev]

For the reference: https://issues.apache.org/jira/browse/BEAM-8839