Skip to content

Conversation

@Yikun
Copy link
Member

@Yikun Yikun commented Apr 26, 2021

What changes were proposed in this pull request?

This PR corrects some exception type when the function input params are failed to validate due to TypeError.

Why are the changes needed?

As suggestion from Python exception doc [1]: "Raised when an operation or function is applied to an object of inappropriate type.", but there are many Value error are raised in some pyspark code, this patch fix them.

[1] https://docs.python.org/3/library/exceptions.html#TypeError

Note that: this patch only addresses the exsiting some wrong raise type for input validation, the input validation decorator/framework which mentioned in SPARK-35176, would be submited in a speparated patch.

Does this PR introduce any user-facing change?

Yes, code can raise the right TypeError instead of ValueError.

How was this patch tested?

Existing test case and UT

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Test build #137945 has finished for PR 32349 at commit ac211c5.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42467/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42467/

@Yikun Yikun changed the title [SPARK-35176][PYTHON] Standardize input validation error type [WIP][SPARK-35176][PYTHON] Standardize input validation error type Apr 26, 2021
@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Test build #137956 has finished for PR 32349 at commit 98f98ad.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@Yikun Yikun marked this pull request as draft April 26, 2021 11:43
@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42474/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Test build #137958 has finished for PR 32349 at commit 6197e7a.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • with self.assertRaisesRegex(TypeError, \"was <class 'int'>\"):
  • with self.assertRaisesRegex(TypeError, \"[<class 'int'>]\"):
  • with self.assertRaisesRegex(TypeError, \"(<class 'int'>, <class 'NoneType'>)\"):

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42478/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42478/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Test build #137962 has finished for PR 32349 at commit 8aff5db.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42483/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42483/

@AmplabJenkins
Copy link

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42483/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants