Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError #38413

Closed

Conversation

LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Oct 27, 2022

What changes were proposed in this pull request?

This pr aims to refactor AnalysisTest#assertAnalysisErrorClass method by reusing the checkError method in SparkFunSuite.

On the other hand, the signature of AnalysisTest#assertAnalysisErrorClass method is changed from

protected def assertAnalysisErrorClass(
      inputPlan: LogicalPlan,
      expectedErrorClass: String,
      expectedMessageParameters: Map[String, String],
      caseSensitive: Boolean = true,
      line: Int = -1,
      pos: Int = -1): Unit

to

protected def assertAnalysisErrorClass(
      inputPlan: LogicalPlan,
      expectedErrorClass: String,
      expectedMessageParameters: Map[String, String],
      queryContext: Array[QueryContext] = Array.empty,
      caseSensitive: Boolean = true): Unit

Then when we need to use queryContext instead of line + pos for assertion

Why are the changes needed?

assertAnalysisErrorClass and checkError does the same work.

Does this PR introduce any user-facing change?

No, just for test

How was this patch tested?

  • Pass GitHub Actions

@github-actions github-actions bot added the SQL label Oct 27, 2022
@LuciferYang
Copy link
Contributor Author

cc @MaxGekk @HyukjinKwon @dongjoon-hyun Think again, does this refactor look more simple?

@MaxGekk
Copy link
Member

MaxGekk commented Oct 27, 2022

I wonder why do we need assertAnalysisErrorClass() at all. checkError does the same job. Seems like assertAnalysisErrorClass() checks additionally case sensitivity (can be done in test explicitly if it is really needed) + checking line + pos (we should check query context instead of that, I guess).

Let's consider to invoke checkError in assertAnalysisErrorClass() or remove it completely (invoke checkError() directly in tests as we do in other places).

@LuciferYang
Copy link
Contributor Author

assertAnalysisErrorClass

Sounds good, let me try. Set this to draft first and will ping you when it can be reviewed @MaxGekk

@LuciferYang LuciferYang marked this pull request as draft October 27, 2022 15:57
@LuciferYang LuciferYang changed the title [SPARK-40936][SQL][TESTS] Remove outer conditions to simplify AnalysisTest#assertAnalysisErrorClass method [WIP][SPARK-40936][SQL][TESTS] Remove outer conditions to simplify AnalysisTest#assertAnalysisErrorClass method Oct 27, 2022
caseSensitive = true,
line = -1,
pos = -1)
Array(ExpectedContext("y", 46, 46))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

val actualQueryContext = exception.getQueryContext()
assert(actualQueryContext.length === queryContext.length, "Invalid length of the query context")
actualQueryContext.zip(queryContext).foreach { case (actual, expected) =>
assert(actual.objectType() === expected.objectType(),
"Invalid objectType of a query context Actual:" + actual.toString)
assert(actual.objectName() === expected.objectName(),
"Invalid objectName of a query context. Actual:" + actual.toString)
assert(actual.startIndex() === expected.startIndex(),
"Invalid startIndex of a query context. Actual:" + actual.toString)
assert(actual.stopIndex() === expected.stopIndex(),
"Invalid stopIndex of a query context. Actual:" + actual.toString)
assert(actual.fragment() === expected.fragment(),
"Invalid fragment of a query context. Actual:" + actual.toString)
}

This change is due to the checkError method will perform a forced check when actualQueryContext is not empty. If we can to relax some check conditions, can add a precondition queryContext.nonEmpty for the queryContext check.

@LuciferYang LuciferYang changed the title [WIP][SPARK-40936][SQL][TESTS] Remove outer conditions to simplify AnalysisTest#assertAnalysisErrorClass method [SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError Oct 28, 2022
@LuciferYang LuciferYang changed the title [SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError [WIP][SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError Oct 28, 2022
@LuciferYang LuciferYang changed the title [WIP][SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError [SPARK-40936][SQL][TESTS] Remove outer conditions to simplify AnalysisTest#assertAnalysisErrorClass method Oct 28, 2022
@LuciferYang LuciferYang marked this pull request as ready for review October 28, 2022 04:11
@LuciferYang
Copy link
Contributor Author

@MaxGekk

Refactor assertAnalysisErrorClass method:

  • Reuse checkError in assertAnalysisErrorClass
  • Use queryContext instead of line + pos in assertAnalysisErrorClass method signature
  • Fixed related tests

Please review this if you have time, thanks ~

@LuciferYang LuciferYang changed the title [SPARK-40936][SQL][TESTS] Remove outer conditions to simplify AnalysisTest#assertAnalysisErrorClass method [SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError Oct 28, 2022
@LuciferYang LuciferYang changed the title [SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError [SPARK-40936][SQL][TESTS] Refactor AnalysisTest#assertAnalysisErrorClass by reusing the SparkFunSuite#checkError Oct 28, 2022
@MaxGekk
Copy link
Member

MaxGekk commented Oct 28, 2022

Waiting for Ci.

@LuciferYang
Copy link
Contributor Author

GA passed

@MaxGekk
Copy link
Member

MaxGekk commented Oct 28, 2022

+1, LGTM. Merging to master.
Thank you, @LuciferYang.

@MaxGekk MaxGekk closed this in 145de7d Oct 28, 2022
SandishKumarHN pushed a commit to SandishKumarHN/spark that referenced this pull request Dec 12, 2022
…lass` by reusing the `SparkFunSuite#checkError`

### What changes were proposed in this pull request?
This pr aims to refactor  `AnalysisTest#assertAnalysisErrorClass` method by reusing the `checkError` method in `SparkFunSuite`.

On the other hand, the signature of `AnalysisTest#assertAnalysisErrorClass` method is changed from

```
protected def assertAnalysisErrorClass(
      inputPlan: LogicalPlan,
      expectedErrorClass: String,
      expectedMessageParameters: Map[String, String],
      caseSensitive: Boolean = true,
      line: Int = -1,
      pos: Int = -1): Unit
```
to

```
protected def assertAnalysisErrorClass(
      inputPlan: LogicalPlan,
      expectedErrorClass: String,
      expectedMessageParameters: Map[String, String],
      queryContext: Array[QueryContext] = Array.empty,
      caseSensitive: Boolean = true): Unit
```

Then when we need to use `queryContext` instead of `line + pos` for assertion

### Why are the changes needed?
`assertAnalysisErrorClass` and `checkError` does the same work.

### Does this PR introduce _any_ user-facing change?
No,  just for test

### How was this patch tested?

- Pass GitHub Actions

Closes apache#38413 from LuciferYang/simplify-assertAnalysisErrorClass.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants