[SPARK-35427][SQL][TESTS] Check the EXCEPTION rebase mode for Avro/Parquet

[MaxGekk]

What changes were proposed in this pull request?

Add tests to check the EXCEPTION rebase mode explicitly in the datasources:

• Parquet: DATE type and TIMESTAMP: INT96, TIMESTAMP_MICROS, TIMESTAMP_MILLIS
• Avro: DATE type and TIMESTAMP: timestamp-millis and timestamp-micros.

Why are the changes needed?

1. To improve test coverage
2. The EXCEPTION rebase mode should be checked independently from the default settings.

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running the affected test suites:

$ build/sbt "test:testOnly *AvroV2Suite"
$ build/sbt "test:testOnly *ParquetRebaseDatetimeV1Suite"

[MaxGekk]

@cloud-fan Could you review this PR, please.

[cloud-fan]

do we need to clean up the existing tests to remove code that are being tested here?

[MaxGekk]

Let me find out the places where we check the default mode (EXCEPTION).

[MaxGekk]

I think we can avoid checking the default EXCEPTION mode at least in the places:

1. spark/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRebaseDatetimeSuite.scala

   Lines 142 to 146 in 0494dc9

   	// By default we should fail to write ancient datetime values.
   	if (tsOutputType != "INT96") {
   	val e = intercept[SparkException](df.write.parquet(path3_0))
   	assert(e.getCause.getCause.getCause.isInstanceOf[SparkUpgradeException])
   	}

2. spark/external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala

   Lines 1835 to 1837 in 2bd3254

   	// By default we should fail to write ancient datetime values.
   	val e = intercept[SparkException](df.write.format("avro").save(path3_0))
   	assert(e.getCause.getCause.getCause.isInstanceOf[SparkUpgradeException])

@cloud-fan Let me know if you think that there are other places.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43158/

[SparkQA]

Test build #138638 has finished for PR 32574 at commit 172f83c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43279/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43279/

[SparkQA]

Test build #138757 has finished for PR 32574 at commit d6e38bd.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[MaxGekk]

@cloud-fan Any objections to the changes?

[MaxGekk]

@gengliangwang @HyukjinKwon Could you take a look at the PR, please.

[cloud-fan]

thanks, merging to master!