HBASE-28054 Add spotless in hbase-connectors pre commit check #122

NihalJain · 2023-08-30T18:33:38Z

No description provided.

NihalJain · 2023-08-30T18:47:35Z

Ran yetus with --plugins=maven,spotless locally and ensured it failed with a bad patch

| Vote |    Subsystem |  Runtime   | Comment
============================================================================
+---------------------------------------------------------------------------
|      |              |            | Prechecks
+---------------------------------------------------------------------------
+---------------------------------------------------------------------------
|      |              |            | HBASE-28054 Compile Tests
+---------------------------------------------------------------------------
|   0  |      mvndep  |   0m 10s   | Maven dependency ordering for branch
|  +1  |    spotless  |   0m 19s   | branch has no errors when running
|      |              |            | spotless:check.
+---------------------------------------------------------------------------
|      |              |            | Patch Compile Tests
+---------------------------------------------------------------------------
|   0  |      mvndep  |   0m 09s   | Maven dependency ordering for patch
|  -1  |    spotless  |   0m 19s   | patch has 26 errors when running
|      |              |            | spotless:check, run spotless:apply to fix.
+---------------------------------------------------------------------------
|      |              |            | Other Tests
+---------------------------------------------------------------------------
|      |              |   1m 25s   |

Output of spotless run file:

➜  hbase-connectors git:(HBASE-28054) ✗ cat /private/tmp/yetus-21136.6492/patch-spotless.txt
Thu Aug 31 00:09:40 IST 2023
cd /Users/nihaljain/code/os/hbase-connectors
mvn --offline --batch-mode spotless:check
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache HBase Connectors                                            [pom]
[INFO] Apache HBase - Kafka                                               [pom]
[INFO] Apache HBase - Model Objects for Kafka Proxy                       [jar]
[INFO] Apache HBase - Kafka Proxy                                         [jar]
[INFO] Apache HBase - Spark                                               [pom]
[INFO] Apache HBase - Spark Protocol                                      [jar]
[INFO] Apache HBase - Spark Protocol (Shaded)                             [jar]
[INFO] Apache HBase - Spark Connector                                     [jar]
[INFO] Apache HBase - Spark Integration Tests                             [jar]
[INFO] Apache HBase Connectors - Assembly                                 [pom]
[INFO]
[INFO] ------------< org.apache.hbase.connectors:hbase-connectors >------------
[INFO] Building Apache HBase Connectors 1.0.1-SNAPSHOT                   [1/10]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ hbase-connectors ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom2175271538862894342.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] -----------------< org.apache.hbase.connectors:kafka >------------------
[INFO] Building Apache HBase - Kafka 1.0.1-SNAPSHOT                      [2/10]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ kafka ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom4621269930024559741.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] --------< org.apache.hbase.connectors.kafka:hbase-kafka-model >---------
[INFO] Building Apache HBase - Model Objects for Kafka Proxy 1.0.1-SNAPSHOT [3/10]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ hbase-kafka-model ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom503402107018101105.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] --------< org.apache.hbase.connectors.kafka:hbase-kafka-proxy >---------
[INFO] Building Apache HBase - Kafka Proxy 1.0.1-SNAPSHOT                [4/10]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ hbase-kafka-proxy ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom677669518756705814.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] -----------------< org.apache.hbase.connectors:spark >------------------
[INFO] Building Apache HBase - Spark 1.0.1-SNAPSHOT                      [5/10]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ spark ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom7109549005648937715.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] -------< org.apache.hbase.connectors.spark:hbase-spark-protocol >-------
[INFO] Building Apache HBase - Spark Protocol 1.0.1-SNAPSHOT             [6/10]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ hbase-spark-protocol ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom6296127495647823376.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] ---< org.apache.hbase.connectors.spark:hbase-spark-protocol-shaded >----
[INFO] Building Apache HBase - Spark Protocol (Shaded) 1.0.1-SNAPSHOT    [7/10]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ hbase-spark-protocol-shaded ---
[INFO] Sorting file /var/folders/xb/j3j5fp5153g06gh5sdkmhf7c0000gq/T/pom2799991780908287063.xml
[INFO] Pom file is already sorted, exiting
[INFO]
[INFO] -----------< org.apache.hbase.connectors.spark:hbase-spark >------------
[INFO] Building Apache HBase - Spark Connector 1.0.1-SNAPSHOT            [8/10]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- spotless-maven-plugin:2.27.2:check (default-cli) @ hbase-spark ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache HBase Connectors 1.0.1-SNAPSHOT:
[INFO]
[INFO] Apache HBase Connectors ............................ SUCCESS [  2.091 s]
[INFO] Apache HBase - Kafka ............................... SUCCESS [  0.116 s]
[INFO] Apache HBase - Model Objects for Kafka Proxy ....... SUCCESS [  0.118 s]
[INFO] Apache HBase - Kafka Proxy ......................... SUCCESS [  2.238 s]
[INFO] Apache HBase - Spark ............................... SUCCESS [  0.178 s]
[INFO] Apache HBase - Spark Protocol ...................... SUCCESS [  0.087 s]
[INFO] Apache HBase - Spark Protocol (Shaded) ............. SUCCESS [  0.064 s]
[INFO] Apache HBase - Spark Connector ..................... FAILURE [  9.008 s]
[INFO] Apache HBase - Spark Integration Tests ............. SKIPPED
[INFO] Apache HBase Connectors - Assembly ................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14.427 s
[INFO] Finished at: 2023-08-31T00:09:56+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.27.2:check (default-cli) on project hbase-spark: The following files had format violations:
[ERROR]     src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala
[ERROR]         @@ -30,10 +30,8 @@
[ERROR]
[ERROR]          @InterfaceAudience.Private
[ERROR]          object·UserCustomizedSampleException·{
[ERROR]         -··def
[ERROR]         -··message(message:·String,·cause:·Throwable)·=
[ERROR]         -····if
[ERROR]         -····(message·!=·null)·message
[ERROR]         +··def·message(message:·String,·cause:·Throwable)·=
[ERROR]         +····if·(message·!=·null)·message
[ERROR]          ····else·if·(cause·!=·null)·cause.toString()
[ERROR]          ····else·null
[ERROR]          }
[ERROR] Run 'mvn spotless:apply' to fix these violations.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hbase-spark

Content of bad patch:

➜  hbase-connectors git:(HBASE-28054) ✗ cat 0001-test-spotless.patch
From 28ee398fe71b154875acab508ce64bf6c8440585 Mon Sep 17 00:00:00 2001
From: "Jain, Nihal" <nihaljain.cs@gmail.com>
Date: Wed, 30 Aug 2023 22:25:34 +0530
Subject: [PATCH] test spotless

---
 .../hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java    | 4 +++-
 .../hadoop/hbase/spark/example/datasources/DataType.scala   | 6 ++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/spark/hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java b/spark/hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java
index 0766915..9299926 100644
--- a/spark/hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java
+++ b/spark/hbase-spark-it/src/test/java/org/apache/hadoop/hbase/spark/IntegrationTestSparkBulkLoad.java
@@ -119,7 +119,9 @@ public class IntegrationTestSparkBulkLoad extends IntegrationTestBase {
    */
   public void runLoad() throws Exception {
     setupTable();
-    int numImportRounds = getConf().getInt(BULKLOAD_IMPORT_ROUNDS, DEFAULT_BULKLOAD_IMPORT_ROUNDS);
+    int numImportRounds
+            =
+            getConf().getInt(BULKLOAD_IMPORT_ROUNDS, DEFAULT_BULKLOAD_IMPORT_ROUNDS);
     LOG.info("Running load with numIterations:" + numImportRounds);
     for (int i = 0; i < numImportRounds; i++) {
       runLinkedListSparkJob(i);
diff --git a/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala b/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala
index d314dc8..be2bf33 100644
--- a/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala
+++ b/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala
@@ -30,8 +30,10 @@ class UserCustomizedSampleException(message: String = null, cause: Throwable = n

 @InterfaceAudience.Private
 object UserCustomizedSampleException {
-  def message(message: String, cause: Throwable) =
-    if (message != null) message
+  def
+  message(message: String, cause: Throwable) =
+    if
+    (message != null) message
     else if (cause != null) cause.toString()
     else null
 }
--
2.33.0

Running with --empty-patch

+1 overall



| Vote |    Subsystem |  Runtime   | Comment
============================================================================
+---------------------------------------------------------------------------
|      |              |            | Prechecks
+---------------------------------------------------------------------------
+---------------------------------------------------------------------------
|      |              |            | Compile Tests
+---------------------------------------------------------------------------
|   0  |      mvndep  |   0m 08s   | Maven dependency ordering
|  +1  |    spotless  |   0m 17s   | patch has no errors when running
|      |              |            | spotless:check.
+---------------------------------------------------------------------------
|      |              |            | Other Tests
+---------------------------------------------------------------------------
|      |              |   0m 26s   |

Apache-HBase · 2023-08-30T18:48:21Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-122/1/console in case of problems.

Apache-HBase · 2023-08-30T18:49:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 23s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 18s	Maven dependency ordering for branch
+1 💚	spotless	0m 15s	branch has no errors when running spotless:check.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 2s	Maven dependency ordering for patch
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	spotless	0m 11s	patch has no errors when running spotless:check.
		_ Other Tests _
		1m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-122/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#122
Optional Tests	dupname spotless shellcheck shelldocs
uname	Linux 06924e36afd5 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 GNU/Linux
Build tool	hb_maven
Personality	dev-support/jenkins/hbase-personality.sh
git revision	master / `ee2b51c`
Max. process+thread count	34 (vs. ulimit of 12500)
modules	C: U:
Console output	https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-122/1/console
versions	git=2.20.1 shellcheck=0.5.0
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

NihalJain · 2023-08-30T18:52:04Z

Hi @Apache9, @Reidddddd Could you please review?

NihalJain · 2023-08-30T19:50:51Z

Rasied #123 to demo failure

HBASE-28054 Add spotless in hbase-connectors pre commit check

4b027fa

NihalJain requested review from Apache9 and Reidddddd August 30, 2023 18:50

Apache9 approved these changes Aug 31, 2023

View reviewed changes

Reidddddd approved these changes Aug 31, 2023

View reviewed changes

Reidddddd merged commit 30142cc into apache:master Aug 31, 2023
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28054 Add spotless in hbase-connectors pre commit check #122

HBASE-28054 Add spotless in hbase-connectors pre commit check #122

NihalJain commented Aug 30, 2023

NihalJain commented Aug 30, 2023 •

edited

Apache-HBase commented Aug 30, 2023

Apache-HBase commented Aug 30, 2023

NihalJain commented Aug 30, 2023

NihalJain commented Aug 30, 2023

HBASE-28054 Add spotless in hbase-connectors pre commit check #122

HBASE-28054 Add spotless in hbase-connectors pre commit check #122

Conversation

NihalJain commented Aug 30, 2023

NihalJain commented Aug 30, 2023 • edited

Apache-HBase commented Aug 30, 2023

Apache-HBase commented Aug 30, 2023

NihalJain commented Aug 30, 2023

NihalJain commented Aug 30, 2023

NihalJain commented Aug 30, 2023 •

edited