[SPARK-3854] Scala style: require spaces before {

[sarutak]

This PR is a solution proposal of SPARK-3854.

Following is quoted from SPARK-3854:

We should require spaces before opening curly braces. This isn't in the style guide, but it probably should be:

// Correct:
if (true) {
  println("Wow!")
}

// Incorrect:
if (true){
   println("Wow!")
}

See #1658 (diff) for an example "in the wild."
git grep "){" shows only a few occurrences of this style.

[AmplabJenkins]

Can one of the admins verify this patch?

[JoshRosen]

Jenkins, add to whitelist. This is ok to test.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21629/Test FAILed.

[JoshRosen]

Jenkins, retest this please.

[SparkQA]

QA tests have started for PR 2761 at commit 86c63e0.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2761 at commit 86c63e0.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class SparkSpaceBeforeLeftBraceChecker extends ScalariformChecker
  • class SparkRunnerSettings(error: String => Unit) extends Settings(error)
  • trait ActorHelper extends Logging

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21632/Test FAILed.

[sarutak]

Oh, I didn't run scalastyle for yarn-alpha.

[SparkQA]

QA tests have started for PR 2761 at commit 86c63e0.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2761 at commit 86c63e0.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class SparkSpaceBeforeLeftBraceChecker extends ScalariformChecker
  • class SparkRunnerSettings(error: String => Unit) extends Settings(error)
  • trait ActorHelper extends Logging

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21633/Test FAILed.

[SparkQA]

QA tests have started for PR 2761 at commit 64b2c46.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2761 at commit 64b2c46.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class SparkSpaceBeforeLeftBraceChecker extends ScalariformChecker
  • class SparkRunnerSettings(error: String => Unit) extends Settings(error)
  • trait ActorHelper extends Logging

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21634/Test FAILed.

[SparkQA]

QA tests have started for PR 2761 at commit d80d71a.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2761 at commit d80d71a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class SparkSpaceBeforeLeftBraceChecker extends ScalariformChecker
  • class SparkRunnerSettings(error: String => Unit) extends Settings(error)
  • trait ActorHelper extends Logging

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21636/Test PASSed.

[srowen]

I quite like standardizing style, but doesn't this have the same problem mentioned before, that it's going to break a lot of potential merge commits? If it's bite-the-bullet time, there are other micro changes that may actually have a little positive impact on execution that might be good to get in too.

[JoshRosen]

It would be neat if there was some way to restrict the style checker to only check new/changed lines introduced by a PR. This could be hard to integrate with local development workflows, though: I might be developing some code locally and periodically running scalastyle before opening a pull request, so we'd need to make sure that we don't emit tons of warnings from existing code.

Maybe one approach would be to find all of the style warnings, then check whether the commits that introduced the lines that triggered the warnings are present in either origin/master or origin/branch-1-1.

[nchammas]

I quite like standardizing style, but doesn't this have the same problem mentioned before, that it's going to break a lot of potential merge commits?

When I did this for Python, I fixed all outstanding style problems as part of the same PR that introduced that check. It forced some people to rebase their open PRs, but it was a once-and-done thing.

Are we opposed to doing that here? Trying to ease this check in by enforcing it only on new code is a good idea, but why not just get the style cleanup over with in this PR? It looks like @sarutak has done just that. Some people will have to rebase once and this style problem is done with.

[sarutak]

I also think, it's difficult to apply new style checker only to new codes. I cleaned up codes in origin/master for the style checker suggested in this PR. So, if this PR is merged, then we can enforce the new style to developers and all developers have to do is to check the style of the code changed by them.

[SparkQA]

QA tests have started for PR 2761 at commit c5f2b30.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2761 at commit c5f2b30.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class SparkSpaceBeforeLeftBraceChecker extends ScalariformChecker
  • class StreamingContext(object):
  • class DStream(object):
  • class TransformedDStream(DStream):
  • class TransformFunction(object):
  • class TransformFunctionSerializer(object):
  • class SparkRunnerSettings(error: String => Unit) extends Settings(error)
  • trait ActorHelper extends Logging

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21670/
Test PASSed.

[JoshRosen]

Hi @sarutak,

We'd like to avoid making large refactorings for style, since these changes tend to create merge-conflicts when backporting to maintenance branches and make git blame significantly less useful. However, we'd be open to automatic style checks if they can be enforced only for new code (see https://issues.apache.org/jira/browse/SPARK-3849 for more details).

In the meantime, do you mind closing this pull request? Thanks!

[sarutak]

O.K. I close this PR for now.