[SPARK-48568][SQL] Improve Performance of CodeFormatter

[anton-erofeev]

What changes were proposed in this pull request?

Replace scala StringBuilder with java StringBuilder

Why are the changes needed?

https://issues.apache.org/jira/browse/SPARK-48568

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass github actions

Was this patch authored or co-authored using generative AI tooling?

No

[HyukjinKwon]

Can we have a micro benchmark?

[LuciferYang]

Can we have a micro benchmark?

+1 for micro benchmark. StringBuilder wraps j.l.StringBuilder,

final class StringBuilder(val underlying: java.lang.StringBuilder) extends AbstractSeq[Char]
  with ReusableBuilder[Char, String]
  with IndexedSeq[Char]
  with IndexedSeqOps[Char, IndexedSeq, StringBuilder]
  with IterableFactoryDefaults[Char, IndexedSeq]
  with java.lang.CharSequence
  with Serializable 

the functions should all directly operate on the underlying, so I'm not sure if this really brings a performance improvement. If it does, is this the only place worth change?

[dongjoon-hyun]

Happy New Year, @anton-erofeev . As the previous reviewers mentioned, this PR needs a proof of your claim in terms of the performance in the community.

[JoshRosen]

The original JIRA description at https://issues.apache.org/jira/browse/SPARK-48568 by @ahirreddy mentions some performance results with a similar change during profiling of certain large test suites:

• The reason for this change is that it substantially improves the performance of the CodeFormatter.
• From some basic profiling, in a ~100s suite, code formatting took ~2.7 seconds of CPU time. Post change it takes about 800ms.
• My hypothesis is that Java StringBuilder is much more JIT friendly. Scala's StringBuilder has many layers as it implements a significant portion of the Scala mutable collection API. It's also likely the case that the JVM has special JIT handling for Java StringBuilder

I vaguely recall having run into Java vs. Scala StringBuilder performance differences in some other context, but can't remember the details.

OTOH, I've also seen this to be a bit of a wash in less-scientific microbenchmarks of raw string appender performance.

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!