[SPARK-37786][SQL] StreamingQueryListener support use SQLConf.get to get corresponding SessionState's SQLConf

[AngersZhuuuu]

What changes were proposed in this pull request?

Current StreamingQueryListener only support add constructor parameter of SparkConf, but if we start a SparkContext first, many SQL conf won't be applied to SparkContext's SparkConf.
Also it's a SQL component, related configuration we defined it as SQLConf. But it's hard to use SQLConf in this plugin listener.

This patch, we use pass SQLConf to StreamingQueryManager and use SQLConf.withExistingConf to wrap the passed SQLConf. Then we can use SQLConf.get to get the same SQLConf of SessionState when initializing StreamingQueryListener

Why are the changes needed?

Pass corresponding SessionState's SQLConf to StreamingQueryListener. Then user can use SQLConf.get to use the SQLConf.

Does this PR introduce any user-facing change?

User can use StreamingQueryListener with corresponding SessionState's SQLConf. For example

private class SQLConfStreamingQueryListener extends StreamingQueryListener {
  val sqlConf = SQLConf.get
  override def onSuccess(funcName: String, qe: QueryExecution, durationNs: Long): Unit = {
  }
  override def onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit = {
  }
}

How was this patch tested?

Added UT

[AngersZhuuuu]

GA passed, can merge this.

[HyukjinKwon]

Merged to master.

[AngersZhuuuu]

@HyukjinKwon Should we add a migration guide to mention this in the migration guide? If need I will raise a follow up for this two prs.

[HyukjinKwon]

what do we need for migration?

[AngersZhuuuu]

what do we need for migration?

Right, it's not a migration. But we should make user know this change. Maybe we can mention this in their configuration doc? Such as

spark.sql.queryExecutionListeners

[AngersZhuuuu]

what do we need for migration?

See #35122