New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SW-2496] Fix the Flow link for Databricks Azure #2417
Conversation
@@ -32,7 +33,24 @@ object SparkSpecificUtils extends CrossSparkUtils { | |||
content: => Seq[Node], | |||
activeTab: SparkUITab, | |||
helpText: String): Seq[Node] = { | |||
UIUtils.headerSparkPage(request, "Sparkling Water", content, activeTab, helpText = Some(helpText)) | |||
val method = UIUtils.getClass.getMethods.find(m => m.getName == "headerSparkPage").get |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one is obviously fragile and can break if an overloaded method is added to spark.ui
But I guess that you can live with this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Otherwise LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, if the method doesn't exist the Spark UI shows an exception if Sparkling Water tab is clicked. For Apache Spark we know that there is such a method. This could if a user deploys SW to a obscure platform with a custom build of Spark which we haven't tested. IMHO, we can live with that.
* [SW-2496] Fix the Flow link for Databricks Azure * spotless apply (cherry picked from commit 5af8b3a)
There are two problems.
More details on:
https://docs.microsoft.com/en-us/azure/databricks/workspace/per-workspace-urls
UIUtils.headerSparkPage
method in Databricks Spark has more parameters than Apache Spark. I solved the problem via reflection call.Databricks:
request: javax.servlet.http.HttpServletRequest, title: String,content: => Seq[scala.xml.Node], activeTab: org.apache.spark.ui.SparkUITab, helpText: Option[String], showVisualization: Boolean, useDataTables: Boolean, supportScreenshot: Boolean
Apache:
request: javax.servlet.http.HttpServletRequest, title: String,content: => Seq[scala.xml.Node], activeTab: org.apache.spark.ui.SparkUITab, helpText: Option[String], showVisualization: Boolean, useDataTables: Boolean