[SPARK-13142] [Web UI] Problem accessing Web UI /logPage/ on Microsoft Windows

[markpavey]

Due to being on a Windows platform I have been unable to run the tests as described in the "Contributing to Spark" instructions. As the change is only to two lines of code in the Web UI, which I have manually built and tested, I am submitting this pull request anyway. I hope this is OK.

Is it worth considering also including this fix in any future 1.5.x releases (if any)?

I confirm this is my own original work and license it to the Spark project under its open source license.

[srowen]

It seems like a small OK change that fixes your problem. I wonder however if there are other instances? I'd love to see if we can fix this more consistently.

I don't know, is it feasible to fix the hard-coding at all? it may be too widespread.

I also wonder if the Path API in JDK 7 helps at all, but that seems unimportant

[andrewor14]

ok to test

[SparkQA]

Test build #50987 has finished for PR 11135 at commit c517ead.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[markpavey]

I am confused as Jenkins is saying that unit tests failed but the test results show they all passed. The console output shows a "Running java...failed" error part way through the test suite but I can't see what has caused this.

[srowen]

Here's the full log, but I agree it's not quite clear what failed:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50987/consoleFull

I'm all but sure it's a spurious failure. We can retest it; it will kick off again if you submit more changes anyway.

[markpavey]

Is it possible to retest without submitting more changes? I haven't got any more to add.

A quick search of the spark-core code base shows that URIs are mostly created with File.toURI(), hence I went for that in my patch.

There are a couple of different approaches:

• org.apache.spark.util.Utils.resolveURI(path) catches the URISyntaxException
• SparkContext.addJar(path) looks for a backslash character to tell if it's a Windows path
• org.apache.spark.deploy.ClientArguments.isValidJarUrl(s) treats paths as invalid if they fail URI parsing

An exhaustive search might reveal more.

It is only the problem with viewing the logs in the web UI that has been causing us pain, hence that was what we fixed.

[srowen]

Jenkins, retest this please

[SparkQA]

Test build #51034 has finished for PR 11135 at commit c517ead.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Merged to master/1.6