[SPARK-32723] [WEBUI] Upgrade to jQuery 3.5.1, 2.4 backport. #29922
Conversation
We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier. jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/) 2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple. Update jquery to 3.4.1; update jquery blockUI and mustache to latest Manual testing of docs build (except R docs), worker/master UI, spark application UI. Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main. Closes apache#24843 from srowen/SPARK-28004. Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Upgrade to the latest available version of jQuery (3.5.1). There are some CVE-s reported (CVE-2020-11022, CVE-2020-11023) affecting older versions of jQuery. Although Spark UI is read-only and those CVEs doesn't seem to affect Spark, using the latest version of this library can help to handle vulnerability reports of security scans. No. Manual tests and checked the jQuery 3.5 upgrade guide. Closes apache#29902 from peter-toth/SPARK-32723-upgrade-to-jquery-3.5.1. Authored-by: Peter Toth <peter.toth@gmail.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
|
Jenkins test this please |
|
Hi, @srowen . Do you want to backport jQuery 3.5.1 to branch-3.0? |
|
Kubernetes integration test starting |
|
Ah, right, needs to be in 3.0 first. Does that need a backport? |
|
Kubernetes integration test status failure |
|
For what it is worth; once I upgrade Spark 2.4 to jQuery 3.4.1; upgrading to jQuery 3.5.1 wasn't difficult at all. So I suspect Spark 3.0 should be the same. Separately, took a look at the Kubernetes integration tests; and I'm unsure if they're related to my fix at all. Also couldn't understand where the failure was in Build and test / Build modules: pyspark-sql, pyspark-mllib (JDK 1.8, hadoop2.6). Can that be rerun? |
|
@srowen . For old releases (3.0.x/2.4.x), I don't think the reported CVEs are required in Apache Spark. So, this backport is not urgent. However, if you want to backport them, I'm not against it. What I want to recommend here is that just holding on for a month or two to stabilize it in |
|
Test build #129310 has finished for PR 29922 at commit
|
|
Yeah that may be a good position - if we don't really have any reason to believe the CVE affects Spark (do we?) then maybe we don't. Or stick to the smaller backport to 3.0 |
|
Thanks, @srowen . |
|
Test build #133779 has finished for PR 29922 at commit
|
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
What changes were proposed in this pull request?
Backport of SPARK-28004 and SPARK-32723 to Spark 2.4.
Why are the changes needed?
jQuery prior to 3.5.1 has been known to have CVEs.
How was this patch tested?
Full unit tests and manual tests focused on UIs for Master, Worker, Application