Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-34151][SQL] Replaces java.io.File.toURL with java.io.File.toURI.toURL #31230

Closed
wants to merge 1 commit into from

Conversation

LuciferYang
Copy link
Contributor

What changes were proposed in this pull request?

java.io.FIle.toURL method does not automatically escape characters that are illegal in URLs.

Java doc recommended that new code convert an abstract pathname into a URL by first converting it into a URI, via the toURI method, and then converting the URI into a URL via the URI.toURL method.

So this pr cleaned up the relevant cases in Spark code.

Why are the changes needed?

Cleaning up @Deprecated Java API usage.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass the Jenkins or GitHub Action

@SparkQA
Copy link

SparkQA commented Jan 18, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38775/

@SparkQA
Copy link

SparkQA commented Jan 18, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38775/

@SparkQA
Copy link

SparkQA commented Jan 18, 2021

Test build #134190 has finished for PR 31230 at commit 22376ed.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@github-actions github-actions bot added the SQL label Jan 18, 2021
@HyukjinKwon
Copy link
Member

Are they all instances to fix @LuciferYang?

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jan 18, 2021

@HyukjinKwon Let me check it again

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jan 18, 2021

private def getExtractPartFunc(partToExtract: UTF8String): URI => String = {
// partToExtract match {
// case HOST => _.toURL().getHost
// case PATH => _.toURL().getPath
// case QUERY => _.toURL().getQuery
// case REF => _.toURL().getRef
// case PROTOCOL => _.toURL().getProtocol
// case FILE => _.toURL().getFile
// case AUTHORITY => _.toURL().getAuthority
// case USERINFO => _.toURL().getUserInfo
// case _ => (url: URI) => null
// }
partToExtract match {

@HyukjinKwon The rest is as above. I don't know if this part of code is still useful.

EDIT: seems _ is URI.

Are they all instances to fix @LuciferYang?

Yes

@HyukjinKwon
Copy link
Member

Merged to master and branch-3.1.

@HyukjinKwon
Copy link
Member

Thanks @LuciferYang.

HyukjinKwon pushed a commit that referenced this pull request Jan 18, 2021
…oURI.toURL`

### What changes were proposed in this pull request?
`java.io.FIle.toURL` method does not automatically escape characters that are illegal in URLs.

Java doc recommended that new code convert an abstract pathname into a URL by first converting it into a URI, via the `toURI` method, and then converting the URI into a URL via the `URI.toURL` method.

So this pr cleaned up the relevant cases in Spark code.

### Why are the changes needed?
Cleaning up `Deprecated` Java API usage.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass the Jenkins or GitHub Action

Closes #31230 from LuciferYang/SPARK-34151.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
(cherry picked from commit 163afa6)
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
@LuciferYang
Copy link
Contributor Author

thx @HyukjinKwon ~

skestle pushed a commit to skestle/spark that referenced this pull request Feb 3, 2021
…oURI.toURL`

### What changes were proposed in this pull request?
`java.io.FIle.toURL` method does not automatically escape characters that are illegal in URLs.

Java doc recommended that new code convert an abstract pathname into a URL by first converting it into a URI, via the `toURI` method, and then converting the URI into a URL via the `URI.toURL` method.

So this pr cleaned up the relevant cases in Spark code.

### Why are the changes needed?
Cleaning up `Deprecated` Java API usage.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass the Jenkins or GitHub Action

Closes apache#31230 from LuciferYang/SPARK-34151.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
@LuciferYang LuciferYang deleted the SPARK-34151 branch June 6, 2022 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants