Skip to content

Commit

Permalink
[SPARK-8952] [SPARKR] - Wrap normalizePath calls with suppressWarnings
Browse files Browse the repository at this point in the history
This is based on davies comment on SPARK-8952 which suggests to only call normalizePath() when path starts with '~'

Author: Luciano Resende <lresende@apache.org>

Closes #8343 from lresende/SPARK-8952.
  • Loading branch information
lresende authored and shivaram committed Aug 28, 2015
1 parent e2a8430 commit 499e8e1
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
4 changes: 2 additions & 2 deletions R/pkg/R/SQLContext.R
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ setMethod("toDF", signature(x = "RDD"),

jsonFile <- function(sqlContext, path) {
# Allow the user to have a more flexible definiton of the text file path
path <- normalizePath(path)
path <- suppressWarnings(normalizePath(path))
# Convert a string vector of paths to a string containing comma separated paths
path <- paste(path, collapse = ",")
sdf <- callJMethod(sqlContext, "jsonFile", path)
Expand Down Expand Up @@ -251,7 +251,7 @@ jsonRDD <- function(sqlContext, rdd, schema = NULL, samplingRatio = 1.0) {
# TODO: Implement saveasParquetFile and write examples for both
parquetFile <- function(sqlContext, ...) {
# Allow the user to have a more flexible definiton of the text file path
paths <- lapply(list(...), normalizePath)
paths <- lapply(list(...), function(x) suppressWarnings(normalizePath(x)))
sdf <- callJMethod(sqlContext, "parquetFile", paths)
dataFrame(sdf)
}
Expand Down
2 changes: 1 addition & 1 deletion R/pkg/R/sparkR.R
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ sparkR.init <- function(
})

if (nchar(sparkHome) != 0) {
sparkHome <- normalizePath(sparkHome)
sparkHome <- suppressWarnings(normalizePath(sparkHome))
}

sparkEnvirMap <- new.env()
Expand Down

0 comments on commit 499e8e1

Please sign in to comment.