From d5a028f98ef84058dc9ba1812b8c3736b16c8fc6 Mon Sep 17 00:00:00 2001 From: Felix Cheung Date: Thu, 28 Dec 2017 14:23:04 -0800 Subject: [PATCH 1/3] update guide, vig --- R/pkg/vignettes/sparkr-vignettes.Rmd | 3 +-- docs/sparkr.md | 6 ++++++ 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/R/pkg/vignettes/sparkr-vignettes.Rmd b/R/pkg/vignettes/sparkr-vignettes.Rmd index 8c4ea2f2db188..2e662424b25f2 100644 --- a/R/pkg/vignettes/sparkr-vignettes.Rmd +++ b/R/pkg/vignettes/sparkr-vignettes.Rmd @@ -391,8 +391,7 @@ We convert `mpg` to `kmpg` (kilometers per gallon). `carsSubDF` is a `SparkDataF ```{r} carsSubDF <- select(carsDF, "model", "mpg") -schema <- structType(structField("model", "string"), structField("mpg", "double"), - structField("kmpg", "double")) +schema <- "model STRING, mpg DOUBLE, kmpg DOUBLE" out <- dapply(carsSubDF, function(x) { x <- cbind(x, x$mpg * 1.61) }, schema) head(collect(out)) ``` diff --git a/docs/sparkr.md b/docs/sparkr.md index a3254e7654134..937410dd1f97b 100644 --- a/docs/sparkr.md +++ b/docs/sparkr.md @@ -657,3 +657,9 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma - By default, derby.log is now saved to `tempdir()`. This will be created when instantiating the SparkSession with `enableHiveSupport` set to `TRUE`. - `spark.lda` was not setting the optimizer correctly. It has been corrected. - Several model summary outputs are updated to have `coefficients` as `matrix`. This includes `spark.logit`, `spark.kmeans`, `spark.glm`. Model summary outputs for `spark.gaussianMixture` have added log-likelihood as `loglik`. + +## Upgrading to SparkR 2.3.0 + + - The `stringsAsFactors` parameter was previously ignored with `collect`, for example, in `collect(createDataFrame(iris), stringsAsFactors = TRUE))`. It has been corrected. + - For `summary`, option for statistics to compute has been added. Its output is changed from that from `describe`. + - A warning can be raised if the version of SparkR package and the Spark JVM do not match. From 918d600bed0e9011c6150619d4d8bd1dcc30b76e Mon Sep 17 00:00:00 2001 From: Felix Cheung Date: Thu, 28 Dec 2017 14:24:13 -0800 Subject: [PATCH 2/3] fix --- docs/sparkr.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sparkr.md b/docs/sparkr.md index 937410dd1f97b..997ea60fb6cf0 100644 --- a/docs/sparkr.md +++ b/docs/sparkr.md @@ -662,4 +662,4 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma - The `stringsAsFactors` parameter was previously ignored with `collect`, for example, in `collect(createDataFrame(iris), stringsAsFactors = TRUE))`. It has been corrected. - For `summary`, option for statistics to compute has been added. Its output is changed from that from `describe`. - - A warning can be raised if the version of SparkR package and the Spark JVM do not match. + - A warning can be raised if versions of SparkR package and the Spark JVM do not match. From 4a026372fcac2073b6cafd61b4d1ff7a1eff1b05 Mon Sep 17 00:00:00 2001 From: Felix Cheung Date: Fri, 29 Dec 2017 10:54:45 -0800 Subject: [PATCH 3/3] try to trigger appveyor --- R/pkg/tests/fulltests/test_Windows.R | 1 + 1 file changed, 1 insertion(+) diff --git a/R/pkg/tests/fulltests/test_Windows.R b/R/pkg/tests/fulltests/test_Windows.R index b2ec6c67311db..209827d9fdc2f 100644 --- a/R/pkg/tests/fulltests/test_Windows.R +++ b/R/pkg/tests/fulltests/test_Windows.R @@ -14,6 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. # + context("Windows-specific tests") test_that("sparkJars tag in SparkContext", {