Skip to content

Conversation

@yanboliang
Copy link
Contributor

Add read/write support to the following estimators under spark.ml:

  • ChiSqSelector
  • PCA
  • VectorIndexer
  • Word2Vec

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46323 has finished for PR 9838 at commit 5e79781.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class LogisticRegressionModelWriter(instance: LogisticRegressionModel)\n * class ChiSqSelectorModelWriter(instance: ChiSqSelectorModel) extends MLWriter\n * class PCA (override val uid: String) extends Estimator[PCAModel] with PCAParams\n * class VectorIndexerModelWriter(instance: VectorIndexerModel) extends MLWriter\n * final class Word2Vec(override val uid: String) extends Estimator[Word2VecModel] with Word2VecBase\n * class Word2VecModelWriter(instance: Word2VecModel) extends MLWriter\n

@yanboliang
Copy link
Contributor Author

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46326 has finished for PR 9838 at commit e3d5fc6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class ChiSqSelectorModelWriter(instance: ChiSqSelectorModel) extends MLWriter\n * class PCA (override val uid: String) extends Estimator[PCAModel] with PCAParams\n * class VectorIndexerModelWriter(instance: VectorIndexerModel) extends MLWriter\n * final class Word2Vec(override val uid: String) extends Estimator[Word2VecModel] with Word2VecBase\n * class Word2VecModelWriter(instance: Word2VecModel) extends MLWriter\n

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46327 has finished for PR 9838 at commit e3d5fc6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class ChiSqSelectorModelWriter(instance: ChiSqSelectorModel) extends MLWriter\n * class PCA (override val uid: String) extends Estimator[PCAModel] with PCAParams\n * class VectorIndexerModelWriter(instance: VectorIndexerModel) extends MLWriter\n * final class Word2Vec(override val uid: String) extends Estimator[Word2VecModel] with Word2VecBase\n * class Word2VecModelWriter(instance: Word2VecModel) extends MLWriter\n

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same as StandardScalerModel, we can not construct PCAModel directly by specifying the variable k, so test estimator and model in one case with testEstimatorAndModelReadWrite.

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46332 has finished for PR 9838 at commit ec969de.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class ChiSqSelectorModelWriter(instance: ChiSqSelectorModel) extends MLWriter\n * class PCA (override val uid: String) extends Estimator[PCAModel] with PCAParams\n * class VectorIndexerModelWriter(instance: VectorIndexerModel) extends MLWriter\n * final class Word2Vec(override val uid: String) extends Estimator[Word2VecModel] with Word2VecBase\n * class Word2VecModelWriter(instance: Word2VecModel) extends MLWriter\n

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we keep them private[spark] for this release? We should not introduce unplanned APIs at this stage.

@mengxr
Copy link
Contributor

mengxr commented Nov 19, 2015

LGTM except two minor issues commented inline.

@SparkQA
Copy link

SparkQA commented Nov 20, 2015

Test build #46384 has finished for PR 9838 at commit 0cb01a3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * class ChiSqSelectorModelWriter(instance: ChiSqSelectorModel) extends MLWriter\n * class PCA (override val uid: String) extends Estimator[PCAModel] with PCAParams\n * class VectorIndexerModelWriter(instance: VectorIndexerModel) extends MLWriter\n * final class Word2Vec(override val uid: String) extends Estimator[Word2VecModel] with Word2VecBase\n * class Word2VecModelWriter(instance: Word2VecModel) extends MLWriter\n

asfgit pushed a commit that referenced this pull request Nov 20, 2015
Add read/write support to the following estimators under spark.ml:
* ChiSqSelector
* PCA
* VectorIndexer
* Word2Vec

Author: Yanbo Liang <ybliang8@gmail.com>

Closes #9838 from yanboliang/spark-11829.

(cherry picked from commit 3b7f056)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
@mengxr
Copy link
Contributor

mengxr commented Nov 20, 2015

Merged into master and branch-1.6. Thanks!

@asfgit asfgit closed this in 3b7f056 Nov 20, 2015
@yanboliang yanboliang deleted the spark-11829 branch November 20, 2015 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants