Skip to content

Comments

[SPARK-38546][EXAMPLES] Replace deprecated ChiSqSelector with UnivariateFeatureSelector#35846

Closed
dcoliversun wants to merge 1 commit intoapache:masterfrom
dcoliversun:SPARK-38546
Closed

[SPARK-38546][EXAMPLES] Replace deprecated ChiSqSelector with UnivariateFeatureSelector#35846
dcoliversun wants to merge 1 commit intoapache:masterfrom
dcoliversun:SPARK-38546

Conversation

@dcoliversun
Copy link
Contributor

What changes were proposed in this pull request?

This PR aims to replace deprecated ChiSqSelector with UnivariateFeatureSelector.

Why are the changes needed?

In #31160, ChiSqSelector was labeled as deprecated.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Run JavaChiSqSelectorExample Before the PR

ChiSqSelector output with top 1 features selected
22/03/14 15:11:56 INFO CodeGenerator: Code generated in 6.124107 ms
22/03/14 15:11:56 INFO CodeGenerator: Code generated in 12.399323 ms
22/03/14 15:11:56 INFO CodeGenerator: Code generated in 6.305541 ms
22/03/14 15:11:56 INFO CodeGenerator: Code generated in 8.784765 ms
+---+------------------+-------+----------------+
| id|          features|clicked|selectedFeatures|
+---+------------------+-------+----------------+
|  7|[0.0,0.0,18.0,1.0]|    1.0|          [18.0]|
|  8|[0.0,1.0,12.0,0.0]|    0.0|          [12.0]|
|  9|[1.0,0.0,15.0,0.1]|    0.0|          [15.0]|
+---+------------------+-------+----------------+

Run JavaChiSqSelectorExample After the PR

UnivariateFeatureSelector for chi-squared output with top 1.0 features selected
22/03/14 19:36:54 INFO CodeGenerator: Code generated in 9.093969 ms
22/03/14 19:36:54 INFO CodeGenerator: Code generated in 15.667814 ms
22/03/14 19:36:54 INFO CodeGenerator: Code generated in 10.053778 ms
22/03/14 19:36:54 INFO CodeGenerator: Code generated in 12.287391 ms
+---+------------------+-------+----------------+
| id|          features|clicked|selectedFeatures|
+---+------------------+-------+----------------+
|  7|[0.0,0.0,18.0,1.0]|    1.0|          [18.0]|
|  8|[0.0,1.0,12.0,0.0]|    0.0|          [12.0]|
|  9|[1.0,0.0,15.0,0.1]|    0.0|          [15.0]|
+---+------------------+-------+----------------+

@dcoliversun
Copy link
Contributor Author

@huaxingao @zhengruifeng @WeichenXu123
It would be good if you could take a look when you have time, thanks!

@HyukjinKwon HyukjinKwon changed the title [SPARK-38546][EXAMPLES] replace deprecated ChiSqSelector with UnivariateFeatureSelector [SPARK-38546][EXAMPLES] Replace deprecated ChiSqSelector with UnivariateFeatureSelector Mar 15, 2022
@huaxingao
Copy link
Contributor

@dcoliversun Thanks for working on this. However, we already have an example for UnivariateFeatureSelector here. Your effort is appreciated!

@huaxingao huaxingao closed this Mar 15, 2022
@dcoliversun dcoliversun deleted the SPARK-38546 branch March 15, 2022 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants