Skip to content

[SPARK-50578][PYTHON][SS][FOLLOW-UP] Add the Python new package into setup.py#50128

Closed
HyukjinKwon wants to merge 1 commit intoapache:masterfrom
HyukjinKwon:SPARK-50578-followup2
Closed

[SPARK-50578][PYTHON][SS][FOLLOW-UP] Add the Python new package into setup.py#50128
HyukjinKwon wants to merge 1 commit intoapache:masterfrom
HyukjinKwon:SPARK-50578-followup2

Conversation

@HyukjinKwon
Copy link
Member

What changes were proposed in this pull request?

This PR is a followup of #49156 that adds the Python new package into setup.py.

Why are the changes needed?

So pip installed pyspark support states with streaming queries

Does this PR introduce any user-facing change?

No, the main change has not been released out yet.

How was this patch tested?

Will monitor the scheduled build https://github.com/apache/spark/actions/runs/13618426037/job/38064498674

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the PYTHON label Mar 3, 2025
@HyukjinKwon
Copy link
Member Author

cc @HeartSaVioR @jingz-db

@jingz-db
Copy link
Contributor

jingz-db commented Mar 3, 2025

cc @HeartSaVioR @jingz-db

Hey @HyukjinKwon , unfortunately this suite is flaky only on CI but I am not able to reproduce the failure locally. Could you run CI one more time and unblock yourself from merging your PR? I filed a JIRA to keep track of the fix (but I probably need sometime to investigate into it as I am not able to reproduce locally): https://issues.apache.org/jira/browse/SPARK-51368

If you think this may impact the overall testing health of Spark tests, feel free to merge this PR for temporarily disabling the flaky schema evolution test case for spark connect suite here: #50130

@HyukjinKwon
Copy link
Member Author

This is for a scheduled build that tests PySpark installed via PyPI. I think this PR fixes something different from what you mentioned.

@jingz-db
Copy link
Contributor

jingz-db commented Mar 3, 2025

This is for a scheduled build that tests PySpark installed via PyPI. I think this PR fixes something different from what you mentioned.

Oh I thought you were tagging me for fixing the failed flaky suite on CI. If not, then please ignore my thread above.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @HyukjinKwon .
Merged to master/4.0.

dongjoon-hyun pushed a commit that referenced this pull request Mar 3, 2025
…setup.py

### What changes were proposed in this pull request?

This PR is a followup of #49156 that adds the Python new package into `setup.py`.

### Why are the changes needed?

So pip installed pyspark support states with streaming queries

### Does this PR introduce _any_ user-facing change?

No, the main change has not been released out yet.

### How was this patch tested?

Will monitor the scheduled build https://github.com/apache/spark/actions/runs/13618426037/job/38064498674

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #50128 from HyukjinKwon/SPARK-50578-followup2.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 4b2ae48)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 14, 2025
…setup.py

### What changes were proposed in this pull request?

This PR is a followup of apache#49156 that adds the Python new package into `setup.py`.

### Why are the changes needed?

So pip installed pyspark support states with streaming queries

### Does this PR introduce _any_ user-facing change?

No, the main change has not been released out yet.

### How was this patch tested?

Will monitor the scheduled build https://github.com/apache/spark/actions/runs/13618426037/job/38064498674

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#50128 from HyukjinKwon/SPARK-50578-followup2.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 73d5506)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants