-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-48205][PYTHON] Remove the private[sql] modifier for Python data sources #46487
[SPARK-48205][PYTHON] Remove the private[sql] modifier for Python data sources #46487
Conversation
cc @HyukjinKwon |
Merged to master. |
@@ -234,7 +234,7 @@ class SparkSession private( | |||
/** | |||
* A collection of methods for registering user-defined data sources. | |||
*/ | |||
private[sql] def dataSource: DataSourceRegistration = sessionState.dataSourceRegistration |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add @since
tag because we open this, @allisonwang-db ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, please set appropriate tags like the following.
@Experimental
@Unstable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh yeah
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do!
### What changes were proposed in this pull request? This is a follow-up PR for #46487 to add missing tags for the `dataSource` API. ### Why are the changes needed? To address comments from a previous PR. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing test ### Was this patch authored or co-authored using generative AI tooling? No Closes #46530 from allisonwang-db/spark-48205-followup. Authored-by: allisonwang-db <allison.wang@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
…a sources ### What changes were proposed in this pull request? This PR removes the `private[sql]` modifier for Python data sources to make it consistent with UDFs and UDTFs. ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#46487 from allisonwang-db/spark-48205-pyds-modifier. Authored-by: allisonwang-db <allison.wang@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
### What changes were proposed in this pull request? This is a follow-up PR for apache#46487 to add missing tags for the `dataSource` API. ### Why are the changes needed? To address comments from a previous PR. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing test ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#46530 from allisonwang-db/spark-48205-followup. Authored-by: allisonwang-db <allison.wang@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
This PR removes the
private[sql]
modifier for Python data sources to make it consistent with UDFs and UDTFs.Why are the changes needed?
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing tests
Was this patch authored or co-authored using generative AI tooling?
No