Skip to content

Conversation

@davidm-db
Copy link
Contributor

@davidm-db davidm-db commented Nov 24, 2025

Porting #53155 to branch-4.1 and resolving merge conflicts.

The only difference is a minor note in docs/sql-ref-scripting.md on how to enable SQL Scripting, since it's not enabled by default in Spark 4.1.

What changes were proposed in this pull request?

Adding documentation for SQL Scripting feature.

Why are the changes needed?

If it is not documented, it never happened :-)

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Documentation change, reviewed manually.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the DOCS label Nov 24, 2025
@davidm-db
Copy link
Contributor Author

cc @dongjoon-hyun


## Enablement

To enable SQL Scripting, set the `spark.sql.scripting.enabled` flag to `true`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of progress on SQL scripting in 4.1, and scripting GA can be a nice achievement to put in the 4.1 release notes. I think the feature is production ready now with the document committed (This feature is available in Databricks products for more than 6 months), but I'll leave the decision to the release manager @dongjoon-hyun

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the meanwhile, let me merge this doc PR first.

@cloud-fan
Copy link
Contributor

thanks, merging to 4.1!

cloud-fan pushed a commit that referenced this pull request Nov 24, 2025
Porting #53155 to `branch-4.1` and resolving merge conflicts.

The only difference is a minor note in `docs/sql-ref-scripting.md` on how to enable SQL Scripting, since it's not enabled by default in Spark 4.1.

### What changes were proposed in this pull request?

Adding documentation for SQL Scripting feature.

### Why are the changes needed?

If it is not documented, it never happened :-)

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Documentation change, reviewed manually.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #53194 from davidm-db/scripting-docs-4.1.

Lead-authored-by: Serge Rielau <serge@rielau.com>
Co-authored-by: David Milicevic <david.milicevic@databricks.com>
Co-authored-by: David Milicevic <163021185+davidm-db@users.noreply.github.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@cloud-fan cloud-fan closed this Nov 24, 2025
@dongjoon-hyun
Copy link
Member

Thank you, @davidm-db and @cloud-fan .

@dongjoon-hyun
Copy link
Member

Got it, @cloud-fan .

There are a lot of progress on SQL scripting in 4.1, and scripting GA can be a nice achievement to put in the 4.1 release notes. I think the feature is production ready now with the document committed (This feature is available in Databricks products for more than 6 months), but I'll leave the decision to the release manager @dongjoon-hyun

@dongjoon-hyun
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants