[SPARK-58519][SQL] Document UPDATE and DELETE FROM statements

[marcuslin123]

What changes were proposed in this pull request?

Add SQL reference pages for the UPDATE and DELETE FROM statements. Both pages document:

• statement syntax
• table aliases
• dynamic table options
• assignment or filtering parameters
• conditional and whole-table examples

This PR also links the new pages from the SQL syntax index and adds them to the related statements
on the MERGE INTO page.

Why are the changes needed?

Spark supports UPDATE and DELETE FROM for Data Source V2 tables, but the SQL reference did not
have dedicated pages for either statement. Users therefore could not find their syntax, parameters,
or examples alongside the other DML statements.

Does this PR introduce any user-facing change?

No. This is a documentation-only change for existing SQL functionality.

How was this patch tested?

The focused parser tests passed:

build/sbt 'catalyst/testOnly *DDLParserSuite -- -z "delete from table" -z "update table"'

The run completed with 10 tests passed and no failures. The staged changes also pass
git diff --cached --check, and the new Markdown links and code fences were checked locally.

The full Jekyll build was not run because Bundler 2.4.22 is not installed in the environment.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-5) was used for general coding assistance.

[anuragmantri]

The SET column = expression entry says a value may be an expression or DEFAULT, which is accurate, but doesn't call out that the expression can be an uncorrelated subquery over a completely different table (e.g. SET salary = (SELECT max(salary) FROM other_table)). Worth adding a sentence here.

Suggested change

	* **SET column = expression [ , ... ]**
	Assigns a value to one or more columns. Each value may be an expression or `DEFAULT`. A nested
	field may be targeted by using a qualified column name.
	* **SET column = value [ , ... ]**
	Specifies the columns to update and the values to assign to them. Each `value` is an
	expression, typically referencing columns of the target table, but it may also be an
	uncorrelated subquery over other tables. A comma separates each assignment.

[anuragmantri]

Both WITH (option_key = option_value) entries (UPDATE and DELETE FROM) describe the clause generically but don't mention: (a) that these options apply to this statement only, without changing the table's persistent configuration, (b) that a key that isn't a valid identifier needs backtick-quoting, and (c) that options the connector doesn't recognize are silently ignored. All three are worth stating explicitly so users know what to expect.

Suggested change

	* **WITH ( option_key = option_value [ , ... ] )**
	Specifies dynamic table options for this `UPDATE` operation. These options are passed to the
	data source connector when writing to the table. The supported options depend on the connector.
	* **WITH ( key = val [ , ... ] )**
	An optional list of dynamic table options passed to the Data Source V2 connector for this
	statement only. The options are surfaced to the connector's row-level write, allowing
	per-statement tuning (for example a write file size or an isolation level) without changing
	the table configuration. Keys and values are treated as strings; a key that is not a valid
	identifier can be quoted with backticks. Options that the connector does not recognize are
	ignored.

[anuragmantri]

For consistency, link to INSERT statement as well?

[anuragmantri]

@peter-toth, @szehon-ho - Could you take a look?

[szehon-ho]

Thanks for adding these -- the syntax all checks out. I verified the clause order against SqlBaseParser.g4 (DELETE FROM identifierReference tableAlias optionsClause? whereClause? and UPDATE identifierReference tableAlias optionsClause? setClause whereClause?, lines 750-751), so alias, then WITH (...), then SET / WHERE is right. The claims in the SET bullet are all backed by UpdateTableSuiteBase: alias-qualified assignment, nested field targeting, DEFAULT, and the uncorrelated scalar subquery. I also confirmed _data/menu-sql.yaml only links section anchors rather than individual DML pages, so the sql-ref-syntax.md update is all the index needs.

A few non-blocking comments below, mostly about making the examples reproducible the way the neighbouring DML pages do.

[szehon-ho]

Two minor things here.

First, consistency: #57724 documents this same WITH (...) clause on the INSERT, MERGE INTO, and SELECT pages in two sentences ("Specifies dynamic table options for this INSERT operation. These options are passed to the data source connector when writing to the table. The supported options depend on the connector."). With both PRs landing around the same time, readers will hit the same clause described at two different levels of detail depending on which page they land on. Worth either syncing the wording across all five pages, or describing it once -- the Row-Level DML section of sql-v2-data-sources.md that this page already links to is a plausible home -- and giving each page the one-line version plus a link.

Second, if the long form stays, it can lose some weight. The second sentence restates "for this statement only" from the first, and the last sentence makes the same point twice ("passes options through without validating their names" / "connectors may ignore options they do not recognize"). The backtick note is the part that really earns its place, since write.split-size in the example below is a parse error unquoted (propertyKey is identifier (DOT identifier)* | stringLit). One thing that might be worth having in place of "keys and values are treated as strings": keys are case-insensitive, since the options end up in a CaseInsensitiveStringMap (AstBuilder.resolveOptions). That is behavior a user cannot guess.

[szehon-ho]

The neighbouring DML pages make their examples runnable end to end. MERGE INTO opens its Examples section with the initial SELECT * FROM target / SELECT * FROM source state and shows the resulting table under each example; INSERT TABLE creates students inline and shows the output after each insert.

Here there is no schema for employees and no results, so a reader cannot tell what status, department, last_active_date, or salary are, or check that they got the expected outcome. Could you add a small CREATE TABLE plus initial SELECT * block here, and a result under each example?

[szehon-ho]

Minor, and it mostly goes away once the example table has a stated schema: if salary is an INT, then salary * 1.05 is a DOUBLE, and ANSI store assignment permits numeric narrowing (Cast.canANSIStoreAssign allows any NumericType -> NumericType), so the 5% raise is silently truncated back to an INT. Giving salary a DECIMAL or DOUBLE type in the setup, or using an increment that stays integral, avoids demonstrating truncation by accident.

[szehon-ho]

Same bullet as on the UPDATE page -- whatever wording you settle on there, please keep these two in sync (and ideally with the INSERT / MERGE INTO / SELECT pages in #57724).

[szehon-ho]

Same as the UPDATE page: no schema for employees and no results under the examples, unlike MERGE INTO and INSERT TABLE.

[szehon-ho]

Nit on placement. The DDL list above is strictly alphabetical, while this DML list is not (INSERT TABLE precedes INSERT OVERWRITE DIRECTORY, and LOAD trails MERGE INTO), so putting DELETE FROM first and UPDATE between MERGE INTO and LOAD ends up arbitrary either way. I would either append UPDATE after LOAD or alphabetize the whole list.

[uros-b]

Should we add [DOCS] to the PR title?

[peter-toth]

Thanks for the PR, @marcuslin123!

I checked both syntax blocks against SqlBaseParser.g4:750-751 and the parameter prose against the analyzer and planner paths -- nothing either page states is wrong, so my findings are all about what they leave out: the catalog part of the table name (these statements only work on V2 tables, so catalog.db.table is the form users type), the extra conditions a filter-only connector puts on DELETE's WHERE clause, and two limits behind the UPDATE SET bullet. On that last one I read it a bit differently from @szehon-ho -- the claims are individually true, but DEFAULT and nested-field targeting each have a restriction the bullet doesn't state.

I'm not re-opening the threads @anuragmantri and @szehon-ho already have. One correction for the dynamic-options wording thread, since it decides what #57724 says too: "passed to the data source connector when writing to the table" is incomplete for UPDATE / DELETE -- the options also reach the read side of the row-level operation (RewriteRowLevelCommand.buildOperationTable puts r.options into RowLevelOperationInfoImpl, and buildRelationWithAttrs keeps them on the scan relation). Also, the red "Report test results" check is only "No test results found!", which is expected for a docs-only change.

Non-blocking

• 1. Table name syntax omits the catalog: both pages document [ database_name. ] table_name, but these statements only work on V2 tables, which normally live in a catalog registered via spark.sql.catalog.*. [inline: docs/sql-ref-syntax-dml-update.md:46, docs/sql-ref-syntax-dml-delete-from.md:45]
• 2. DELETE FROM doesn't state the filter-only connector restrictions: on a connector that supports only filter-based delete, the condition must be subquery-free, must translate into V2 predicates, and must be accepted by canDeleteWhere -- any of the three failing is a hard planning error, not a fallback to rewriting. Two of the three already have tests. Every example on the page is a simple pushable predicate, so nothing hints at this. [inline: docs/sql-ref-syntax-dml-delete-from.md:63]
• 3. SET bullet leaves out the DEFAULT and nested-field limits: DEFAULT has to be the whole value, and nested targeting only works through structs. Two more rules belong here too -- at most one assignment per column, and values read the pre-update row. [inline: docs/sql-ref-syntax-dml-update.md:66]

Minor

• 4. UPDATE on a table with generated columns is rejected: RewriteUpdateTable.scala:43 fails the statement when Spark owns the generated values, which the "is supported on" sentence doesn't allow for. MERGE INTO's page has the same gap. [inline: docs/sql-ref-syntax-dml-update.md:29]
• 5. Title tag: comparable docs-only SQL-reference commits carry [DOCS] / [DOC] -- SPARK-57359 (the MERGE INTO page), SPARK-57643, SPARK-58176. [SPARK-58519][SQL][DOCS] would match.

[peter-toth]

Finding 1. UPDATE only works on Data Source V2 tables, and a V2 table normally lives in a catalog registered through spark.sql.catalog.* rather than in the session catalog -- so the form users actually type is catalog.db.table, which this syntax doesn't cover. The grammar allows it: identifierReference resolves to multipartIdentifier (SqlBaseParser.g4:751), so any number of name parts is accepted. Several reference pages already use the fuller form, e.g. docs/sql-ref-syntax-ddl-drop-view.md:44 and docs/sql-ref-syntax-aux-describe-function.md:52.

Suggested change

	Specifies the table to update, which may be optionally qualified with a database name.
	**Syntax:** `[ database_name. ] table_name`
	Specifies the table to update, which may be optionally qualified with a catalog and a database
	name.
	**Syntax:** `[ catalog_name. ] [ database_name. ] table_name`

Same on docs/sql-ref-syntax-dml-delete-from.md:45.

[peter-toth]

Finding 1. Same as on the UPDATE page (docs/sql-ref-syntax-dml-update.md:46): a V2 table is normally reached as catalog.db.table, and identifierReference (SqlBaseParser.g4:750) accepts any number of name parts.

Suggested change

	Specifies the table from which rows are deleted. The table name may be optionally qualified
	with a database name.
	**Syntax:** `[ database_name. ] table_name`
	Specifies the table from which rows are deleted. The table name may be optionally qualified
	with a catalog and a database name.
	**Syntax:** `[ catalog_name. ] [ database_name. ] table_name`

[peter-toth]

Finding 2. The page presents a single DELETE FROM, but there are two execution paths with different rules for the condition, and the stricter one isn't mentioned. When the table supports only filter-based delete (SupportsDeleteV2 / SupportsDelete without SupportsRowLevelOperations), RewriteDeleteFromTable.scala:55-56 leaves the command alone and DataSourceV2Strategy.scala:533-552 then demands all three of: no subquery in the condition, every conjunct translatable into a V2 Predicate, and canDeleteWhere accepting the resulting set. Any one failing is a hard error at planning, not a fallback to rewriting.

Two of the three are already covered by tests against InMemoryTable, which is filter-only (InMemoryTable.scala:53): DeleteFromTests.scala:74 -- DELETE FROM t WHERE id IN (SELECT id FROM t) fails with "Delete by condition with subquery is not supported"; DeleteFromTests.scala:88 and DataSourceV2SQLSuite.scala:2578 -- WHERE id > 3 AND p > 3 on a non-partitioned table fails with "Cannot delete from table" because canDeleteWhere returns false. The untranslatable-conjunct branch (DataSourceV2Strategy.scala:544-545) I only traced in code.

Every example on this page is a simple predicate on the target table, so a reader has no way to learn that DELETE FROM t WHERE id IN (SELECT ...) works on one connector and is a planning error on another.

Suggested change

	Specifies an optional condition that selects the rows to delete. If the `WHERE` clause is
	omitted, all rows are deleted.
	Specifies an optional condition that selects the rows to delete. If the `WHERE` clause is
	omitted, all rows are deleted.
	Connectors that support only filter-based deletes place extra restrictions on the condition: it
	must not contain a subquery, and every conjunct must be convertible into a data source
	predicate that the connector accepts -- a connector may, for instance, accept predicates only
	on partition columns. If either does not hold, the statement fails when the query is planned.
	Connectors that support row-level operations have no such restriction.

One aside on the page you link to: sql-v2-data-sources.md:409 says a false from canDeleteWhere makes Spark "fall back to row-level rewriting", which only holds if the table also implements SupportsRowLevelOperations -- a separate fix, not this PR.

[peter-toth]

Finding 3. Each claim here is individually true, but two of them carry a restriction that changes what a reader can write.

DEFAULT is only resolved when it is the whole assignment value and the key is a top-level column: ResolveReferencesInUpdate.scala:49-60 passes defaultReferencesNotAllowedInComplexExpressionsInUpdateSetClause as the error for anything else, so SET salary = DEFAULT + 1 is rejected.

"A nested field may be targeted by using a qualified column name" reads as the alias-qualified e.salary in the example right below, which is a top-level column, not a nested field. And nested targeting only goes through structs -- AssignmentUtils.scala:231 errors with "Updating nested fields is only supported for StructType" for a field inside an array or a map.

Two more rules belong in this bullet, both enforced in AssignmentUtils.scala:175 and :180: a column may be assigned at most once, and a column and one of its nested fields cannot both be assigned. And since RewriteUpdateTable.scala:75 computes every assignment value in one projection over the pre-update row, SET a = b, b = a swaps the two columns rather than setting both to the old b -- exactly the kind of thing a reference page should pin down.

Suggested change

	Specifies the columns to update and the values to assign to them. Each `value` is an expression,
	typically referencing columns of the target table, but it may also be `DEFAULT` or an
	uncorrelated scalar subquery over another table. A comma separates each assignment. A nested
	field may be targeted by using a qualified column name.
	Specifies the columns to update and the values to assign to them. Each `value` is an expression,
	typically referencing columns of the target table, but it may also be `DEFAULT` or an
	uncorrelated scalar subquery over another table. `DEFAULT` must be the whole value; it cannot
	appear inside a larger expression. A comma separates each assignment. Every value is evaluated
	against the row as it was before the update, so `SET a = b, b = a` swaps the two columns. A
	column may be assigned at most once, and a column and one of its nested fields cannot both be
	assigned in the same statement. A field of a struct column may be targeted with a dotted path,
	for example `SET address.city = 'Berlin'`; fields nested inside an array or a map cannot be.

[peter-toth]

Finding 4. RewriteUpdateTable.scala:43 calls checkNoGeneratedColumns(r, UPDATE), which fails the statement with UNSUPPORTED_FEATURE.TABLE_OPERATION ("... does not support UPDATE TABLE with generated columns") when the table has a generated column and declares GENERATE_COLUMN_VALUES_ON_WRITE, i.e. asks Spark to compute the value. The comment above RewriteRowLevelCommand.scala:61 says it's a deliberate fail-fast until recomputation is implemented. Since this sentence is where the page says when UPDATE works, one clause here saves a user the error. MERGE INTO's page has the same gap, so a follow-up covering both is fine if you'd rather keep them consistent.

Suggested change

	`UPDATE` is supported on tables backed by
	[Data Source V2](sql-v2-data-sources.html#row-level-dml) connectors that support row-level
	operations.
	`UPDATE` is supported on tables backed by
	[Data Source V2](sql-v2-data-sources.html#row-level-dml) connectors that support row-level
	operations. Tables with generated columns whose values Spark computes are not supported, because
	Spark cannot yet recompute those values for rewritten rows.