From 44f6374819f675fdc9aa9f7eadbd48542d38c616 Mon Sep 17 00:00:00 2001 From: Serge Rielau Date: Tue, 15 Apr 2025 07:09:27 -0700 Subject: [PATCH 1/4] [SPARK-51765] SQL Scripting --- docs/control-flow/case-stmt.md | 105 ++++++++++++++ docs/control-flow/compound-stmt.md | 169 ++++++++++++++++++++++ docs/control-flow/for-stmt.md | 94 ++++++++++++ docs/control-flow/get-diagnostics-stmt.md | 122 ++++++++++++++++ docs/control-flow/if-stmt.md | 77 ++++++++++ docs/control-flow/iterate-stmt.md | 72 +++++++++ docs/control-flow/leave-stmt.md | 76 ++++++++++ docs/control-flow/loop-stmt.md | 88 +++++++++++ docs/control-flow/repeat-stmt.md | 90 ++++++++++++ docs/control-flow/resignal-stmt.md | 94 ++++++++++++ docs/control-flow/signal-stmt.md | 89 ++++++++++++ docs/control-flow/while-stmt.md | 88 +++++++++++ docs/sql-ref-scripting.md | 91 ++++++++++++ docs/sql-ref-syntax.md | 17 +++ docs/sql-ref.md | 2 + 15 files changed, 1274 insertions(+) create mode 100644 docs/control-flow/case-stmt.md create mode 100644 docs/control-flow/compound-stmt.md create mode 100644 docs/control-flow/for-stmt.md create mode 100644 docs/control-flow/get-diagnostics-stmt.md create mode 100644 docs/control-flow/if-stmt.md create mode 100644 docs/control-flow/iterate-stmt.md create mode 100644 docs/control-flow/leave-stmt.md create mode 100644 docs/control-flow/loop-stmt.md create mode 100644 docs/control-flow/repeat-stmt.md create mode 100644 docs/control-flow/resignal-stmt.md create mode 100644 docs/control-flow/signal-stmt.md create mode 100644 docs/control-flow/while-stmt.md create mode 100644 docs/sql-ref-scripting.md diff --git a/docs/control-flow/case-stmt.md b/docs/control-flow/case-stmt.md new file mode 100644 index 000000000000..52dc6f23a2c4 --- /dev/null +++ b/docs/control-flow/case-stmt.md @@ -0,0 +1,105 @@ +--- +layout: global +title: CASE statement +displayTitle: CASE statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# CASE statement + +Executes `thenStmtN` for the first `optN` that equals `expr` or `elseStmt` if no `optN` matches `expr`. +This is called a _simple case statement_. + +Executes `thenStmtN` for the first `condN` evaluating to `true`, or `elseStmt` if no `condN` evaluates to `true`. +This is called a _searched case statement_. + +For case expressions that yield result values, see [CASE expression](../functions/case.md) + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +CASE expr + { WHEN opt THEN { thenStmt ; } [...] } [...] + [ ELSE { elseStmt ; } [...] ] +END CASE + +CASE + { WHEN cond THEN { thenStmt ; } [...] } [...] + [ ELSE { elseStmt ; } [...] ] +END CASE +``` + +## Parameters + +- **`expr`**: Any expression for which a comparison is defined. +- **`opt`**: An expression with a least common type with `expr` and all other `optN`. +- **`thenStmt`**: A SQL Statement to execute if preceding condition is `true`. +- **`elseStmt`**: A SQL Statement to execute if no condition is `true`. +- **`cond`**: A `BOOLEAN` expression. + +Conditions are evaluated in order, and only the first set of `stmt` for which `opt` or `cond` evaluate to true will be executed. + +## Examples + +```SQL +-- a simple case statement +> BEGIN + DECLARE choice INT DEFAULT 3; + DECLARE result STRING; + CASE choice + WHEN 1 THEN + VALUES ('one fish'); + WHEN 2 THEN + VALUES ('two fish'); + WHEN 3 THEN + VALUES ('red fish'); + WHEN 4 THEN + VALUES ('blue fish'); + ELSE + VALUES ('no fish'); + END CASE; + END; + red fish + +-- A searched case statement +> BEGIN + DECLARE choice DOUBLE DEFAULT 3.9; + DECLARE result STRING; + CASE + WHEN choice < 2 THEN + VALUES ('one fish'); + WHEN choice < 3 THEN + VALUES ('two fish'); + WHEN choice < 4 THEN + VALUES ('red fish'); + WHEN choice < 5 OR choice IS NULL THEN + VALUES ('blue fish'); + ELSE + VALUES ('no fish'); + END CASE; + END; + red fish +``` + +## Related articles + +- [SQL Scripting](../sql-ref-scripting.md) +- [Compound statement](compound-stmt.md) +- [CASE expression](../functions/case.md) +- [IF statement](if-stmt.md) diff --git a/docs/control-flow/compound-stmt.md b/docs/control-flow/compound-stmt.md new file mode 100644 index 000000000000..935675c1d0c3 --- /dev/null +++ b/docs/control-flow/compound-stmt.md @@ -0,0 +1,169 @@ +--- +layout: global +title: compound statement +displayTitle: compound statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# BEGIN END compound statement + +Implements a SQL Script block that can contain a sequence of SQL statements, control-of-flow statements, local variable declarations, and exception handlers. + +## Syntax + +``` +[ label : ] + BEGIN + [ { declare_variable | declare_condition } ; [...] ] + [ declare_handler ; [...] ] + [ SQL_statement ; [...] ] + END [ label ] + +declare_variable + DECLARE variable_name datatype [ DEFAULT default_expr ] + +declare_condition + DECLARE condition_name CONDITION [ FOR SQLSTATE [ VALUE ] sqlstate ] + +declare_handler + DECLARE handler_type HANDLER FOR condition_values handler_action + +handler_type + EXIT + +condition_values + { { SQLSTATE [ VALUE ] sqlstate | condition_name } [, ...] | + { SQLEXCEPTION | NOT FOUND } [, ...] } +``` + +## Parameters + +- **`label`** + + An optional identifier is used to qualify variables defined within the compound and to leave the compound. + Both label occurrences must match, and the `END` label can only be specified if `label:` is specified. + + `label` must not be specified for a top level compound statement. + +- **`NOT ATOMIC`** + + Specifies that, if an SQL statement within the compound fails, previous SQL statements will not be rolled back. + This is the default and only behavior. + +- **`declare_variable`** + + A local variable declaration for one or more variables + + - **`variable_name`** + + A name for the variable. + The name must not be qualified, and be unique within the compound statement. + + - **`data_type`** + + Any supported data type. If data_type is omitted, you must specify DEFAULT, and the type is derived from the default_expression. + + - **`{ DEFAULT | = } default_expression`** + + Defines the variable's initial value after declaration. default_expression must be castable to data_type. If no default is specified, the variable is initialized with NULL. + +- **`Declare_condition`** + + A local condition declaration + + - **`condition_name`** + + The unqualified name of the condition is scoped to the compound statement. + + - **`sqlstate`** + + A `STRING` literal of 5 alphanumeric characters (case insensitive) consisting of A-Z and 0..9. The SQLSTATE must not start with ‘00’, ‘01’, or ‘XX’. Any SQLSTATE starting with ‘02’ will be caught by the predefined NOT FOUND exception as well. If not specified, the SQLSTATE is ‘45000’. + +- **`declare_handler`** + + A declaration for an error handler. + + - **`handler_type`** + + - **`EXIT`** + + Classifies the handler to exit the compound statement after the condition is handled. + + - **`condition_values`** + + Specifies to which sqlstates or conditions the handler applies. + Condition values must be unique within all handlers within the compound statement. + Specific condition values take precedence over `SQLEXCEPTION`. + + - **`sqlstate`** + + A `STRING` literal of 5 characters `'A'-'Z'` and `'0'-'9'` (case insensitive). + + - **`condition_name`** + + A condition defined within this compound, an outer compound statement, or a system-defined error class. + + - **`SQLEXCEPTION`** + + Applies to any user-facing error condition. + + - **`NOT FOUND`** + + Applies to any error condition with a SQLSTATE ‘02’ class. + + - **`handler_action`** + + A SQL statement to execute when any of the condition values occur. + To add multiple statements, use a nested compound statement. + +- **`SQL_statement`** + + A SQL statement such as a DDL, DML, control statement, or compound statement. + Any `SELECT` or `VALUES` statement produces a result set that the invoker can consume. + +## Examples + +```SQL +-- A compound statement with local variables, and exit hanlder and a nested compound. +> BEGIN + DECLARE a INT DEFAULT 1; + DECLARE b INT DEFAULT 5; + DECLARE EXIT HANDLER FOR DIVIDE_BY_ZERO + div0: BEGIN + VALUES (15); + END div0; + SET a = 10; + SET a = b / 0; + VALUES (a); +END; +15 +``` + +## Related articles + +- [SQL Scripting](../sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) +- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) +- [SIGNAL Statement](/sql/language-manual/control-flow/signal-stmt.md) +- [RESIGNAL Statement](/sql/language-manual/control-flow/resignal-stmt.md) +- [GET DIAGNOSTICS Statement](/sql/language-manual/control-flow/get-diagnostics-stmt.md) diff --git a/docs/control-flow/for-stmt.md b/docs/control-flow/for-stmt.md new file mode 100644 index 000000000000..7541847376d6 --- /dev/null +++ b/docs/control-flow/for-stmt.md @@ -0,0 +1,94 @@ +--- +layout: global +title: FOR statement +displayTitle: FOR statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# FOR statement + +Repeat the execution of a list of statements for each row returned by query. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +[ label : ] FOR [ variable_name AS ] query + DO + { stmt ; } [...] + END FOR [ label ] +``` + +## Parameters + +- **[label](/sql/language-manual/sql-ref-names.md#label-name)** + + An optional label for the loop which is unique amongst all labels for statements within which the `FOR` statement is contained. + If an end label is specified, it must match the beginning label. + The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + To qualify loop column references, use the `variable_name`, not the `label`. + +- **[variable_name](/sql/language-manual/sql-ref-names.md#variable-name)** + + An optional name you can use as a qualifier when referencing the columns in the cursor. + +- **`stmt`** + + A SQL statement + +## Notes + +If the query operates on a table that is also modified within the loop's body, the semantics depend on the data source. +For Delta tables, the query will remain unaffected. +Databricks does not guarantee the full execution of the query if the `FOR` loop completes prematurely due to a `LEAVE` statement or an error condition. +When exceptions or side-effects occur during the execution of the query, Databricks does not guarantee at which point in time within the loop these occur. +Often `FOR` loops can be replaced with relational queries, which are typically more efficient. + +## Examples + +```SQL +-- sum up all odd numbers from 1 through 10 +> BEGIN + DECLARE sum INT DEFAULT 0; + sumNumbers: FOR row AS SELECT num FROM range(1, 20) AS t(num) DO + IF num > 10 THEN + LEAVE sumNumbers; + ELSEIF num % 2 = 0 THEN + ITERATE sumNumbers; + END IF; + SET sum = sum + row.num; + END FOR sumNumbers; + VALUES (sum); + END; + 25 + +-- Compare with the much more efficient relational computation: +> SELECT sum(num) FROM range(1, 10) AS t(num) WHERE num % 2 = 1; + 25 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) +- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) diff --git a/docs/control-flow/get-diagnostics-stmt.md b/docs/control-flow/get-diagnostics-stmt.md new file mode 100644 index 000000000000..dd2cda9d0fc6 --- /dev/null +++ b/docs/control-flow/get-diagnostics-stmt.md @@ -0,0 +1,122 @@ +--- +layout: global +title: GET DIAGNOSTICS statement +displayTitle: GET DIAGNOSTICS statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# GET DIAGNOSTICS statement + +Retrieve information about a condition handled in an exception handler. + +This statement may only be used within a condition handler in a [compound statement](compound-stmt.md). + +## Syntax + +``` +GET DIAGNOSTICS CONDITION 1 + { variable_name = condition_info_item } [, ...] + +condition_info_item + { MESSAGE_TEXT | + RETURNED_SQLSTATE | + MESSAGE_ARGUMENTS | + CONDITION_IDENTIFIER | + LINE_NUMBER } +``` + +## Parameters + +- **[variable_name](/sql/language-manual/sql-ref-names.md#variable-name)** + + A local variable or session variable. + +- **`CONDITION 1`** + + Returns the condition that triggered the condition handler. + You must call issue `GET DIAGNOSTICS CONDITION 1` as the first statement in the handler. + + - **`MESSAGE_TEXT`** + + Returns the message text associated with the condition as a `STRING`. + `variable_name` must be a `STRING`. + + - **`RETURNED_SQLSTATE`** + + Returns the `SQLSTATE` associated with the condition being handled as a `STRING`. + `variable_name` must be a `STRING`. + + - **`MESSAGE_ARGUMENTS`** + + Returns a `MAP` mapping provided as arguments to the parameters of Databricks conditions. + For declared conditions, the only map key is `MESSAGE_TEXT`. + `variable_name` must be a `MAP` + + - **`CONDITION_IDENTIFIER`** + + Returns the condition name that caused the exception. + `variable_name` must be a `STRING`. + + - **`LINE_NUMBER`** + + Returns the line number of the statement raising the condition. + `NULL` if not available. + +## Examples + +```SQL +-- Retrieve the number of rows inserted by an INSERT statement +> CREATE OR REPLACE TABLE emp(name STRING, salary DECIMAL(10, 2)); + +> BEGIN + DECLARE EXIT HANDLER FOR DIVIDE_BY_ZERO + BEGIN + DECLARE cond STRING; + DECLARE message STRING; + DECLARE state STRING; + DECLARE args MAP; + DECLARE line BIGINT; + DECLARE argstr STRING; + DECLARE log STRING; + GET DIAGNOSTICS CONDITION 1 + cond = CONDITION_IDENTIFIER, + message = MESSAGE_TEXT, + state = RETURNED_SQLSTATE, + args = MESSAGE_ARGUMENTS, + line = LINE_NUMBER; + SET argstr = + (SELECT aggregate(array_agg('Parm:' || key || ' Val: value '), + '', (acc, x)->(acc || ' ' || x)) + FROM explode(args) AS args(key, val)); + SET log = 'Condition: ' || cond || + ' Message: ' || message || + ' SQLSTATE: ' || state || + ' Args: ' || argstr || + ' Line: ' || line; + VALUES (log); + END; + SELECT 10/0; + END; + Condition: DIVIDE_BY_ZERO Message: Division by zero. Use try_divide to tolerate divisor being 0 and return NULL instead. If necessary, set to “false” to bypass this error. SQLATTE: 22012 Args: Parm: config Val: ANSI_MODE Line: 28 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) diff --git a/docs/control-flow/if-stmt.md b/docs/control-flow/if-stmt.md new file mode 100644 index 000000000000..79e29c7df38f --- /dev/null +++ b/docs/control-flow/if-stmt.md @@ -0,0 +1,77 @@ +--- +layout: global +title: IF statement +displayTitle: IF statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# IF THEN ELSE statement + +Executes lists of statements based on the first condition that evaluates to true. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +IF condition THEN { stmt ; } [...] + [ { ELSEIF condition THEN { stmt ; } [...] } [...] ] + [ ELSE { stmt ; } [...] ] + END IF +``` + +## Parameters + +- **`condition`** + + Any expression evaluating to a BOOLEAN. + +- **`stmt`** + + A SQL statement to execute if the `condition` is `true`. + +## Examples + +```SQL +> BEGIN + DECLARE choice DOUBLE DEFAULT 3.9; + DECLARE result STRING; + IF choice < 2 THEN + VALUES ('one fish'); + ELSEIF choice < 3 THEN + VALUES ('two fish'); + ELSEIF choice < 4 THEN + VALUES ('red fish'); + ELSEIF choice < 5 OR choice IS NULL THEN + VALUES ('blue fish'); + ELSE + VALUES ('no fish'); + END IF; + END; + red fish +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) + +``` + +``` diff --git a/docs/control-flow/iterate-stmt.md b/docs/control-flow/iterate-stmt.md new file mode 100644 index 000000000000..1466fd3a69f9 --- /dev/null +++ b/docs/control-flow/iterate-stmt.md @@ -0,0 +1,72 @@ +--- +layout: global +title: ITERATE statement +displayTitle: ITERATE statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# ITERATE statement + +Terminates the execution of an iteration of a looping statement and continues with the next iteration if the looping condition is met. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +ITERATE label +``` + +## Parameters + +- **[label](/sql/language-manual/sql-ref-names.md#label-name)** + + The label identifies a looping statement that contains the `ITERATE` statement directly or indirectly. + +## Examples + +```SQL +-- sum up all odd numbers from 1 through 10 +> BEGIN + DECLARE sum INT DEFAULT 0; + DECLARE num INT DEFAULT 0; + sumNumbers: LOOP + SET num = num + 1; + IF num > 10 THEN + LEAVE sumNumbers; + END IF; + IF num % 2 = 0 THEN + ITERATE sumNumbers; + END IF; + SET sum = sum + num; + END LOOP sumNumbers; + VALUES (sum); + END; +25 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) +- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) diff --git a/docs/control-flow/leave-stmt.md b/docs/control-flow/leave-stmt.md new file mode 100644 index 000000000000..b33a90e8f47a --- /dev/null +++ b/docs/control-flow/leave-stmt.md @@ -0,0 +1,76 @@ +--- +layout: global +title: LEAVE statement +displayTitle: LEAVE statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# LEAVE statement + +Terminates the execution of an iteration of a looping statement and continues with the next iteration if the looping condition is met. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +ITERATE label +``` + +## Parameters + +- **[label](/sql/language-manual/sql-ref-names.md#label-name)** + + The label identifies a statement to leave that directly or indirectly contains the `LEAVE` statement. + +## Examples + +```SQL +-- sum up all odd numbers from 1 through 10 +-- Iterate over even numbers and leave the loop after 10 has been reached. +> BEGIN + DECLARE sum INT DEFAULT 0; + DECLARE num INT DEFAULT 0; + sumNumbers: LOOP + SET num = num + 1; + IF num > 10 THEN + LEAVE sumNumbers; + END IF; + IF num % 2 = 0 THEN + ITERATE sumNumbers; + END IF; + SET sum = sum + num; + END LOOP sumNumbers; + VALUES (sum); + END; +25 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) +- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) + +``` + +``` diff --git a/docs/control-flow/loop-stmt.md b/docs/control-flow/loop-stmt.md new file mode 100644 index 000000000000..b81a10108afd --- /dev/null +++ b/docs/control-flow/loop-stmt.md @@ -0,0 +1,88 @@ +--- +layout: global +title: LOOP statement +displayTitle: LOOP statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# LOOP statement + +Repeat the execution of a list of statements. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +[ label : ] LOOP + { stmt ; } [...] + END LOOP [ label ] +``` + +## Parameters + +- **[label](/sql/language-manual/sql-ref-names.md#label-name)** + + An optional label for the loop, which is unique amongst all labels for statements within which the `LOOP` statement is contained. + If an end label is specified, it must match the beginning label. + The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + +- **`stmt`** + + A SQL statement + +## Examples + +```SQL +-- sum up all odd numbers from 1 through 10 +> BEGIN + DECLARE sum INT DEFAULT 0; + DECLARE num INT DEFAULT 0; + sumNumbers: LOOP + SET num = num + 1; + IF num > 10 THEN + LEAVE sumNumbers; + END IF; + IF num % 2 = 0 THEN + ITERATE sumNumbers; + END IF; + SET sum = sum + num; + END LOOP sumNumbers; + VALUES (sum); + END; + 25 + +-- Compare with the much more efficient relational computation: +> SELECT sum(num) FROM range(1, 10) AS t(num) WHERE num % 2 = 1; + 25 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) + +``` + +``` diff --git a/docs/control-flow/repeat-stmt.md b/docs/control-flow/repeat-stmt.md new file mode 100644 index 000000000000..e60a8331a59d --- /dev/null +++ b/docs/control-flow/repeat-stmt.md @@ -0,0 +1,90 @@ +--- +layout: global +title: REPEAT statement +displayTitle: REPEAT statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# REPEAT statement + +Repeat the execution of a list of statements until a condition is true. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +[ label : ] REPEAT + { stmt ; } [...] + UNTIL cond + END REPEAT [ label ] +``` + +## Parameters + +- **[label](/sql/language-manual/sql-ref-names.md#label-name)** + + An optional label for the loop, which is unique amongst all labels for statements within which the `REPEAT` statement is contained. + The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + +- **`cond`** + + Any expression evaluating to a BOOLEAN + +- **`stmt`** + + A SQL statement + +## Examples + +```SQL +-- sum up all odd numbers from 1 through 10 +> BEGIN + DECLARE sum INT DEFAULT 0; + DECLARE num INT DEFAULT 0; + sumNumbers: REPEAT + SET num = num + 1; + IF num % 2 = 0 THEN + ITERATE sumNumbers; + END IF; + SET sum = sum + num; + UNTIL num = 10 + END REPEAT sumNumbers; + VALUES (sum); + END; + 25 + +-- Compare with the much more efficient relational computation: +> SELECT sum(num) FROM range(1, 10) AS t(num) WHERE num % 2 = 1; + 25 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) +- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) +- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) + +``` + +``` diff --git a/docs/control-flow/resignal-stmt.md b/docs/control-flow/resignal-stmt.md new file mode 100644 index 000000000000..30e7d2e9b616 --- /dev/null +++ b/docs/control-flow/resignal-stmt.md @@ -0,0 +1,94 @@ +--- +layout: global +title: RESIGNAL statement +displayTitle: RESIGNAL statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# RESIGNAL statement + +Re-raises the condition handled by the condition handler. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +RESIGNAL +``` + +## Parameters + +None + +## Examples + +```SQL +> CREATE TABLE log(eventtime TIMESTAMP, log STRING); + +> BEGIN + DECLARE EXIT HANDLER FOR DIVIDE_BY_ZERO + BEGIN + DECLARE cond STRING; + DECLARE message STRING; + DECLARE state STRING; + DECLARE args MAP; + DECLARE line BIGINT; + DECLARE argstr STRING; + DECLARE log STRING; + GET DIAGNOSTICS CONDITION 1 + cond = CONDITION_IDENTIFIER, + message = MESSAGE_TEXT, + state = RETURNED_SQLSTATE, + args = MESSAGE_ARGUMENTS, + line = LINE_NUMBER; + SET argstr = + (SELECT aggregate(array_agg('Parm:' || key || ' Val: value '), + '', (acc, x)->(acc || ' ' || x)) + FROM explode(args) AS args(key, val)); + SET log = 'Condition: ' || cond || + ' Message: ' || message || + ' SQLSTATE: ' || state || + ' Args: ' || argstr || + ' Line: ' || line; + INSERT INTO log VALUES(current_timestamp(), log); + RESIGNAL; + END; + SELECT 10/0; + END; + [DIVIDE_BY_ZERO] Division by zero. Use try_divide to tolerate divisor being 0 and return NULL instead. + +> SELECT * FROM log ORDER BY eventtime DESC LIMIT 1; + Condition: DIVIDE_BY_ZERO Message: Division by zero. Use try_divide to tolerate divisor being 0 and return NULL instead. SQLSTATE: 22012 Args: Line: 28 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [SIGNAL Statement](/sql/language-manual/control-flow/signal-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [SIGNAL Statement](/sql/language-manual/control-flow/signal-stmt.md) +- [Error handling and error messages](/error-messages/index.md) + +``` + +``` diff --git a/docs/control-flow/signal-stmt.md b/docs/control-flow/signal-stmt.md new file mode 100644 index 000000000000..2b2119983a44 --- /dev/null +++ b/docs/control-flow/signal-stmt.md @@ -0,0 +1,89 @@ +--- +layout: global +title: SIGNAL statement +displayTitle: SIGNAL statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# SIGNAL statement + +Raises a condition. + +This statement may only be used within a [compound statement](compound-stmt.md). + +Note: Databricks recommends using [RESIGNAL](resignal-stmt.md) to raise conditions from within a handler. +RESIGNAL builds a diagnostic stack in the SQL Standard, while `SIGNAL` clears the stack. +Using `RESIGNAL` within a handler preserves future exploitation of the diagnostic stack. + +## Syntax + +``` +SIGNAL { condition_name + [ SET { MESSAGE_ARGUMENTS = argument_map | + MESSAGE_TEXT = message_str } ] | + SQLSTATE [VALUE] sqlstate [ SET MESSAGE_TEXT = message_str ] } +``` + +## Parameters + +- **[condition_name](/sql/language-manual/sql-ref-names.md#condition-name)** + + The name of a locally defined condition or system-defined error condition. + +- **`argument_map`** + + Optionally, a `MAP` literal that assigns values to a system-defined parameterized condition message. + +- **`message_str`** + + Optionally, a `STRING` literal that provides a message string to the raised `SQLSTATE` or user-defined condition. + +- **`sqlstate`** + + A `STRING` literal of length 5. If specified, raise `USER_RAISED_EXCEPTION` with the specified `SQLSTATE`. + +## Examples + +```SQL +> DECLARE input INT DEFAULT 5; + +> BEGIN + DECLARE arg_map MAP; + IF input > 4 THEN + SET arg_map = map('errorMessage', + 'Input must be <= 4.'); + SIGNAL USER_RAISED_EXCEPTION + SET MESSAGE_ARGUMENTS = arg_map; + END IF; + END; +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [RESIGNAL Statement](/sql/language-manual/control-flow/resignal-stmt.md) +- [Error handling and error messages](/error-messages/index.md) + +``` + +``` diff --git a/docs/control-flow/while-stmt.md b/docs/control-flow/while-stmt.md new file mode 100644 index 000000000000..651f7a8bc57b --- /dev/null +++ b/docs/control-flow/while-stmt.md @@ -0,0 +1,88 @@ +--- +layout: global +title: WHILE statement +displayTitle: WHILE statement +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# WHILE statement + +Repeat the execution of a list of statements while a condition is true. + +This statement may only be used within a [compound statement](compound-stmt.md). + +## Syntax + +``` +[ label : ] WHILE cond DO + { stmt ; } [...] + END WHILE [ label ] +``` + +## Parameters + +- **[label](/sql/language-manual/sql-ref-names.md#label-name)** + + An optional label for the loop, which is unique amongst all labels for statements within which the `WHILE` statement is contained. + The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + +- **`cond`** + + Any expression evaluating to a `BOOLEAN` + +- **`stmt`** + + A SQL statement + +## Examples + +```SQL +-- sum up all odd numbers from 1 through 10 +> BEGIN + DECLARE sum INT DEFAULT 0; + DECLARE num INT DEFAULT 0; + sumNumbers: WHILE num < 10 DO + SET num = num + 1; + IF num % 2 = 0 THEN + ITERATE sumNumbers; + END IF; + SET sum = sum + num; + END WHILE sumNumbers; + VALUES (sum); + END; + 25 + +-- Compare with the much more efficient relational computation: +> SELECT sum(num) FROM range(1, 10) AS t(num) WHERE num % 2 = 1; + 25 +``` + +## Related articles + +- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) +- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) +- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) +- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) +- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) +- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) +- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) + +``` + +``` diff --git a/docs/sql-ref-scripting.md b/docs/sql-ref-scripting.md new file mode 100644 index 000000000000..abc0b1854328 --- /dev/null +++ b/docs/sql-ref-scripting.md @@ -0,0 +1,91 @@ +--- +layout: global +title: SQL Syntax +displayTitle: SQL Syntax +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +# SQL scripting + +You can employ powerful procedural logic using SQL/PSM standard-based scripting syntax. +Any SQL script consists of and starts with a [compound statement](control-flow/compound-stmt.md) block (`BEGIN ... END`). +A compound statement starts with a section to declare local variables, user-defined conditions, and condition handlers, which are used to catch exceptions. +This is followed by the compound statement body, which consists of: + +- Flow control statements include loops over predicate expressions, [FOR](control-flow/for-stmt.md) loops over query results, conditional logic such as [IF](control-flow/if-stmt.md) and [CASE](control-flow/case-stmt.md), and means to break out loops such as [LEAVE](control-flow/leave-stmt.md) and [ITERATE](control-flow/iterate-stmt.md). +- DDL statements such as `ALTER`, `CREATE`, `DROP`. +- DML statements [INSERT](sql-ref-syntax-dml-insert-into.md), [UPDATE](delta-update.md), [DELETE](delta-delete-from.md), and [MERGE](delta-merge-into.md). +- [Queries](sql-ref-syntax-qry-query.md) that return result sets to the invoker of the script. +- [SET](sql-ref-syntax-aux-set-variable.md) statements to set local variables as well as session variables. +- The [EXECUTE IMMEDIATE](sql-ref-syntax-aux-execute-immediate.md) statement. +- Nested compound statements, which provide nested scopes for variables, conditions, and condition handlers. + +## Passing data between the invoker and the compound statement + +There are two ways to pass data to and from a SQL script: + +- Use session variables to pass scalar values or small sets of arrays or maps from one SQL script to another. +- Use parameter markers to pass scalar values or small sets of arrays or map data from a notebook widget, Python, or another language to the SQL Script. + +## Variable scoping + +Variables declared within a compound statement can be referenced in any expression within a compound statement. +Spark resolves identifiers from the innermost scope outward, following the rules described in [Name Resolution](sql-ref-name-resolution.md). +You can use the optional compound statement [labels](sql-ref-names.md#label-name) to disambiguate duplicate [variable names](sql-ref-names.md#variable-name). + +## Condition handling + +SQL Scripting supports condition handlers, which are used to intercept and process exceptions to `EXIT` processing of the SQL script. +Within the condition handler, you can [RESIGNAL](control-flow/resignal-stmt.md) the original exception, [SIGNAL](control-flow/signal-stmt.md) a new exception, or exit the compound statement without an exception. + +Condition handlers can be defined to handle three distinct classes of conditions: + +- One or more named conditions that can be a specific Spark-defined error class such as `DIVIDE_BY_ZERO` or a user-declared condition. + These handlers handle these specific conditions. + +- One or more `SQLSTATE`s, that can be raised by Spark or a user `SIGNAL` statement. + These handlers can handle any condition associated with that `SQLSTATE`. + +- A generic `SQLEXCEPTION` handler can catch all conditions falling into the `SQLEXCEPTION` (any `SQLSTATE` which is not `XX***` and not `02***`). + +The following are used to decide which condition handler applies to an exception. +This condition handler is called the **most appropriate handler**: + +- A condition handler cannot apply to any statement defined in its own body or the body of any condition handler declared in the same compound statement. + +- The applicable condition handlers defined in the innermost compound statement within which the exception was raised are appropriate. + +- If more than one appropriate handler is available, the most specific handler is the most appropriate. + For example, a handler on a named condition is more specific than one on a named `SQLSTATE`. + A generic `EXCEPTION` handler is the least specific. + +Unless a handler `SIGNAL`s or `RESIGNAL`s a condition of its own, the outcome of a condition handler is to execute the statement following the compound statement that declared the handler to execute next. + +The following is a list of supported control flow statement: + +* [CASE](control-flow/case-stmt.md) +* [compound statement](control-flow/compound-stmt.md) +* [FOR](control-flow/for-stmt.md) +* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.md) +* [IF](control-flow/if-stmt.md) +* [ITERATE](control-flow/iterate-stmt.md) +* [LEAVE](control-flow/leave-stmt.md) +* [LOOP](control-flow/loop-stmt.md) +* [REPEAT](control-flow/repeat-stmt.md) +* [RESIGNAL](control-flow/resignal-stmt.md) +* [SIGNAL](control-flow/signal-stmt.md) +* [WHILE](control-flow/while-stmt.md) diff --git a/docs/sql-ref-syntax.md b/docs/sql-ref-syntax.md index 3dc7d47c4f45..5a67f8b04bc5 100644 --- a/docs/sql-ref-syntax.md +++ b/docs/sql-ref-syntax.md @@ -90,6 +90,23 @@ ability to generate logical and physical plan for a given query using * [star (*) Clause](sql-ref-syntax-qry-star.html) * [EXPLAIN](sql-ref-syntax-qry-explain.html) +### SQL Scripting Statements + +You use SQL scripting to execute procedural logic in SQL. + +* [CASE](control-flow/case-stmt.md) +* [compound statement](control-flow/compound-stmt.md) +* [FOR](control-flow/for-stmt.md) +* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.md) +* [IF](control-flow/if-stmt.md) +* [ITERATE](control-flow/iterate-stmt.md) +* [LEAVE](control-flow/leave-stmt.md) +* [LOOP](control-flow/loop-stmt.md) +* [REPEAT](control-flow/repeat-stmt.md) +* [RESIGNAL](control-flow/resignal-stmt.md) +* [SIGNAL](control-flow/signal-stmt.md) +* [WHILE](control-flow/while-stmt.md) + ### Auxiliary Statements * [ADD FILE](sql-ref-syntax-aux-resource-mgmt-add-file.html) diff --git a/docs/sql-ref.md b/docs/sql-ref.md index 6d557caaca3d..95933eb0efb4 100644 --- a/docs/sql-ref.md +++ b/docs/sql-ref.md @@ -37,9 +37,11 @@ Spark SQL is Apache Spark's module for working with structured data. This guide * [IDENTIFIER clause](sql-ref-identifier-clause.html) * [Literals](sql-ref-literals.html) * [Null Semantics](sql-ref-null-semantics.html) + * [SQL Scripting](sql-ref-scripting.html) * [SQL Syntax](sql-ref-syntax.html) * [DDL Statements](sql-ref-syntax.html#ddl-statements) * [DML Statements](sql-ref-syntax.html#dml-statements) * [Data Retrieval Statements](sql-ref-syntax.html#data-retrieval-statements) + * [SQL Scripting](sql-ref-syntax.html#sql-scripting) * [Auxiliary Statements](sql-ref-syntax.html#auxiliary-statements) * [Pipe Syntax](sql-pipe-syntax.html) From e961518f3b096f9c3be605e50ca3836d8dd743c2 Mon Sep 17 00:00:00 2001 From: Serge Rielau Date: Wed, 16 Apr 2025 11:22:58 -0700 Subject: [PATCH 2/4] CLean up --- docs/_data/menu-sql.yaml | 6 + docs/control-flow/case-stmt.md | 13 +- docs/control-flow/compound-stmt.md | 26 +- docs/control-flow/for-stmt.md | 32 +- docs/control-flow/get-diagnostics-stmt.md | 16 +- docs/control-flow/if-stmt.md | 20 +- docs/control-flow/iterate-stmt.md | 24 +- docs/control-flow/leave-stmt.md | 25 +- docs/control-flow/loop-stmt.md | 31 +- docs/control-flow/repeat-stmt.md | 34 +- docs/control-flow/resignal-stmt.md | 28 +- docs/control-flow/signal-stmt.md | 36 +- docs/control-flow/while-stmt.md | 37 +- docs/sql-ref-name-resolution.md | 423 ++++++++++++++++++++++ docs/sql-ref-scripting.md | 52 ++- docs/sql-ref-syntax-qry-select-tvf.md | 2 +- docs/sql-ref-syntax.md | 24 +- docs/sql-ref.md | 3 +- 18 files changed, 605 insertions(+), 227 deletions(-) create mode 100644 docs/sql-ref-name-resolution.md diff --git a/docs/_data/menu-sql.yaml b/docs/_data/menu-sql.yaml index b1688aec57f0..f530d78dae45 100644 --- a/docs/_data/menu-sql.yaml +++ b/docs/_data/menu-sql.yaml @@ -99,6 +99,10 @@ url: sql-ref-literals.html - text: Null Semantics url: sql-ref-null-semantics.html + - text: Name Resolution + url: sql-ref-name-resolution.html + - text: SQL Scripting + url: sql-ref-scripting.html - text: SQL Syntax url: sql-ref-syntax.html subitems: @@ -108,6 +112,8 @@ url: sql-ref-syntax.html#dml-statements - text: Data Retrieval(Queries) url: sql-ref-syntax.html#data-retrieval-statements + - text: SQL Scripting Statements + url: sql-ref-syntax.html#sql-scripting-statements - text: Auxiliary Statements url: sql-ref-syntax.html#auxiliary-statements - text: Pipe Syntax diff --git a/docs/control-flow/case-stmt.md b/docs/control-flow/case-stmt.md index 52dc6f23a2c4..c92663905b06 100644 --- a/docs/control-flow/case-stmt.md +++ b/docs/control-flow/case-stmt.md @@ -19,17 +19,15 @@ license: | limitations under the License. --- -# CASE statement - Executes `thenStmtN` for the first `optN` that equals `expr` or `elseStmt` if no `optN` matches `expr`. This is called a _simple case statement_. Executes `thenStmtN` for the first `condN` evaluating to `true`, or `elseStmt` if no `condN` evaluates to `true`. This is called a _searched case statement_. -For case expressions that yield result values, see [CASE expression](../functions/case.md) +For case expressions that yield result values, see `CASE expression`) -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -99,7 +97,6 @@ Conditions are evaluated in order, and only the first set of `stmt` for which `o ## Related articles -- [SQL Scripting](../sql-ref-scripting.md) -- [Compound statement](compound-stmt.md) -- [CASE expression](../functions/case.md) -- [IF statement](if-stmt.md) +- [SQL Scripting](../sql-ref-scripting.html) +- [compound statement](compound-stmt.html) +- [IF statement](if-stmt.html) diff --git a/docs/control-flow/compound-stmt.md b/docs/control-flow/compound-stmt.md index 935675c1d0c3..82beade5cb53 100644 --- a/docs/control-flow/compound-stmt.md +++ b/docs/control-flow/compound-stmt.md @@ -19,8 +19,6 @@ license: | limitations under the License. --- -# BEGIN END compound statement - Implements a SQL Script block that can contain a sequence of SQL statements, control-of-flow statements, local variable declarations, and exception handlers. ## Syntax @@ -155,15 +153,15 @@ END; ## Related articles -- [SQL Scripting](../sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) -- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) -- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) -- [SIGNAL Statement](/sql/language-manual/control-flow/signal-stmt.md) -- [RESIGNAL Statement](/sql/language-manual/control-flow/resignal-stmt.md) -- [GET DIAGNOSTICS Statement](/sql/language-manual/control-flow/get-diagnostics-stmt.md) +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [LOOP Statement](../control-flow/loop-stmt.html) +- [WHILE Statement](../control-flow/while-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) +- [LEAVE Statement](../control-flow/leave-stmt.html) +- [SIGNAL Statement](../control-flow/signal-stmt.html) +- [RESIGNAL Statement](../control-flow/resignal-stmt.html) +- [GET DIAGNOSTICS Statement](../control-flow/get-diagnostics-stmt.html) diff --git a/docs/control-flow/for-stmt.md b/docs/control-flow/for-stmt.md index 7541847376d6..25a1cfa7218e 100644 --- a/docs/control-flow/for-stmt.md +++ b/docs/control-flow/for-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# FOR statement - Repeat the execution of a list of statements for each row returned by query. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -36,18 +34,18 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Parameters -- **[label](/sql/language-manual/sql-ref-names.md#label-name)** +- **label** An optional label for the loop which is unique amongst all labels for statements within which the `FOR` statement is contained. If an end label is specified, it must match the beginning label. - The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + The label can be used to [LEAVE](leave-stmt.html) or [ITERATE](iterate-stmt.html) the loop. To qualify loop column references, use the `variable_name`, not the `label`. -- **[variable_name](/sql/language-manual/sql-ref-names.md#variable-name)** +- **variable_name** An optional name you can use as a qualifier when referencing the columns in the cursor. -- **`stmt`** +- **stmt** A SQL statement @@ -55,8 +53,8 @@ This statement may only be used within a [compound statement](compound-stmt.md). If the query operates on a table that is also modified within the loop's body, the semantics depend on the data source. For Delta tables, the query will remain unaffected. -Databricks does not guarantee the full execution of the query if the `FOR` loop completes prematurely due to a `LEAVE` statement or an error condition. -When exceptions or side-effects occur during the execution of the query, Databricks does not guarantee at which point in time within the loop these occur. +Spark does not guarantee the full execution of the query if the `FOR` loop completes prematurely due to a `LEAVE` statement or an error condition. +When exceptions or side-effects occur during the execution of the query, Spark does not guarantee at which point in time within the loop these occur. Often `FOR` loops can be replaced with relational queries, which are typically more efficient. ## Examples @@ -84,11 +82,11 @@ Often `FOR` loops can be replaced with relational queries, which are typically m ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) -- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [LOOP Statement](../control-flow/loop-stmt.html) +- [WHILE Statement](../control-flow/while-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [LEAVE Statement](../control-flow/leave-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) diff --git a/docs/control-flow/get-diagnostics-stmt.md b/docs/control-flow/get-diagnostics-stmt.md index dd2cda9d0fc6..ecf61c7c49f3 100644 --- a/docs/control-flow/get-diagnostics-stmt.md +++ b/docs/control-flow/get-diagnostics-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# GET DIAGNOSTICS statement - Retrieve information about a condition handled in an exception handler. -This statement may only be used within a condition handler in a [compound statement](compound-stmt.md). +This statement may only be used within a condition handler in a [compound statement](compound-stmt.html). ## Syntax @@ -41,7 +39,7 @@ condition_info_item ## Parameters -- **[variable_name](/sql/language-manual/sql-ref-names.md#variable-name)** +- **variable_name** A local variable or session variable. @@ -62,7 +60,7 @@ condition_info_item - **`MESSAGE_ARGUMENTS`** - Returns a `MAP` mapping provided as arguments to the parameters of Databricks conditions. + Returns a `MAP` mapping provided as arguments to the parameters of Spark conditions. For declared conditions, the only map key is `MESSAGE_TEXT`. `variable_name` must be a `MAP` @@ -116,7 +114,7 @@ condition_info_item ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) diff --git a/docs/control-flow/if-stmt.md b/docs/control-flow/if-stmt.md index 79e29c7df38f..2e93dc7f6d50 100644 --- a/docs/control-flow/if-stmt.md +++ b/docs/control-flow/if-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# IF THEN ELSE statement - Executes lists of statements based on the first condition that evaluates to true. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -36,11 +34,11 @@ IF condition THEN { stmt ; } [...] ## Parameters -- **`condition`** +- **condition** Any expression evaluating to a BOOLEAN. -- **`stmt`** +- **stmt** A SQL statement to execute if the `condition` is `true`. @@ -67,11 +65,7 @@ IF condition THEN { stmt ; } [...] ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) - -``` - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) diff --git a/docs/control-flow/iterate-stmt.md b/docs/control-flow/iterate-stmt.md index 1466fd3a69f9..d73f33a26bf9 100644 --- a/docs/control-flow/iterate-stmt.md +++ b/docs/control-flow/iterate-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# ITERATE statement - Terminates the execution of an iteration of a looping statement and continues with the next iteration if the looping condition is met. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -33,7 +31,7 @@ ITERATE label ## Parameters -- **[label](/sql/language-manual/sql-ref-names.md#label-name)** +- **label** The label identifies a looping statement that contains the `ITERATE` statement directly or indirectly. @@ -61,12 +59,12 @@ ITERATE label ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) -- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [LOOP Statement](../control-flow/loop-stmt.html) +- [WHILE Statement](../control-flow/while-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [LEAVE Statement](../control-flow/leave-stmt.html) diff --git a/docs/control-flow/leave-stmt.md b/docs/control-flow/leave-stmt.md index b33a90e8f47a..a705c48b239a 100644 --- a/docs/control-flow/leave-stmt.md +++ b/docs/control-flow/leave-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# LEAVE statement - Terminates the execution of an iteration of a looping statement and continues with the next iteration if the looping condition is met. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -33,7 +31,7 @@ ITERATE label ## Parameters -- **[label](/sql/language-manual/sql-ref-names.md#label-name)** +- **label** The label identifies a statement to leave that directly or indirectly contains the `LEAVE` statement. @@ -62,15 +60,12 @@ ITERATE label ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) -- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [LOOP Statement](../control-flow/loop-stmt.html) +- [WHILE Statement](../control-flow/while-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) -``` diff --git a/docs/control-flow/loop-stmt.md b/docs/control-flow/loop-stmt.md index b81a10108afd..7ca3b3b5bbf9 100644 --- a/docs/control-flow/loop-stmt.md +++ b/docs/control-flow/loop-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# LOOP statement - Repeat the execution of a list of statements. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -35,13 +33,13 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Parameters -- **[label](/sql/language-manual/sql-ref-names.md#label-name)** +- **label** An optional label for the loop, which is unique amongst all labels for statements within which the `LOOP` statement is contained. If an end label is specified, it must match the beginning label. - The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + The label can be used to [LEAVE](leave-stmt.html) or [ITERATE](iterate-stmt.html) the loop. -- **`stmt`** +- **stmt** A SQL statement @@ -73,16 +71,13 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) -- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [WHILE Statement](../control-flow/while-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) +- [LEAVE Statement](../control-flow/leave-stmt.html) -``` diff --git a/docs/control-flow/repeat-stmt.md b/docs/control-flow/repeat-stmt.md index e60a8331a59d..4d28a6b05e0e 100644 --- a/docs/control-flow/repeat-stmt.md +++ b/docs/control-flow/repeat-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# REPEAT statement - Repeat the execution of a list of statements until a condition is true. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -36,16 +34,16 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Parameters -- **[label](/sql/language-manual/sql-ref-names.md#label-name)** +- **label** An optional label for the loop, which is unique amongst all labels for statements within which the `REPEAT` statement is contained. - The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + The label can be used to [LEAVE](leave-stmt.html) or [ITERATE](iterate-stmt.html) the loop. -- **`cond`** +- **cond** Any expression evaluating to a BOOLEAN -- **`stmt`** +- **stmt** A SQL statement @@ -75,16 +73,12 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) -- [WHILE Statement](/sql/language-manual/control-flow/while-stmt.md) -- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) -- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) - -``` - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) +- [WHILE Statement](../control-flow/while-stmt.html) +- [LEAVE Statement](../control-flow/leave-stmt.html) +- [LOOP Statement](../control-flow/loop-stmt.html) diff --git a/docs/control-flow/resignal-stmt.md b/docs/control-flow/resignal-stmt.md index 30e7d2e9b616..e2b24221e977 100644 --- a/docs/control-flow/resignal-stmt.md +++ b/docs/control-flow/resignal-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# RESIGNAL statement - Re-raises the condition handled by the condition handler. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -78,17 +76,13 @@ None ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [SIGNAL Statement](/sql/language-manual/control-flow/signal-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [SIGNAL Statement](/sql/language-manual/control-flow/signal-stmt.md) -- [Error handling and error messages](/error-messages/index.md) - -``` - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [SIGNAL Statement](../control-flow/signal-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [SIGNAL Statement](../control-flow/signal-stmt.html) +- [Error handling and error messages](/error-messages/index.html) diff --git a/docs/control-flow/signal-stmt.md b/docs/control-flow/signal-stmt.md index 2b2119983a44..e74727c4014d 100644 --- a/docs/control-flow/signal-stmt.md +++ b/docs/control-flow/signal-stmt.md @@ -19,13 +19,11 @@ license: | limitations under the License. --- -# SIGNAL statement - Raises a condition. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). -Note: Databricks recommends using [RESIGNAL](resignal-stmt.md) to raise conditions from within a handler. +Note: Spark recommends using [RESIGNAL](resignal-stmt.html) to raise conditions from within a handler. RESIGNAL builds a diagnostic stack in the SQL Standard, while `SIGNAL` clears the stack. Using `RESIGNAL` within a handler preserves future exploitation of the diagnostic stack. @@ -40,19 +38,19 @@ SIGNAL { condition_name ## Parameters -- **[condition_name](/sql/language-manual/sql-ref-names.md#condition-name)** +- **condition_name** The name of a locally defined condition or system-defined error condition. -- **`argument_map`** +- **argument_map** Optionally, a `MAP` literal that assigns values to a system-defined parameterized condition message. -- **`message_str`** +- **message_str** Optionally, a `STRING` literal that provides a message string to the raised `SQLSTATE` or user-defined condition. -- **`sqlstate`** +- **sqlstate** A `STRING` literal of length 5. If specified, raise `USER_RAISED_EXCEPTION` with the specified `SQLSTATE`. @@ -74,16 +72,12 @@ SIGNAL { condition_name ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [RESIGNAL Statement](/sql/language-manual/control-flow/resignal-stmt.md) -- [Error handling and error messages](/error-messages/index.md) - -``` - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [RESIGNAL Statement](../control-flow/resignal-stmt.html) +- [Error handling and error messages](/error-messages/index.html) diff --git a/docs/control-flow/while-stmt.md b/docs/control-flow/while-stmt.md index 651f7a8bc57b..0edf77f0cba0 100644 --- a/docs/control-flow/while-stmt.md +++ b/docs/control-flow/while-stmt.md @@ -19,11 +19,9 @@ license: | limitations under the License. --- -# WHILE statement - Repeat the execution of a list of statements while a condition is true. -This statement may only be used within a [compound statement](compound-stmt.md). +This statement may only be used within a [compound statement](compound-stmt.html). ## Syntax @@ -35,18 +33,18 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Parameters -- **[label](/sql/language-manual/sql-ref-names.md#label-name)** +- **label** An optional label for the loop, which is unique amongst all labels for statements within which the `WHILE` statement is contained. - The label can be used to [LEAVE](leave-stmt.md) or [ITERATE](iterate-stmt.md) the loop. + The label can be used to [LEAVE](leave-stmt.html) or [ITERATE](iterate-stmt.html) the loop. -- **`cond`** +- **cond** - Any expression evaluating to a `BOOLEAN` + Any expression evaluating to a `BOOLEAN`. -- **`stmt`** +- **stmt** - A SQL statement + A SQL statement. ## Examples @@ -73,16 +71,13 @@ This statement may only be used within a [compound statement](compound-stmt.md). ## Related articles -- [SQL Scripting](/sql/language-manual/sql-ref-scripting.md) -- [CASE Statement](/sql/language-manual/control-flow/case-stmt.md) -- [Compound Statement](/sql/language-manual/control-flow/compound-stmt.md) -- [FOR Statement](/sql/language-manual/control-flow/for-stmt.md) -- [REPEAT Statement](/sql/language-manual/control-flow/repeat-stmt.md) -- [IF Statement](/sql/language-manual/control-flow/if-stmt.md) -- [ITERATE Statement](/sql/language-manual/control-flow/iterate-stmt.md) -- [LEAVE Statement](/sql/language-manual/control-flow/leave-stmt.md) -- [LOOP Statement](/sql/language-manual/control-flow/loop-stmt.md) - -``` +- [SQL Scripting](../sql-ref-scripting.html) +- [CASE Statement](../control-flow/case-stmt.html) +- [Compound Statement](../control-flow/compound-stmt.html) +- [FOR Statement](../control-flow/for-stmt.html) +- [REPEAT Statement](../control-flow/repeat-stmt.html) +- [IF Statement](../control-flow/if-stmt.html) +- [ITERATE Statement](../control-flow/iterate-stmt.html) +- [LEAVE Statement](../control-flow/leave-stmt.html) +- [LOOP Statement](../control-flow/loop-stmt.html) -``` diff --git a/docs/sql-ref-name-resolution.md b/docs/sql-ref-name-resolution.md new file mode 100644 index 000000000000..2532f05e164b --- /dev/null +++ b/docs/sql-ref-name-resolution.md @@ -0,0 +1,423 @@ +--- +layout: global +title: Name Resolution +displayTitle: Name Resolution +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +Name resolution is the process by which [identifiers](sql-ref-identifier.html) are resolved to specific column-, field-, parameter-, or table-references. + +## Column, field, parameter, and variable resolution + +Identifiers in expressions can be references to any one of the following: + +- Column name based on a view, table, common table expression (CTE), or a column_alias. +- Field name or map key within a struct or map. + Fields and keys can never be unqualified. +- Parameter name of a SQL User Defined Function. +- Session or SQL script local variable name. +- A special function such as `current_user` or `current_date` which does not require the usage of `()`. +- The `DEFAULT` keyword which is used in the context of `INSERT`, or `SET VARIABLE` to set a column or variable value to its default. + +Name resolution applies the following principles: + +- The _closest_ matching reference wins, and +- Columns and parameter win over fields and keys. + +In detail, resolution of identifiers to a specific reference follows these rules in order: + +1. **Local references** + + 1. **Column reference** + + Match the identifier, which may be qualified, to a column name in a table reference of the `FROM clause`. + + If there is more than one such match, raise an AMBIGUOUS_COLUMN_OR_FIELD error. + + 1. **Parameterless function reference** + + If the identifier is unqualified and matches `current_user`, `current_date`, or `current_timestamp`: Resolve it as one of these functions. + + 1. **Column DEFAULT specification** + + If the identifier is unqualified, matches `default` and makes up the entire expression in the context of an `UPDATE SET`, `INSERT VALUES`, or `MERGE WHEN [NOT] MATCHED`: Resolve as the respective `DEFAULT` value of the target table of the `INSERT`. + + 1. **Struct field or map key reference** + + If the identifier is qualified, then endeavor to match it to a field or map key according to the following steps: + + A. Remove the last identifier and treat it as a field or key. + + B. Match the remainder to a column in table reference of the `FROM clause`. + + - If there is more than one such match, raise an AMBIGUOUS_COLUMN_OR_FIELD error. + + - If there is a match and the column is a: + + - **`STRUCT`**: Match the field. + + If the field cannot be matched, raise a FIELD_NOT_FOUND error. + + If there is more than one field, raise a AMBIGUOUS_COLUMN_OR_FIELD error. + + - **`MAP`**: Raise an error if the key is qualified. + + A runtime error may occur if the key is not actually present in the map. + + - Any other type: Raise an error. + + C. Repeat the preceding step to remove the trailing identifier as a field. Apply rules (A) and (B) while there is an identifier left to interpret as a column. + +1. **Lateral column aliasing** + + If the expression is within a `SELECT` list, match the leading identifier to a preceding column alias in that `SELECT` list. + + If there is more than one such match, raise an AMBIGUOUS_LATERAL_COLUMN_ALIAS error. + + Match each remaining identifier as a field or a map key, and raise FIELD_NOT_FOUND or AMBIGUOUS_COLUMN_OR_FIELD error if they cannot be matched. + +1. **Correlation** + + - **LATERAL** + + If the query is preceded by a `LATERAL` keyword, apply rules 1.a and 1.d considering the table references in the `FROM` containing the query and preceding the `LATERAL`. + + - **Regular** + + If the query is a scalar subquery, `IN`, or `EXISTS` subquery apply rules 1.a, 1.d, and 2 considering the table references in the containing query’s `FROM` clause. + +1. **Nested correlation** + + Re-apply rule 3 iterating over the nesting levels of the query. + +1. **[FOR loop](control-flow/for-stmt.md]** + + If the statement is contained in a `FOR` loop: + + A. Match the identifier to a column in a `FOR` loop statement query. + + If the identifier is qualified, the qualifier must match the name of the FOR loop variable if defined. + + B. If the identifier is qualified, match to a field or map key of a parameter following rule 1.c + +1. **[Compound statement](control-flow/compound-stmt.html)** + + If the statement is contained in a compound statement: + + A. Match the identifier to a variable declared in that compound statement. + + If the identifier is qualified, the qualifier must match the label of the compound statement if one was defined. + + B. If the identifier is qualified, match to a field or map key of a variable following rule 1.c + +1. **Nested compound statement or `FOR` loop** + + Re-apply rules 5 and 6, iterating over the nesting levels of the compound statement. + +1. **Routine parameters** + + If the expression is part of a CREATE FUNCTION statement: + + 1. Match the identifier to a parameter name. If the identifier is qualified, the qualifier must match the name of the routine. + 1. If the identifier is qualified, match to a field or map key of a parameter following rule 1.c + +1. **Session Variables** + + 1. Match the identifier to a variable name. If the identifier is qualified, the qualifier must be `session` or `system.session`. + 1. If the identifier is qualified, match to a field or map key of a variable following rule 1.c + +### Limitations + +To prevent execution of potentially expensive correlated queries, Spark limits supported correlation to one level. +This restriction also applies to parameter references in SQL functions. + +### Examples + +```sql +-- Differentiating columns and fields +> SELECT a FROM VALUES(1) AS t(a); + 1 + +> SELECT t.a FROM VALUES(1) AS t(a); + 1 + +> SELECT t.a FROM VALUES(named_struct('a', 1)) AS t(t); + 1 + +-- A column takes precendece over a field +> SELECT t.a FROM VALUES(named_struct('a', 1), 2) AS t(t, a); + 2 + +-- Implict lateral column alias +> SELECT c1 AS a, a + c1 FROM VALUES(2) AS T(c1); + 2 4 + +-- A local column reference takes precedence, over a lateral column alias +> SELECT c1 AS a, a + c1 FROM VALUES(2, 3) AS T(c1, a); + 2 5 + +-- A scalar subquery correlation to S.c3 +> SELECT (SELECT c1 FROM VALUES(1, 2) AS t(c1, c2) + WHERE t.c2 * 2 = c3) + FROM VALUES(4) AS s(c3); + 1 + +-- A local reference takes precedence over correlation +> SELECT (SELECT c1 FROM VALUES(1, 2, 2) AS t(c1, c2, c3) + WHERE t.c2 * 2 = c3) + FROM VALUES(4) AS s(c3); + NULL + +-- An explicit scalar subquery correlation to s.c3 +> SELECT (SELECT c1 FROM VALUES(1, 2, 2) AS t(c1, c2, c3) + WHERE t.c2 * 2 = s.c3) + FROM VALUES(4) AS s(c3); + 1 + +-- Correlation from an EXISTS predicate to t.c2 +> SELECT c1 FROM VALUES(1, 2) AS T(c1, c2) + WHERE EXISTS(SELECT 1 FROM VALUES(2) AS S(c2) + WHERE S.c2 = T.c2); + 1 + +-- Attempt a lateral correlation to t.c2 +> SELECT c1, c2, c3 + FROM VALUES(1, 2) AS t(c1, c2), + (SELECT c3 FROM VALUES(3, 4) AS s(c3, c4) + WHERE c4 = c2 * 2); + [UNRESOLVED_COLUMN] `c2` + +-- Successsful usage of lateral correlation with keyword LATERAL +> SELECT c1, c2, c3 + FROM VALUES(1, 2) AS t(c1, c2), + LATERAL(SELECT c3 FROM VALUES(3, 4) AS s(c3, c4) + WHERE c4 = c2 * 2); + 1 2 3 + +-- Referencing a parameter of a SQL function +> CREATE OR REPLACE TEMPORARY FUNCTION func(a INT) RETURNS INT + RETURN (SELECT c1 FROM VALUES(1) AS T(c1) WHERE c1 = a); +> SELECT func(1), func(2); + 1 NULL + +-- A column takes precedence over a parameter +> CREATE OR REPLACE TEMPORARY FUNCTION func(a INT) RETURNS INT + RETURN (SELECT a FROM VALUES(1) AS T(a) WHERE t.a = a); +> SELECT func(1), func(2); + 1 1 + +-- Qualify the parameter with the function name +> CREATE OR REPLACE TEMPORARY FUNCTION func(a INT) RETURNS INT + RETURN (SELECT a FROM VALUES(1) AS T(a) WHERE t.a = func.a); +> SELECT func(1), func(2); + 1 NULL + +-- Lateral alias takes precedence over correlated reference +> SELECT (SELECT c2 FROM (SELECT 1 AS c1, c1 AS c2) WHERE c2 > 5) + FROM VALUES(6) AS t(c1) + NULL + +-- Lateral alias takes precedence over function parameters +> CREATE OR REPLACE TEMPORARY FUNCTION func(x INT) + RETURNS TABLE (a INT, b INT, c DOUBLE) + RETURN SELECT x + 1 AS x, x +> SELECT * FROM func(1) + 2 2 + +-- All together now +> CREATE OR REPLACE TEMPORARY VIEW lat(a, b) AS VALUES('lat.a', 'lat.b'); + +> CREATE OR REPLACE TEMPORARY VIEW frm(a) AS VALUES('frm.a'); + +> CREATE OR REPLACE TEMPORARY FUNCTION func(a INT, b int, c int) + RETURNS TABLE + RETURN SELECT t.* + FROM lat, + LATERAL(SELECT a, b, c + FROM frm) AS t; + +> VALUES func('func.a', 'func.b', 'func.c'); + a b c + ----- ----- ------ + frm.a lat.b func.c +``` + +## Table and view resolution + +An identifier in table-reference can be any one of the following: + +- Persistent table or view +- Common table expression (CTE) +- [Temporary view](sql-ref-syntax-ddl-create-view.html) + +Resolution of an identifier depends on whether it is qualified: + +- **Qualified** + + If the identifier is fully qualified with three parts: `catalog.schema.relation`, it is unique. + + If the identifier consists of two parts: `schema.relation`, it is further qualified with the result of `SELECT current_catalog()` to make it unique. + +- **Unqualified** + + 1. **Common table expression** + + If the reference is within the scope of a `WITH` clause, match the identifier to a CTE starting with the immediately containing `WITH` clause and moving outwards from there. + + 1. **Temporary view** + + Match the identifier to any temporary view defined within the current session. + + 1. **Persisted table** + + Fully qualify the identifier by pre-pending the result of `SELECT current_catalog()` and `SELECT current_schema()` and look it up as a persistent relation. + +If the relation cannot be resolved to any table, view, or CTE, Databricks raises a TABLE_OR_VIEW_NOT_FOUND error. + +### Examples + +```sql +-- Setting up a scenario +> USE CATALOG spark_catalog; +> USE SCHEMA default; + +> CREATE TABLE rel(c1 int); +> INSERT INTO rel VALUES(1); + +-- An fully qualified reference to rel: +> SELECT c1 FROM spark_catalog.default.rel; + 1 + +-- A partially qualified reference to rel: +> SELECT c1 FROM default.rel; + 1 + +-- An unqualified reference to rel: +> SELECT c1 FROM rel; + 1 + +-- Add a temporary view with a conflicting name: +> CREATE TEMPORARY VIEW rel(c1) AS VALUES(2); + +-- For unqualified references the temporary view takes precedence over the persisted table: +> SELECT c1 FROM rel; + 2 + +-- Temporary views cannot be qualified, so qualifiecation resolved to the table: +> SELECT c1 FROM default.rel; + 1 + +-- An unqualified reference to a common table expression wins even over a temporary view: +> WITH rel(c1) AS (VALUES(3)) + SELECT * FROM rel; + 3 + +-- If CTEs are nested, the match nearest to the table reference takes precedence. +> WITH rel(c1) AS (VALUES(3)) + (WITH rel(c1) AS (VALUES(4)) + SELECT * FROM rel); + 4 + +-- To resolve the table instead of the CTE, qualify it: +> WITH rel(c1) AS (VALUES(3)) + (WITH rel(c1) AS (VALUES(4)) + SELECT * FROM default.rel); + 1 + +-- For a CTE to be visible it must contain the query +> SELECT * FROM (WITH cte(c1) AS (VALUES(1)) + SELECT 1), + cte; + [TABLE_OR_VIEW_NOT_FOUND] The table or view `cte` cannot be found. +``` + +## Function resolution + +A function reference is recognized by the mandatory trailing set of parentheses. + +It can resolve to: + +- A builtin function provided by Spark, +- A temporary user defined function scoped to the current session, or +- A persistent user defined function. + +Resolution of a function name depends on whether it is qualified: + +- **Qualified** + + If the name is fully qualified with three parts: `catalog.schema.function`, it is unique. + + If the name consists of two parts: `schema.function`, it is further qualified with the result of `SELECT current_catalog()` to make it unique. + + The function is then looked up in the catalog. + +- **Unqualified** + + For unqualified function names Spark follows a fixed order of precedence (`PATH`): + + 1. **Builtin function** + + If a function by this name exists among the set of built-in functions, that function is chosen. + + 1. **Temporary function** + + If a function by this name exists among the set of temporary functions, that function is chosen. + + 1. **Persisted function** + + Fully qualify the function name by pre-pending the result of `SELECT current_catalog()` and `SELECT current_schema()` and look it up as a persistent function. + +If the function cannot be resolved Spark raises an `UNRESOLVED_ROUTINE` error. + +### Examples + +```sql +> USE CATALOG spark_catalog; +> USE SCHEMA default; + +-- Create a function with the same name as a builtin +> CREATE FUNCTION concat(a STRING, b STRING) RETURNS STRING + RETURN b || a; + +-- unqualified reference resolves to the builtin CONCAT +> SELECT concat('hello', 'world'); + helloworld + +-- Qualified reference resolves to the persistent function +> SELECT default.concat('hello', 'world'); + worldhello + +-- Create a persistent function +> CREATE FUNCTION func(a INT, b INT) RETURNS INT + RETURN a + b; + +-- The persistent function is resolved without qualifying it +> SELECT func(4, 2); + 6 + +-- Create a conflicting temporary function +> CREATE FUNCTION func(a INT, b INT) RETURNS INT + RETURN a / b; + +-- The temporary function takes precedent +> SELECT func(4, 2); + 2 + +-- To resolve the persistent function it now needs qualification +> SELECT spark_catalog.default.func(4, 3); + 6 +``` diff --git a/docs/sql-ref-scripting.md b/docs/sql-ref-scripting.md index abc0b1854328..57f224531da6 100644 --- a/docs/sql-ref-scripting.md +++ b/docs/sql-ref-scripting.md @@ -1,7 +1,7 @@ --- layout: global -title: SQL Syntax -displayTitle: SQL Syntax +title: SQL Scripting +displayTitle: SQL Scripting license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with @@ -19,19 +19,17 @@ license: | limitations under the License. --- -# SQL scripting - You can employ powerful procedural logic using SQL/PSM standard-based scripting syntax. -Any SQL script consists of and starts with a [compound statement](control-flow/compound-stmt.md) block (`BEGIN ... END`). +Any SQL script consists of and starts with a [compound statement](control-flow/compound-stmt.html) block (`BEGIN ... END`). A compound statement starts with a section to declare local variables, user-defined conditions, and condition handlers, which are used to catch exceptions. This is followed by the compound statement body, which consists of: -- Flow control statements include loops over predicate expressions, [FOR](control-flow/for-stmt.md) loops over query results, conditional logic such as [IF](control-flow/if-stmt.md) and [CASE](control-flow/case-stmt.md), and means to break out loops such as [LEAVE](control-flow/leave-stmt.md) and [ITERATE](control-flow/iterate-stmt.md). +- Flow control statements include loops over predicate expressions, [FOR](control-flow/for-stmt.html) loops over query results, conditional logic such as [IF](control-flow/if-stmt.html) and [CASE](control-flow/case-stmt.html), and means to break out loops such as [LEAVE](control-flow/leave-stmt.html) and [ITERATE](control-flow/iterate-stmt.html). - DDL statements such as `ALTER`, `CREATE`, `DROP`. -- DML statements [INSERT](sql-ref-syntax-dml-insert-into.md), [UPDATE](delta-update.md), [DELETE](delta-delete-from.md), and [MERGE](delta-merge-into.md). -- [Queries](sql-ref-syntax-qry-query.md) that return result sets to the invoker of the script. -- [SET](sql-ref-syntax-aux-set-variable.md) statements to set local variables as well as session variables. -- The [EXECUTE IMMEDIATE](sql-ref-syntax-aux-execute-immediate.md) statement. +- DML statements [INSERT](sql-ref-syntax-dml-insert-into.html). +- [Queries](sql-ref-syntax-qry-select.html) that return result sets to the invoker of the script. +- [SET](sql-ref-syntax-aux-set-var.html) statements to set local variables as well as session variables. +- The [EXECUTE IMMEDIATE](sql-ref-syntax-aux-exec-imm.html) statement. - Nested compound statements, which provide nested scopes for variables, conditions, and condition handlers. ## Passing data between the invoker and the compound statement @@ -44,13 +42,13 @@ There are two ways to pass data to and from a SQL script: ## Variable scoping Variables declared within a compound statement can be referenced in any expression within a compound statement. -Spark resolves identifiers from the innermost scope outward, following the rules described in [Name Resolution](sql-ref-name-resolution.md). -You can use the optional compound statement [labels](sql-ref-names.md#label-name) to disambiguate duplicate [variable names](sql-ref-names.md#variable-name). +Spark resolves identifiers from the innermost scope outward, following the rules described in [Name Resolution](sql-ref-name-resolution.html). +You can use the optional compound statement labels to disambiguate duplicate variable names. ## Condition handling SQL Scripting supports condition handlers, which are used to intercept and process exceptions to `EXIT` processing of the SQL script. -Within the condition handler, you can [RESIGNAL](control-flow/resignal-stmt.md) the original exception, [SIGNAL](control-flow/signal-stmt.md) a new exception, or exit the compound statement without an exception. +Within the condition handler, you can [RESIGNAL](control-flow/resignal-stmt.html) the original exception, [SIGNAL](control-flow/signal-stmt.html) a new exception, or exit the compound statement without an exception. Condition handlers can be defined to handle three distinct classes of conditions: @@ -75,17 +73,17 @@ This condition handler is called the **most appropriate handler**: Unless a handler `SIGNAL`s or `RESIGNAL`s a condition of its own, the outcome of a condition handler is to execute the statement following the compound statement that declared the handler to execute next. -The following is a list of supported control flow statement: - -* [CASE](control-flow/case-stmt.md) -* [compound statement](control-flow/compound-stmt.md) -* [FOR](control-flow/for-stmt.md) -* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.md) -* [IF](control-flow/if-stmt.md) -* [ITERATE](control-flow/iterate-stmt.md) -* [LEAVE](control-flow/leave-stmt.md) -* [LOOP](control-flow/loop-stmt.md) -* [REPEAT](control-flow/repeat-stmt.md) -* [RESIGNAL](control-flow/resignal-stmt.md) -* [SIGNAL](control-flow/signal-stmt.md) -* [WHILE](control-flow/while-stmt.md) +The following is a list of supported control flow statements: + +* [CASE](control-flow/case-stmt.html) +* [compound statement](control-flow/compound-stmt.html) +* [FOR](control-flow/for-stmt.html) +* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.html) +* [IF](control-flow/if-stmt.html) +* [ITERATE](control-flow/iterate-stmt.html) +* [LEAVE](control-flow/leave-stmt.html) +* [LOOP](control-flow/loop-stmt.html) +* [REPEAT](control-flow/repeat-stmt.html) +* [RESIGNAL](control-flow/resignal-stmt.html) +* [SIGNAL](control-flow/signal-stmt.html) +* [WHILE](control-flow/while-stmt.html) diff --git a/docs/sql-ref-syntax-qry-select-tvf.md b/docs/sql-ref-syntax-qry-select-tvf.md index 099f423f1853..9dd73a3f2526 100644 --- a/docs/sql-ref-syntax-qry-select-tvf.md +++ b/docs/sql-ref-syntax-qry-select-tvf.md @@ -172,5 +172,5 @@ SELECT * FROM test, LATERAL explode (ARRAY(3,4)) AS c2; ### Related Statements * [SELECT](sql-ref-syntax-qry-select.html) -* [LATERAL](sql-ref-syntax-qry-select-lateral-subquery.md) +* [LATERAL](sql-ref-syntax-qry-select-lateral-subquery.html) * [LATERAL VIEW Clause](sql-ref-syntax-qry-select-lateral-view.html) diff --git a/docs/sql-ref-syntax.md b/docs/sql-ref-syntax.md index 5a67f8b04bc5..ab7a8f464796 100644 --- a/docs/sql-ref-syntax.md +++ b/docs/sql-ref-syntax.md @@ -94,18 +94,18 @@ ability to generate logical and physical plan for a given query using You use SQL scripting to execute procedural logic in SQL. -* [CASE](control-flow/case-stmt.md) -* [compound statement](control-flow/compound-stmt.md) -* [FOR](control-flow/for-stmt.md) -* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.md) -* [IF](control-flow/if-stmt.md) -* [ITERATE](control-flow/iterate-stmt.md) -* [LEAVE](control-flow/leave-stmt.md) -* [LOOP](control-flow/loop-stmt.md) -* [REPEAT](control-flow/repeat-stmt.md) -* [RESIGNAL](control-flow/resignal-stmt.md) -* [SIGNAL](control-flow/signal-stmt.md) -* [WHILE](control-flow/while-stmt.md) +* [CASE](control-flow/case-stmt.html) +* [compound statement](control-flow/compound-stmt.html) +* [FOR](control-flow/for-stmt.html) +* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.html) +* [IF](control-flow/if-stmt.html) +* [ITERATE](control-flow/iterate-stmt.html) +* [LEAVE](control-flow/leave-stmt.html) +* [LOOP](control-flow/loop-stmt.html) +* [REPEAT](control-flow/repeat-stmt.html) +* [RESIGNAL](control-flow/resignal-stmt.html) +* [SIGNAL](control-flow/signal-stmt.html) +* [WHILE](control-flow/while-stmt.html) ### Auxiliary Statements diff --git a/docs/sql-ref.md b/docs/sql-ref.md index 95933eb0efb4..cf9a887bd492 100644 --- a/docs/sql-ref.md +++ b/docs/sql-ref.md @@ -37,11 +37,12 @@ Spark SQL is Apache Spark's module for working with structured data. This guide * [IDENTIFIER clause](sql-ref-identifier-clause.html) * [Literals](sql-ref-literals.html) * [Null Semantics](sql-ref-null-semantics.html) + * [Name Resolution](sql-ref-name-resolution.html) * [SQL Scripting](sql-ref-scripting.html) * [SQL Syntax](sql-ref-syntax.html) * [DDL Statements](sql-ref-syntax.html#ddl-statements) * [DML Statements](sql-ref-syntax.html#dml-statements) * [Data Retrieval Statements](sql-ref-syntax.html#data-retrieval-statements) - * [SQL Scripting](sql-ref-syntax.html#sql-scripting) + * [SQL Scripting Statements](sql-ref-syntax.html#sql-scripting-statements) * [Auxiliary Statements](sql-ref-syntax.html#auxiliary-statements) * [Pipe Syntax](sql-pipe-syntax.html) From b6fab1232581d7322e583ecff24746785035fa13 Mon Sep 17 00:00:00 2001 From: David Milicevic Date: Fri, 21 Nov 2025 16:11:07 +0100 Subject: [PATCH 3/4] Update docs to reflect the current state --- docs/control-flow/compound-stmt.md | 5 +- docs/control-flow/get-diagnostics-stmt.md | 120 ---------------------- docs/control-flow/resignal-stmt.md | 88 ---------------- docs/control-flow/signal-stmt.md | 83 --------------- docs/sql-ref-scripting.md | 8 +- docs/sql-ref-syntax.md | 3 - 6 files changed, 3 insertions(+), 304 deletions(-) delete mode 100644 docs/control-flow/get-diagnostics-stmt.md delete mode 100644 docs/control-flow/resignal-stmt.md delete mode 100644 docs/control-flow/signal-stmt.md diff --git a/docs/control-flow/compound-stmt.md b/docs/control-flow/compound-stmt.md index 82beade5cb53..d34e70648de4 100644 --- a/docs/control-flow/compound-stmt.md +++ b/docs/control-flow/compound-stmt.md @@ -32,7 +32,7 @@ Implements a SQL Script block that can contain a sequence of SQL statements, con END [ label ] declare_variable - DECLARE variable_name datatype [ DEFAULT default_expr ] + DECLARE variable_name [, ...] datatype [ DEFAULT default_expr ] declare_condition DECLARE condition_name CONDITION [ FOR SQLSTATE [ VALUE ] sqlstate ] @@ -162,6 +162,3 @@ END; - [FOR Statement](../control-flow/for-stmt.html) - [ITERATE Statement](../control-flow/iterate-stmt.html) - [LEAVE Statement](../control-flow/leave-stmt.html) -- [SIGNAL Statement](../control-flow/signal-stmt.html) -- [RESIGNAL Statement](../control-flow/resignal-stmt.html) -- [GET DIAGNOSTICS Statement](../control-flow/get-diagnostics-stmt.html) diff --git a/docs/control-flow/get-diagnostics-stmt.md b/docs/control-flow/get-diagnostics-stmt.md deleted file mode 100644 index ecf61c7c49f3..000000000000 --- a/docs/control-flow/get-diagnostics-stmt.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -layout: global -title: GET DIAGNOSTICS statement -displayTitle: GET DIAGNOSTICS statement -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Retrieve information about a condition handled in an exception handler. - -This statement may only be used within a condition handler in a [compound statement](compound-stmt.html). - -## Syntax - -``` -GET DIAGNOSTICS CONDITION 1 - { variable_name = condition_info_item } [, ...] - -condition_info_item - { MESSAGE_TEXT | - RETURNED_SQLSTATE | - MESSAGE_ARGUMENTS | - CONDITION_IDENTIFIER | - LINE_NUMBER } -``` - -## Parameters - -- **variable_name** - - A local variable or session variable. - -- **`CONDITION 1`** - - Returns the condition that triggered the condition handler. - You must call issue `GET DIAGNOSTICS CONDITION 1` as the first statement in the handler. - - - **`MESSAGE_TEXT`** - - Returns the message text associated with the condition as a `STRING`. - `variable_name` must be a `STRING`. - - - **`RETURNED_SQLSTATE`** - - Returns the `SQLSTATE` associated with the condition being handled as a `STRING`. - `variable_name` must be a `STRING`. - - - **`MESSAGE_ARGUMENTS`** - - Returns a `MAP` mapping provided as arguments to the parameters of Spark conditions. - For declared conditions, the only map key is `MESSAGE_TEXT`. - `variable_name` must be a `MAP` - - - **`CONDITION_IDENTIFIER`** - - Returns the condition name that caused the exception. - `variable_name` must be a `STRING`. - - - **`LINE_NUMBER`** - - Returns the line number of the statement raising the condition. - `NULL` if not available. - -## Examples - -```SQL --- Retrieve the number of rows inserted by an INSERT statement -> CREATE OR REPLACE TABLE emp(name STRING, salary DECIMAL(10, 2)); - -> BEGIN - DECLARE EXIT HANDLER FOR DIVIDE_BY_ZERO - BEGIN - DECLARE cond STRING; - DECLARE message STRING; - DECLARE state STRING; - DECLARE args MAP; - DECLARE line BIGINT; - DECLARE argstr STRING; - DECLARE log STRING; - GET DIAGNOSTICS CONDITION 1 - cond = CONDITION_IDENTIFIER, - message = MESSAGE_TEXT, - state = RETURNED_SQLSTATE, - args = MESSAGE_ARGUMENTS, - line = LINE_NUMBER; - SET argstr = - (SELECT aggregate(array_agg('Parm:' || key || ' Val: value '), - '', (acc, x)->(acc || ' ' || x)) - FROM explode(args) AS args(key, val)); - SET log = 'Condition: ' || cond || - ' Message: ' || message || - ' SQLSTATE: ' || state || - ' Args: ' || argstr || - ' Line: ' || line; - VALUES (log); - END; - SELECT 10/0; - END; - Condition: DIVIDE_BY_ZERO Message: Division by zero. Use try_divide to tolerate divisor being 0 and return NULL instead. If necessary, set to “false” to bypass this error. SQLATTE: 22012 Args: Parm: config Val: ANSI_MODE Line: 28 -``` - -## Related articles - -- [SQL Scripting](../sql-ref-scripting.html) -- [CASE Statement](../control-flow/case-stmt.html) -- [Compound Statement](../control-flow/compound-stmt.html) -- [FOR Statement](../control-flow/for-stmt.html) diff --git a/docs/control-flow/resignal-stmt.md b/docs/control-flow/resignal-stmt.md deleted file mode 100644 index e2b24221e977..000000000000 --- a/docs/control-flow/resignal-stmt.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -layout: global -title: RESIGNAL statement -displayTitle: RESIGNAL statement -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Re-raises the condition handled by the condition handler. - -This statement may only be used within a [compound statement](compound-stmt.html). - -## Syntax - -``` -RESIGNAL -``` - -## Parameters - -None - -## Examples - -```SQL -> CREATE TABLE log(eventtime TIMESTAMP, log STRING); - -> BEGIN - DECLARE EXIT HANDLER FOR DIVIDE_BY_ZERO - BEGIN - DECLARE cond STRING; - DECLARE message STRING; - DECLARE state STRING; - DECLARE args MAP; - DECLARE line BIGINT; - DECLARE argstr STRING; - DECLARE log STRING; - GET DIAGNOSTICS CONDITION 1 - cond = CONDITION_IDENTIFIER, - message = MESSAGE_TEXT, - state = RETURNED_SQLSTATE, - args = MESSAGE_ARGUMENTS, - line = LINE_NUMBER; - SET argstr = - (SELECT aggregate(array_agg('Parm:' || key || ' Val: value '), - '', (acc, x)->(acc || ' ' || x)) - FROM explode(args) AS args(key, val)); - SET log = 'Condition: ' || cond || - ' Message: ' || message || - ' SQLSTATE: ' || state || - ' Args: ' || argstr || - ' Line: ' || line; - INSERT INTO log VALUES(current_timestamp(), log); - RESIGNAL; - END; - SELECT 10/0; - END; - [DIVIDE_BY_ZERO] Division by zero. Use try_divide to tolerate divisor being 0 and return NULL instead. - -> SELECT * FROM log ORDER BY eventtime DESC LIMIT 1; - Condition: DIVIDE_BY_ZERO Message: Division by zero. Use try_divide to tolerate divisor being 0 and return NULL instead. SQLSTATE: 22012 Args: Line: 28 -``` - -## Related articles - -- [SQL Scripting](../sql-ref-scripting.html) -- [CASE Statement](../control-flow/case-stmt.html) -- [Compound Statement](../control-flow/compound-stmt.html) -- [SIGNAL Statement](../control-flow/signal-stmt.html) -- [FOR Statement](../control-flow/for-stmt.html) -- [IF Statement](../control-flow/if-stmt.html) -- [ITERATE Statement](../control-flow/iterate-stmt.html) -- [REPEAT Statement](../control-flow/repeat-stmt.html) -- [SIGNAL Statement](../control-flow/signal-stmt.html) -- [Error handling and error messages](/error-messages/index.html) diff --git a/docs/control-flow/signal-stmt.md b/docs/control-flow/signal-stmt.md deleted file mode 100644 index e74727c4014d..000000000000 --- a/docs/control-flow/signal-stmt.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -layout: global -title: SIGNAL statement -displayTitle: SIGNAL statement -license: | - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---- - -Raises a condition. - -This statement may only be used within a [compound statement](compound-stmt.html). - -Note: Spark recommends using [RESIGNAL](resignal-stmt.html) to raise conditions from within a handler. -RESIGNAL builds a diagnostic stack in the SQL Standard, while `SIGNAL` clears the stack. -Using `RESIGNAL` within a handler preserves future exploitation of the diagnostic stack. - -## Syntax - -``` -SIGNAL { condition_name - [ SET { MESSAGE_ARGUMENTS = argument_map | - MESSAGE_TEXT = message_str } ] | - SQLSTATE [VALUE] sqlstate [ SET MESSAGE_TEXT = message_str ] } -``` - -## Parameters - -- **condition_name** - - The name of a locally defined condition or system-defined error condition. - -- **argument_map** - - Optionally, a `MAP` literal that assigns values to a system-defined parameterized condition message. - -- **message_str** - - Optionally, a `STRING` literal that provides a message string to the raised `SQLSTATE` or user-defined condition. - -- **sqlstate** - - A `STRING` literal of length 5. If specified, raise `USER_RAISED_EXCEPTION` with the specified `SQLSTATE`. - -## Examples - -```SQL -> DECLARE input INT DEFAULT 5; - -> BEGIN - DECLARE arg_map MAP; - IF input > 4 THEN - SET arg_map = map('errorMessage', - 'Input must be <= 4.'); - SIGNAL USER_RAISED_EXCEPTION - SET MESSAGE_ARGUMENTS = arg_map; - END IF; - END; -``` - -## Related articles - -- [SQL Scripting](../sql-ref-scripting.html) -- [CASE Statement](../control-flow/case-stmt.html) -- [Compound Statement](../control-flow/compound-stmt.html) -- [FOR Statement](../control-flow/for-stmt.html) -- [IF Statement](../control-flow/if-stmt.html) -- [ITERATE Statement](../control-flow/iterate-stmt.html) -- [REPEAT Statement](../control-flow/repeat-stmt.html) -- [RESIGNAL Statement](../control-flow/resignal-stmt.html) -- [Error handling and error messages](/error-messages/index.html) diff --git a/docs/sql-ref-scripting.md b/docs/sql-ref-scripting.md index 57f224531da6..e9407cff2301 100644 --- a/docs/sql-ref-scripting.md +++ b/docs/sql-ref-scripting.md @@ -48,14 +48,13 @@ You can use the optional compound statement labels to disambiguate duplicate var ## Condition handling SQL Scripting supports condition handlers, which are used to intercept and process exceptions to `EXIT` processing of the SQL script. -Within the condition handler, you can [RESIGNAL](control-flow/resignal-stmt.html) the original exception, [SIGNAL](control-flow/signal-stmt.html) a new exception, or exit the compound statement without an exception. Condition handlers can be defined to handle three distinct classes of conditions: - One or more named conditions that can be a specific Spark-defined error class such as `DIVIDE_BY_ZERO` or a user-declared condition. These handlers handle these specific conditions. -- One or more `SQLSTATE`s, that can be raised by Spark or a user `SIGNAL` statement. +- One or more `SQLSTATE`s, that can be raised by Spark. These handlers can handle any condition associated with that `SQLSTATE`. - A generic `SQLEXCEPTION` handler can catch all conditions falling into the `SQLEXCEPTION` (any `SQLSTATE` which is not `XX***` and not `02***`). @@ -71,19 +70,16 @@ This condition handler is called the **most appropriate handler**: For example, a handler on a named condition is more specific than one on a named `SQLSTATE`. A generic `EXCEPTION` handler is the least specific. -Unless a handler `SIGNAL`s or `RESIGNAL`s a condition of its own, the outcome of a condition handler is to execute the statement following the compound statement that declared the handler to execute next. +The outcome of a condition handler is to execute the statement following the compound statement that declared the handler to execute next. The following is a list of supported control flow statements: * [CASE](control-flow/case-stmt.html) * [compound statement](control-flow/compound-stmt.html) * [FOR](control-flow/for-stmt.html) -* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.html) * [IF](control-flow/if-stmt.html) * [ITERATE](control-flow/iterate-stmt.html) * [LEAVE](control-flow/leave-stmt.html) * [LOOP](control-flow/loop-stmt.html) * [REPEAT](control-flow/repeat-stmt.html) -* [RESIGNAL](control-flow/resignal-stmt.html) -* [SIGNAL](control-flow/signal-stmt.html) * [WHILE](control-flow/while-stmt.html) diff --git a/docs/sql-ref-syntax.md b/docs/sql-ref-syntax.md index ab7a8f464796..0a2b9ba34b52 100644 --- a/docs/sql-ref-syntax.md +++ b/docs/sql-ref-syntax.md @@ -97,14 +97,11 @@ You use SQL scripting to execute procedural logic in SQL. * [CASE](control-flow/case-stmt.html) * [compound statement](control-flow/compound-stmt.html) * [FOR](control-flow/for-stmt.html) -* [GET DIAGNOSTICS](control-flow/get-diagnostics-stmt.html) * [IF](control-flow/if-stmt.html) * [ITERATE](control-flow/iterate-stmt.html) * [LEAVE](control-flow/leave-stmt.html) * [LOOP](control-flow/loop-stmt.html) * [REPEAT](control-flow/repeat-stmt.html) -* [RESIGNAL](control-flow/resignal-stmt.html) -* [SIGNAL](control-flow/signal-stmt.html) * [WHILE](control-flow/while-stmt.html) ### Auxiliary Statements From 3ca308a5d35ba33e6d64f4444f246453ca6d9f8a Mon Sep 17 00:00:00 2001 From: David Milicevic <163021185+davidm-db@users.noreply.github.com> Date: Mon, 24 Nov 2025 14:13:38 +0100 Subject: [PATCH 4/4] Add a note on how to enable SQL Scripting --- docs/sql-ref-scripting.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/sql-ref-scripting.md b/docs/sql-ref-scripting.md index e9407cff2301..a8bccb471d56 100644 --- a/docs/sql-ref-scripting.md +++ b/docs/sql-ref-scripting.md @@ -32,6 +32,10 @@ This is followed by the compound statement body, which consists of: - The [EXECUTE IMMEDIATE](sql-ref-syntax-aux-exec-imm.html) statement. - Nested compound statements, which provide nested scopes for variables, conditions, and condition handlers. +## Enablement + +To enable SQL Scripting, set the `spark.sql.scripting.enabled` flag to `true`. + ## Passing data between the invoker and the compound statement There are two ways to pass data to and from a SQL script: