layout | title | displayTitle | license |
---|---|---|---|
global |
Error Conditions |
Error Conditions |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
|
This is a list of common, named error conditions returned by Spark SQL.
Also see SQLSTATE Codes.
Column or field <name>
is ambiguous and has <n>
matches.
Lateral column alias <name>
is ambiguous and has <n>
matches.
Reference <name>
is ambiguous, could be: <referenceNames>
.
Ambiguous reference to the field <field>
. It appears <count>
times in the schema.
<message>
.<alternative>
If necessary set <config>
to "false" to bypass this error.
Cannot cast <sourceType>
to <targetType>
.
SQLSTATE: none assigned
Error constructing FileDescriptor for <descFilePath>
.
SQLSTATE: none assigned
Cannot convert Protobuf <protobufColumn>
to SQL <sqlColumn>
because schema is incompatible (protobufType = <protobufType>
, sqlType = <sqlType>
).
SQLSTATE: none assigned
Unable to convert <protobufType>
of Protobuf to SQL type <toType>
.
SQLSTATE: none assigned
Cannot convert SQL <sqlColumn>
to Protobuf <protobufColumn>
because <data>
cannot be written since it's not defined in ENUM <enumString>
.
SQLSTATE: none assigned
Cannot convert SQL <sqlColumn>
to Protobuf <protobufColumn>
because schema is incompatible (protobufType = <protobufType>
, sqlType = <sqlType>
).
Cannot decode url : <url>
.
SQLSTATE: none assigned
Cannot load class <className>
when registering the function <functionName>
, please make sure it is on the classpath.
SQLSTATE: none assigned
Could not load Protobuf class with name <protobufClassName>
. <explanation>
.
Failed to merge incompatible data types <left>
and <right>
.
Cannot modify the value of the Spark config: <key>
.
See also '<docroot>
/sql-migration-guide.html#ddl-statements'.
Cannot parse decimal.
Cannot parse the field name <fieldName>
and the value <fieldValue>
of the JSON token type <jsonType>
to target Spark data type <dataType>
.
SQLSTATE: none assigned
Error parsing file <descFilePath>
descriptor byte[] into Descriptor object.
<message>
. If necessary set <ansiConfig>
to "false" to bypass this error.
SQLSTATE: none assigned
Could not read footer for file: <file>
.
Cannot recognize hive type string: <fieldType>
, column: <fieldName>
.
SQLSTATE: none assigned
Failed to set permissions on created path <path>
back to <permission>
.
SQLSTATE: none assigned
Cannot up cast <expression>
from <sourceType>
to <targetType>
.
<details>
The value <expression>
of the type <sourceType>
cannot be cast to <targetType>
because it is malformed. Correct the value as per the syntax, or change its target type. Use try_cast
to tolerate malformed input and return NULL instead. If necessary set <ansiConfig>
to "false" to bypass this error.
The value <value>
of the type <sourceType>
cannot be cast to <targetType>
due to an overflow. Use try_cast
to tolerate overflow and return NULL instead. If necessary set <ansiConfig>
to "false" to bypass this error.
Fail to insert a value of <sourceType>
type into the <targetType>
type column <columnName>
due to an overflow. Use try_cast
on the input value to tolerate overflow and return NULL instead.
The column <columnName>
already exists. Consider to choose another name or rename the existing column.
The column <colName>
cannot be found. Verify the spelling and correctness of the column name according to the SQL config <caseSensitiveConfig>
.
SQLSTATE: none assigned
The comparator has returned a NULL for a comparison between <firstValue>
and <secondValue>
. It should return a positive integer for "greater than", 0 for "equal" and a negative integer for "less than". To revert to deprecated behavior where NULL is treated as 0 (equal), you must set "spark.sql.legacy.allowNullComparisonResultInArraySort" to "true".
SQLSTATE: none assigned
Another instance of this query was just started by a concurrent session.
SQLSTATE: none assigned
Generic Spark Connect error.
For more details see CONNECT
The value <str>
(<fmt>
) cannot be converted to <targetType>
because it is malformed. Correct the value as per the syntax, or change its format. Use <suggestion>
to tolerate malformed input and return NULL instead.
CREATE TABLE column <columnName>
specifies option "<optionName>
" more than once, which is invalid.
Cannot resolve <sqlExpr>
due to data type mismatch:
For more details see DATATYPE_MISMATCH
DataType <type>
requires a length parameter, for example <type>
(10). Please specify the length.
Failed to find the data source: <provider>
. Please find packages at https://spark.apache.org/third-party-projects.html
.
Datetime operation overflow: <operation>
.
Decimal precision <precision>
exceeds max precision <maxPrecision>
.
Default database <defaultDatabase>
does not exist, please create it first or change default database to <defaultDatabase>
.
Division by zero. Use try_divide
to tolerate divisor being 0 and return NULL instead. If necessary set <config>
to "false" to bypass this error.
Duplicate map key <key>
was found, please check the input data. If you want to remove the duplicated keys, you can set <mapKeyDedupPolicy>
to "LAST_WIN" so that the key inserted at last takes precedence.
Found duplicate keys <keyColumn>
.
Failed to parse an empty string for data type <dataType>
.
SQLSTATE: none assigned
Not found an encoder of the type <typeName>
to Spark SQL internal representation. Consider to change the input type to one of supported at '<docroot>
/sql-ref-datatypes.html'.
Failed to execute user defined function (<functionName>
: (<signature>
) => <result>
).
Failed preparing of the function <funcName>
for call. Please, double check function's arguments.
Failed to rename <sourcePath>
to <targetPath>
as destination already exists.
No such struct field <fieldName>
in <fields>
.
The operation <statement>
is not allowed on the <objectType>
: <objectName>
.
SQLSTATE: none assigned
A column cannot have both a default value and a generation expression but column <colName>
has default value: (<defaultValue>
) and generation expression: (<genExpr>
).
SQLSTATE: none assigned
Invalid Graphite protocol: <protocol>
.
SQLSTATE: none assigned
Graphite sink requires '<property>
' property.
Column of grouping (<grouping>
) can't be found in grouping columns <groupingColumns>
.
Columns of grouping_id (<groupingIdColumn>
) does not match grouping columns (<groupByColumns>
).
Grouping sets size cannot be greater than <maxSize>
.
Aggregate functions are not allowed in GROUP BY, but found <sqlExpr>
.
GROUP BY <index>
refers to an expression <aggExpr>
that contains an aggregate function. Aggregate functions are not allowed in GROUP BY.
GROUP BY position <index>
is not in select list (valid range is [1, <size>
]).
<identifier>
is not a valid identifier as it has more than 2 name parts.
Invalid pivot column <columnName>
. Pivot columns must be comparable.
<operator>
can only be performed on tables with compatible column types. The <columnOrdinalNumber>
column of the <tableOrdinalNumber>
table is type which is not compatible with at the same column of the first table.<hint>
.
SQLSTATE: none assigned
Detected an incompatible DataSourceRegister. Please remove the incompatible library from classpath or upgrade it. Error: <message>
The join types and are incompatible.
SQLSTATE: none assigned
The SQL query of view <viewName>
has an incompatible schema change and column <colName>
cannot be resolved. Expected <expectedNum>
columns named <colName>
but got <actualCols>
.
Please try to re-create the view by running: <suggestion>
.
Incomplete complex type:
For more details see INCOMPLETE_TYPE_DEFINITION
You may get a different result due to the upgrading to
For more details see INCONSISTENT_BEHAVIOR_CROSS_VERSION
Max offset with <rowsPerSecond>
rowsPerSecond is <maxSeconds>
, but it's <endSeconds>
now.
Max offset with <rowsPerSecond>
rowsPerSecond is <maxSeconds>
, but 'rampUpTimeSeconds' is <rampUpTimeSeconds>
.
Cannot create the index <indexName>
on table <tableName>
because it already exists.
Cannot find the index <indexName>
on table <tableName>
.
<message>
<message>
.<alternative>
Division by zero. Use try_divide
to tolerate divisor being 0 and return NULL instead.
The index <indexValue>
is out of bounds. The array has <arraySize>
elements. Use the SQL function get()
to tolerate accessing element at invalid index and return NULL instead. If necessary set <ansiConfig>
to "false" to bypass this error.
The index <indexValue>
is out of bounds. The array has <arraySize>
elements. Use try_element_at
to tolerate accessing element at invalid index and return NULL instead. If necessary set <ansiConfig>
to "false" to bypass this error.
SQLSTATE: none assigned
Invalid bucket file: <path>
.
SQLSTATE: none assigned
The expected format is ByteString, but was <unsupported>
(<class>
).
The datasource <datasource>
cannot save the column <columnName>
because its name contains some characters that are not allowed in file paths. Please, use an alias to rename it.
Column or field <name>
is of type <type>
while it's required to be <expectedType>
.
The location name cannot be empty string, but <location>
was given.
Can't extract a value from <base>
. Need a complex type [STRUCT, ARRAY, MAP] but got <other>
.
Cannot extract <field>
from <expr>
.
Field name should be a non-null string literal, but it's <extraction>
.
Field name <fieldName>
is invalid: <path>
is not a struct.
The format is invalid: <format>
.
For more details see INVALID_FORMAT
The fraction of sec must be zero. Valid range is [0, 60]. If necessary set <ansiConfig>
to "false" to bypass this error.
The identifier <ident>
is invalid. Please, consider quoting it with back-quotes as <ident>
.
The index 0 is invalid. An index shall be either < 0 or > 0 (the first element has index 1).
Cannot convert JSON root field to target Spark type.
Input schema <jsonSchema>
can only contain STRING as a key type for a MAP.
The <joinType>
JOIN with LATERAL correlation is not allowed because an OUTER subquery cannot correlate to its join partner. Remove the LATERAL correlation or use an INNER JOIN, or LEFT OUTER JOIN instead.
Invalid options:
For more details see INVALID_OPTIONS
The group aggregate pandas UDF <functionList>
cannot be invoked together with as other, non-pandas aggregate functions.
The value of parameter(s) <parameter>
in <functionName>
is invalid:
For more details see INVALID_PARAMETER_VALUE
<key>
is an invalid property key, please use quotes, e.g. SET <key>
=<value>
.
<value>
is an invalid property value, please use quotes, e.g. SET <key>
=<value>
The input schema <inputSchema>
is not a valid schema string.
For more details see INVALID_SCHEMA
Expected format is 'SET', 'SET key', or 'SET key=value'. If you want to include special characters in key, or include semicolon in value, please use backquotes, e.g., SET key
=value
.
SQLSTATE: none assigned
The argument <name>
of sql()
is invalid. Consider to replace it by a SQL literal.
Invalid SQL syntax: <inputString>
.
Invalid subquery:
For more details see INVALID_SUBQUERY_EXPRESSION
SQLSTATE: none assigned
Cannot create the persistent object <objName>
of the type <obj>
because it references to the temporary object <tempObjName>
of the type <tempObj>
. Please make the temporary object <tempObjName>
persistent, or make the persistent object <objName>
temporary.
The value of the typed literal <valueType>
is invalid: <value>
.
The WHERE condition <condition>
contains invalid expressions: <expressionList>
.
Rewrite the query to avoid window functions, aggregate functions, and generator functions in the WHERE clause.
Cannot name the managed table as <identifier>
, as its associated location <location>
already exists. Please pick a different table name, or remove the existing location first.
SQLSTATE: none assigned
Malformed CSV record: <badRecord>
SQLSTATE: none assigned
Malformed Protobuf messages are detected in message deserialization. Parse Mode: <failFastMode>
. To process malformed protobuf message as null result, try setting the option 'mode' as 'PERMISSIVE'.
SQLSTATE: none assigned
Malformed records are detected in record parsing: <badRecord>
.
Parse Mode: <failFastMode>
. To process malformed records as null result, try setting the option 'mode' as 'PERMISSIVE'.
The non-aggregating expression <expression>
is based on columns which are not participating in the GROUP BY clause.
Add the columns or the expression to the GROUP BY, aggregate the expression, or use <expressionAnyValue>
if you do not care which of the values within a group is returned.
The query does not include a GROUP BY clause. Add GROUP BY or turn it into the window functions using OVER clauses.
SQLSTATE: none assigned
Not allowed to implement multiple UDF interfaces, UDF class <className>
.
It is not allowed to use an aggregate function in the argument of another aggregate function. Please use the inner aggregate function in a sub-query.
When there are more than one MATCHED clauses in a MERGE statement, only the last MATCHED clause can omit the condition.
When there are more than one NOT MATCHED BY SOURCE clauses in a MERGE statement, only the last NOT MATCHED BY SOURCE clause can omit the condition.
When there are more than one NOT MATCHED [BY TARGET] clauses in a MERGE statement, only the last NOT MATCHED [BY TARGET] clause can omit the condition.
Literal expressions required for pivot values, found <expression>
.
PARTITION clause cannot contain the non-partition column: <columnName>
.
SQLSTATE: none assigned
Operation <operation>
is not allowed for <tableIdentWithDB>
because it is not a partitioned table.
Assigning a NULL is not allowed here.
For more details see NOT_NULL_CONSTRAINT_VIOLATION
SQLSTATE: none assigned
No handler for UDAF '<functionName>
'. Use sparkSession.udf.register(...) instead.
SQLSTATE: none assigned
Cannot find <catalystFieldPath>
in Protobuf schema.
SQLSTATE: none assigned
UDF class <className>
doesn't implement any UDF interface.
Column or field <name>
is nullable while it's required to be non-nullable.
Row ID attributes cannot be nullable: <nullableRowIdAttrs>
.
Cannot use null as map key.
The value <value>
cannot be interpreted as a numeric since it has more than 38 digits.
<value>
cannot be represented as Decimal(<precision>
, <scale>
). If necessary set <config>
to "false" to bypass this error, and return NULL instead.
<operator>
can only be performed on inputs with the same number of columns, but the first input has <firstNumColumns>
columns and the <invalidOrdinalNum>
input has <invalidNumColumns>
columns.
ORDER BY position <index>
is not in select list (valid range is [1, <size>
]).
Syntax error, unexpected empty statement.
Syntax error at or near <error>``<hint>
.
Cannot ADD or RENAME TO partition(s) <partitionList>
in table <tableName>
because they already exist.
Choose a different name, drop the existing partition, or add the IF NOT EXISTS clause to tolerate a pre-existing partition.
The partition(s) <partitionList>
cannot be found in table <tableName>
.
Verify the partition specification and table name.
To tolerate the error on drop use ALTER TABLE … DROP IF EXISTS PARTITION.
Path <outputPath>
already exists. Set mode as "overwrite" to overwrite the existing path.
Path does not exist: <path>
.
Invalid pivot value '<value>
': value data type <valueType>
does not match pivot column data type <pivotType>
.
SQLSTATE: none assigned
The input plan of <ruleExecutor>
is invalid: <reason>
SQLSTATE: none assigned
Rule <rule>
in batch <batch>
generated an invalid plan: <reason>
SQLSTATE: none assigned
Could not find dependency: <dependencyName>
.
SQLSTATE: none assigned
Error reading Protobuf descriptor file at path: <filePath>
.
SQLSTATE: none assigned
Searching for <field>
in Protobuf schema at <protobufSchema>
gave <matchSize>
matches. Candidates: <matches>
.
SQLSTATE: none assigned
Found <field>
in Protobuf schema but there is no match in the SQL schema.
SQLSTATE: none assigned
Type mismatch encountered for field: <field>
.
SQLSTATE: none assigned
Unable to locate Message <messageName>
in Descriptor.
SQLSTATE: none assigned
Protobuf type not yet supported: <protobufType>
.
SQLSTATE: none assigned
Found recursive reference in Protobuf schema, which can not be processed by Spark by default: <fieldDescriptor>
. try setting the option recursive.fields.max.depth
0 to 10. Going beyond 10 levels of recursion is not allowed.
Failed to rename as <sourcePath>
was not found.
The <clause>
clause may be used at most once per <operation>
operation.
<sessionCatalog>
requires a single-part namespace, but got <namespace>
.
Cannot create the function <routineName>
because it already exists.
Choose a different name, drop or replace the existing function, or add the IF NOT EXISTS clause to tolerate a pre-existing function.
The function <routineName>
cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a schema and catalog, verify the current_schema() output, or qualify the name with the correct schema and catalog.
To tolerate the error on drop use DROP FUNCTION IF EXISTS.
More than one row returned by a subquery used as an expression.
Cannot create schema <schemaName>
because it already exists.
Choose a different name, drop the existing schema, or add the IF NOT EXISTS clause to tolerate pre-existing schema.
Cannot drop a schema <schemaName>
because it contains objects.
Use DROP SCHEMA ... CASCADE to drop the schema and all its objects.
The schema <schemaName>
cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog.
To tolerate the error on drop use DROP SCHEMA IF EXISTS.
The second argument of <functionName>
function needs to be an integer.
SQLSTATE: none assigned
sortBy must be used together with bucketBy.
Star (*) is not allowed in a select list when GROUP BY an ordinal position is used.
SQLSTATE: none assigned
Static partition column <staticName>
is also specified in the column list.
SQLSTATE: none assigned
Query [id = <id>
, runId = <runId>
] terminated with exception: <message>
Cannot create table or view <relationName>
because it already exists.
Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing objects.
The table or view <relationName>
cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog.
To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS.
SQLSTATE: none assigned
Task failed while writing rows to <path>
.
Cannot create the temporary view <relationName>
because it already exists.
Choose a different name, drop or replace the existing view, or add the IF NOT EXISTS clause to tolerate pre-existing views.
CREATE TEMPORARY VIEW or the corresponding Dataset APIs only accept single-part view names, but got: <actualName>
.
Cannot initialize array with <numElements>
elements of size <size>
.
Unable to acquire <requestedBytes>
bytes of memory, got <receivedBytes>
.
SQLSTATE: none assigned
Unable to convert SQL type <toType>
to Protobuf type <protobufType>
.
Unable to infer schema for <format>
. It must be specified manually.
Found the unbound parameter: <name>
. Please, fix args
and provide a mapping of the parameter to a SQL literal.
Found an unclosed bracketed comment. Please, append */ at the end of the comment.
Parameter <paramIndex>
of function <functionName>
requires the <requiredType>
type, however <inputSql>
has the type <inputType>
.
SQLSTATE: none assigned
Attempting to treat <descriptorName>
as a Message, but it was <containingType>
.
UNPIVOT requires all given <given>
expressions to be columns when no <empty>
expressions are given. These are not columns: [<expressions>
].
At least one value column needs to be specified for UNPIVOT, all columns specified as ids.
Unpivot value columns must share a least common type, some types do not: [<types>
].
All unpivot value columns must have the same size as there are value column names (<names>
).
Unrecognized SQL type <typeName>
.
Cannot infer grouping columns for GROUP BY ALL based on the select clause. Please explicitly specify the grouping columns.
A column or function parameter with name <objectName>
cannot be resolved.
For more details see UNRESOLVED_COLUMN
A field with name <fieldName>
cannot be resolved with the struct-type column <columnPath>
.
For more details see UNRESOLVED_FIELD
Cannot resolve column <objectName>
as a map key. If the key is a string literal, add the single quotes '' around it.
For more details see UNRESOLVED_MAP_KEY
Cannot resolve function <routineName>
on search path <searchPath>
.
USING column <colName>
cannot be resolved on the <side>
side of the join. The <side>
-side columns: [<suggestion>
].
Unsupported arrow type <typeName>
.
SQLSTATE: none assigned
Unsupported data source type for direct query on files: <dataSourceType>
Unsupported data type <typeName>
.
The deserializer is not supported:
For more details see UNSUPPORTED_DESERIALIZER
SQLSTATE: none assigned
Cannot create generated column <fieldName>
with generation expression <expressionStr>
because <reason>
.
SQLSTATE: none assigned
A query operator contains one or more unsupported expressions. Consider to rewrite it to avoid window functions, aggregate functions, and generator functions in the WHERE clause.
Invalid expressions: [<invalidExprSqls>
]
Expression <sqlExpr>
not supported within a window function.
The feature is not supported:
For more details see UNSUPPORTED_FEATURE
The generator is not supported:
For more details see UNSUPPORTED_GENERATOR
SQLSTATE: none assigned
grouping()/grouping_id() can only be used with GroupingSets/Cube/Rollup.
SQLSTATE: none assigned
The save mode <saveMode>
is not supported for:
For more details see UNSUPPORTED_SAVE_MODE
Unsupported subquery expression:
For more details see UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY
Literals of the type <unsupportedType>
are not supported. Supported types are <supportedTypes>
.
SQLSTATE: none assigned
You're using untyped Scala UDF, which does not have the input type information. Spark may blindly pass null to the Scala closure with primitive-type argument, and the closure will see the default value of the Java type for the null argument, e.g. udf((x: Int) => x, IntegerType)
, the result is 0 for null input. To get rid of this error, you could:
-
use typed Scala UDF APIs(without return type parameter), e.g.
udf((x: Int) => x)
. -
use Java UDF APIs, e.g.
udf(new UDF1[String, Integer] { override def call(s: String): Integer = s.length() }, IntegerType)
, if input types are all non primitive. -
set "spark.sql.legacy.allowUntypedScalaUDF" to "true" and use this API with caution.
Cannot create view <relationName>
because it already exists.
Choose a different name, drop or replace the existing object, or add the IF NOT EXISTS clause to tolerate pre-existing objects.
The view <relationName>
cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog.
To tolerate the error on drop use DROP VIEW IF EXISTS.
SQLSTATE: none assigned
writeStream
can be called only on streaming Dataset/DataFrame.
SQLSTATE: none assigned
The operation <operation>
requires a <requiredType>
. But <objectName>
is a <foundType>
. Use <alternative>
instead.
The <functionName>
requires <expectedNum>
parameters but the actual number is <actualNum>
.
For more details see WRONG_NUM_ARGS