Skip to content

Commit

Permalink
[3.4][SPARK-42229][CORE] Migrate SparkCoreErrors into error classes
Browse files Browse the repository at this point in the history 
### What changes were proposed in this pull request?

backport for apache#39791

### Why are the changes needed?

We should want to include PRs related to error classes into Spark 3.4.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

The CI should pass.

Closes apache#39832 from itholic/42229-3.4.

Authored-by: itholic <haejoon.lee@databricks.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
  • Loading branch information
@itholic @dongjoon-hyun
itholic authored and dongjoon-hyun committed Feb 1, 2023
1 parent 87ed98f commit f5abb1b
Show file tree
Hide file tree
Showing 2 changed files with 415 additions and 67 deletions.
222 changes: 222 additions & 0 deletions core/src/main/resources/error/error-classes.json
View file
Expand Up @@ -5288,5 +5288,227 @@
"message" : [
"Operation not allowed: <cmd> only works on table with location provided: <tableIdentWithDB>"
]
},
"_LEGACY_ERROR_TEMP_3000" : {
"message" : [
"Unexpected Py4J server <class>."
]
},
"_LEGACY_ERROR_TEMP_3001" : {
"message" : [
"EOFException occurred while reading the port number from <daemonModule>'s stdout<additionalMessage>."
]
},
"_LEGACY_ERROR_TEMP_3002" : {
"message" : [
"Data of type <other> is not supported"
]
},
"_LEGACY_ERROR_TEMP_3003" : {
"message" : [
"Could not compute split, block <blockId> of RDD <id> not found"
]
},
"_LEGACY_ERROR_TEMP_3004" : {
"message" : [
"Attempted to use <string> after its blocks have been removed!"
]
},
"_LEGACY_ERROR_TEMP_3005" : {
"message" : [
"Histogram on either an empty RDD or RDD containing +/-infinity or NaN"
]
},
"_LEGACY_ERROR_TEMP_3006" : {
"message" : [
"empty RDD"
]
},
"_LEGACY_ERROR_TEMP_3007" : {
"message" : [
"Checkpoint block <rddBlockId> not found! Either the executor",
"that originally checkpointed this partition is no longer alive, or the original RDD is",
"unpersisted. If this problem persists, you may consider using `rdd.checkpoint()`",
"instead, which is slower than local checkpointing but more fault-tolerant."
]
},
"_LEGACY_ERROR_TEMP_3008" : {
"message" : [
"Cannot use map-side combining with array keys."
]
},
"_LEGACY_ERROR_TEMP_3009" : {
"message" : [
"HashPartitioner cannot partition array keys."
]
},
"_LEGACY_ERROR_TEMP_3010" : {
"message" : [
"reduceByKeyLocally() does not support array keys"
]
},
"_LEGACY_ERROR_TEMP_3011" : {
"message" : [
"This RDD lacks a SparkContext. It could happen in the following cases:",
"(1) RDD transformations and actions are NOT invoked by the driver, but inside of other transformations; for example, rdd1.map(x => rdd2.values.count() * x) is invalid because the values transformation and count action cannot be performed inside of the rdd1.map transformation. For more information, see SPARK-5063.",
"(2) When a Spark Streaming job recovers from checkpoint, this exception will be hit if a reference to an RDD not defined by the streaming job is used in DStream operations. For more information, See SPARK-13758."
]
},
"_LEGACY_ERROR_TEMP_3012" : {
"message" : [
"Cannot change storage level of an RDD after it was already assigned a level"
]
},
"_LEGACY_ERROR_TEMP_3013" : {
"message" : [
"Can only zip RDDs with same number of elements in each partition"
]
},
"_LEGACY_ERROR_TEMP_3014" : {
"message" : [
"empty collection"
]
},
"_LEGACY_ERROR_TEMP_3015" : {
"message" : [
"countByValueApprox() does not support arrays"
]
},
"_LEGACY_ERROR_TEMP_3016" : {
"message" : [
"Checkpoint directory has not been set in the SparkContext"
]
},
"_LEGACY_ERROR_TEMP_3017" : {
"message" : [
"Invalid checkpoint file: <path>"
]
},
"_LEGACY_ERROR_TEMP_3018" : {
"message" : [
"Failed to create checkpoint path <checkpointDirPath>"
]
},
"_LEGACY_ERROR_TEMP_3019" : {
"message" : [
"Checkpoint RDD has a different number of partitions from original RDD. Original",
"RDD [ID: <originalRDDId>, num of partitions: <originalRDDLength>];",
"Checkpoint RDD [ID: <newRDDId>, num of partitions: <newRDDLength>]."
]
},
"_LEGACY_ERROR_TEMP_3020" : {
"message" : [
"Checkpoint dir must be specified."
]
},
"_LEGACY_ERROR_TEMP_3021" : {
"message" : [
"Error asking standalone scheduler to shut down executors"
]
},
"_LEGACY_ERROR_TEMP_3022" : {
"message" : [
"Error stopping standalone scheduler's driver endpoint"
]
},
"_LEGACY_ERROR_TEMP_3023" : {
"message" : [
"Can't run submitMapStage on RDD with 0 partitions"
]
},
"_LEGACY_ERROR_TEMP_3024" : {
"message" : [
"attempted to access non-existent accumulator <id>"
]
},
"_LEGACY_ERROR_TEMP_3025" : {
"message" : [
"TaskSetManagers should only send Resubmitted task statuses for tasks in ShuffleMapStages."
]
},
"_LEGACY_ERROR_TEMP_3026" : {
"message" : [
"duration() called on unfinished task"
]
},
"_LEGACY_ERROR_TEMP_3027" : {
"message" : [
"Unrecognized <schedulerModeProperty>: <schedulingModeConf>"
]
},
"_LEGACY_ERROR_TEMP_3028" : {
"message" : [
"<errorMsg>"
]
},
"_LEGACY_ERROR_TEMP_3029" : {
"message" : [
"Exiting due to error from cluster scheduler: <message>"
]
},
"_LEGACY_ERROR_TEMP_3030" : {
"message" : [
"Task <currentTaskAttemptId> has not locked block <blockId> for writing"
]
},
"_LEGACY_ERROR_TEMP_3031" : {
"message" : [
"Block <blockId> does not exist"
]
},
"_LEGACY_ERROR_TEMP_3032" : {
"message" : [
"Error occurred while waiting for replication to finish"
]
},
"_LEGACY_ERROR_TEMP_3033" : {
"message" : [
"Unable to register with external shuffle server due to : <message>"
]
},
"_LEGACY_ERROR_TEMP_3034" : {
"message" : [
"Error occurred while waiting for async. reregistration"
]
},
"_LEGACY_ERROR_TEMP_3035" : {
"message" : [
"Unexpected shuffle block <blockId> with unsupported shuffle resolver <shuffleBlockResolver>"
]
},
"_LEGACY_ERROR_TEMP_3036" : {
"message" : [
"Failure while trying to store block <blockId> on <blockManagerId>."
]
},
"_LEGACY_ERROR_TEMP_3037" : {
"message" : [
"Block <blockId> was not found even though it's read-locked"
]
},
"_LEGACY_ERROR_TEMP_3038" : {
"message" : [
"get() failed for block <blockId> even though we held a lock"
]
},
"_LEGACY_ERROR_TEMP_3039" : {
"message" : [
"BlockManager returned null for BlockStatus query: <blockId>"
]
},
"_LEGACY_ERROR_TEMP_3040" : {
"message" : [
"BlockManagerMasterEndpoint returned false, expected true."
]
},
"_LEGACY_ERROR_TEMP_3041" : {
"message" : [
""
]
},
"_LEGACY_ERROR_TEMP_3042" : {
"message" : [
"Failed to get block <blockId>, which is not a shuffle block"
]
}
}

0 comments on commit f5abb1b

Please sign in to comment.