Skip to content

Commit

Permalink
[SPARK-23484][SS] Fix possible race condition in KafkaContinuousReader
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

var `KafkaContinuousReader.knownPartitions` should be threadsafe as it is accessed from multiple threads - the query thread at the time of reader factory creation, and the epoch tracking thread at the time of `needsReconfiguration`.

## How was this patch tested?

Existing tests.

Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes apache#20655 from tdas/SPARK-23484.
  • Loading branch information
tdas committed Feb 21, 2018
1 parent e836c27 commit 3fd0ccb
Showing 1 changed file with 1 addition and 1 deletion.
Expand Up @@ -66,7 +66,7 @@ class KafkaContinuousReader(
// Initialized when creating reader factories. If this diverges from the partitions at the latest
// offsets, we need to reconfigure.
// Exposed outside this object only for unit tests.
private[sql] var knownPartitions: Set[TopicPartition] = _
@volatile private[sql] var knownPartitions: Set[TopicPartition] = _

override def readSchema: StructType = KafkaOffsetReader.kafkaSchema

Expand Down

0 comments on commit 3fd0ccb

Please sign in to comment.