[Snap 2828] Serialize the write ops on Column Table #1362
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 14 commits
January 8, 2019 14:11
Conflicts: core/src/dunit/scala/io/snappydata/cluster/SplitClusterDUnitTestBase.scala core/src/main/scala/org/apache/spark/sql/SnappySession.scala core/src/main/scala/org/apache/spark/sql/execution/columnar/impl/ColumnFormatRelation.scala store
Conflicts: store
Conflicts: store
suranjan
changed the title
kneeraj
reviewed
Jul 23, 2019
| @@ -370,6 +370,7 @@ trait SplitClusterDUnitTestObject extends Logging { | |||
| // val connectionURL = "jdbc:snappydata://localhost:" + locatorClientPort + "/" | |||
| val connectionURL = s"localhost:$locatorClientPort" | |||
| logInfo(s"Starting spark job using spark://$hostName:7077, connectionURL=$connectionURL") | |||
|
|
|||
There was a problem hiding this comment.
Inadvertent change looks like. Please revert if not intended.
| logDebug(s" Going to take lock on server for table ${table}," + | ||
| s" current Thread ${Thread.currentThread().getId}") | ||
| val ps = conn.prepareCall(s"VALUES sys.ACQUIRE_REGION_LOCK(?)") | ||
| ps.setString(1, "BULKWRITE_" + table) |
| JdbcExtendedUtils.getTableWithSchema(table, conn = null, Some(sqlContext.sparkSession)) | ||
| val lock = grabLock(table, schemaName, defaultConnectionProps) | ||
|
|
||
| if (lock.isInstanceOf[RegionLock]) lock.asInstanceOf[RegionLock].lock() |
There was a problem hiding this comment.
For uniformity I think you can move the lock acquisition inside grabLock method itself even for embedded mode. Cosmetic change, not insisting.
core/src/main/scala/org/apache/spark/sql/execution/columnar/impl/ColumnFormatRelation.scala
Show resolved
Hide resolved
Conflicts: core/src/main/scala/org/apache/spark/sql/SnappySession.scala store
kneeraj
approved these changes
Jul 26, 2019
added 3 commits
July 27, 2019 06:17
Conflicts: store
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this pull request
Take Write lock on table in case of Insert, PutInto, Update and Delete.
Release them on completion of operation. This is to serialize the write operations to avoid inconsistency.
Handles the smart connector case where we use procedure to take and release lock for a write operation on the table.
Patch testing
precheckin and hydra tests, manual tests.
ReleaseNotes.txt changes
(Does this change require an entry in ReleaseNotes.txt? If yes, has it been added to it?)
Other PRs
TIBCOSoftware/snappy-store#488
(Does this change require changes in other projects- store, spark, spark-jobserver, aqp? Add the links of PR of the other subprojects that are related to this change)