-
Notifications
You must be signed in to change notification settings - Fork 996
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add kvserver tests #63
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: andylokandy <andylokandy@hotmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
8 tasks
196d205
to
ff13b4b
Compare
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Connor1996
pushed a commit
that referenced
this pull request
Apr 30, 2020
support basic functionality Prewrite/Commit/Rollback/Get/Scan Fill more implementations support coprocessor request. It's runnable now. Most of the code is copied from mocktikv. fix build support concurrent writer support more commands fix bugs support multiple regions and auto split. only resolve lock in region range remove tableScan and indexScan limit Avoid creating iterator too many times. support near seek optimize codec fix check prewrite conflict Use ValidForPrefix avoids conflict. And add message 'try again later' will make the error retryable in TiDB. fix delete range assert fix lock bug and support GC 1. lock key should be raw key rather than mvcc key, or resolve lock may send to another region. 2. support GC. fix scan lock is ignored. Should check pair.Err before check pair.Key. fix rollback command and update options change storage model After this change, lock and latest version is stored together with the same key, old version has a different prefix. During scan, we don't need to traverse every old version. GC is not implement yet. fix and optimize resolve lock fix prewrite keys support GC fix GC bug fix panic rename package name to fork fix build The NewChunk has been removed in TiDB. serialize write operation always use file-io for value log use a wake up channel to save CPU usage. fix reverse scan and limit scan count. rename lock/unlock. Delete instead of SetEntry if UserMeta is maixedDelFlag fix gc panic do not trace already committed error fix panic reduce prewrite conflict possibility and fix resolve lock panic extract acquire lock function and log latency refactor for efficient Resolve request. address comment user defer unlock write requests in batch to avoid exceed max transaction size. And provide an option to set the max transaction size. fix lock bug and implement efficient ScanLock 4k entries in a batch makes prewrite idempotent fix table scan and index scan bugs. makes commit request idempotent fix reverse scan use getter to avoid creating iterator in BatchGet close the DB when received signal. If we set sync write to false, the data may be corrupted if we do not close DB. fix GC bug (#25) * fix GC bug fix the lock unresolvable bug. (#26) * fix lock unresolvable remove key encoding (#27) * remove mv keys, always use raw key fix split bug after removing key encoding (#29) This bug will cause a single region as big as the whole database. Should use region.startKey instead of region.meta.StartKey in split check. return ErrLocked instead of ErrRetryable in Prewrite (#30) introduce lockstore (#31) * introduce lockstore A in-memory lock-free data-structure to store lock. - allows single writer and multiple readers without lock. - memory can be reused after all the entries in a block are deleted. * manual inline getNext add an command line flog to set value log path. (#32) kv: tiny clean up RUnlock and RLock might cost more then a memory allocation, let's keep things simple lock less scope avoid holding txnKeysMu for long time (#35) * avoid holding txnKeysMu for long time move lock out of LSM tree (#36) * move lock out of LSM tree. fix panic introduced by DBReader refactoring (#37) tiny fix Make code more clear Update mvcc.go move acquireLatches method from MVCCStore to regionCtx clear magic number (#41) move write batch out of latch scope (#42) move write batch construction out of latch scope (#43) Tiny refactor (#44) check `badger.ErrKeyNotFound` error returned by `txn.Get`. (#45) fix duplicate prewrite request caused panic. (#47) The lock store only supports insert and delete, if there is a same key, insert would fail and panic. When we meet a duplicate prewrite, we should not write the lock store. avoid `txn.Get` in `Commit` if there is no old version. (#46) unify names (#48) unify iterator style (#49) fix timestamp comparation in KvGet (#50) rename LockStore to MemStore (#51) typo (#52) fix bugs found in high conflicting concurrent transaction. (#53) 1. Fix `Cleanup` mistakenly write the batch to DB instead of lock store and use separate write batch type for Lock and DB to prevent error like this. 2. `Rollback` function should read all keys in lock store before read DB. 3. Fix delete key do actual delete instead of write tombstone. make conflict errors retryable (#54) fix unused import library (#55) fix WaitGroup (#56) After `WaitGroup.Wait` is called, another `Add` call will cause panic, so call `Add` in the Lock to prevent this. * call done after epoch not match split rollbackGC batch (#57) In TPCC benchmark, the number of rollback keys in one minutes are very large, up to hundreds of thousands. It increases latency a lot. Split the rollbackGC batch into small batches to avoid high latency. Make map with capacity (#58) preallocate slices (#59) make source more readable (#60) make it compile with latest tidb (#61) config: add max-procs (#63) support chunk execution (#64) * support chunk execution inline errors.Trace call explicitly (#65) unify Scan function to use closure and clean up code (#66) specify startKey and endKey in iterator options (#67) use Value instead of ValueCopy (#69) clear db reader (#70) optimize count(*) (#71) * optimize count(*) fix compile issue (#72) rename buf to cols (#73) fix build with latest tipb (#74) fix build for removing types.DatumRow (#75) implement closure executor (#76) * implement closure executor handle end key checking implicitly and remove the checking out of hot… (#77) support more executors in closure executor (#78) support the new row format (#79) clean up logs (#80) optimize and refactor encoding and add benchmark (#81) add old value to lock (#82) Avoid one DB.Get in Commit operation. continue handle request if necessary (#84) Rename and cleanup (#85) * use multiple badger DB * pre-split more regions and use 8 DBs * rename project name fix build error and use dep to manage dependencies (#86) * fix build error and use dep to manage dependencies add a flag to control shard-key feature (#88) update dep (#90) add topn closure executor (#89) * add topn closure executor update badger version (#93) use go mod (#94) fix memory leak in lock store (#95) move startTS and commitTS from mvccValue to UserMeta. (#96) fix crash when setting shardKey to false (#97) implement GCCompactionFilter (#98) Disable sharding by default (#99) fix delete panic (#100) upgrade mod (#101)
Connor1996
added a commit
that referenced
this pull request
Apr 30, 2020
* add kvserver tests Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.