Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add some checks to Split/Unite #275

Merged
merged 3 commits into from
Aug 14, 2023
Merged

add some checks to Split/Unite #275

merged 3 commits into from
Aug 14, 2023

Conversation

diPhantxm
Copy link
Collaborator

@diPhantxm diPhantxm commented Aug 11, 2023 •
edited
Loading

Unite:

  • check shardID is the same on left and right key ranges
  • check the UpperBound on the left key range matches LowerBound on the right key range

Split:

  • check that bound is not less than lower bound
  • check that bound is not greater than or equal to upper bound

@diPhantxm diPhantxm marked this pull request as ready for review August 11, 2023 14:56
@diPhantxm diPhantxm changed the title add some checks to Unite in coordinator add some checks to Split/Unite Aug 11, 2023
Denchick
Denchick requested changes Aug 11, 2023
View reviewed changes
test/feature/spqr_test.go Outdated Show resolved Hide resolved
test/feature/features/coordinator.feature Show resolved Hide resolved
test/feature/features/coordinator.feature Show resolved Hide resolved
test/feature/features/coordinator.feature Show resolved Hide resolved
coordinator/provider/coordinator.go Outdated Show resolved Hide resolved
@Denchick Denchick merged commit d8718ea into pg-sharding:master Aug 14, 2023
12 checks passed
CBists pushed a commit to CBists/spqr that referenced this pull request Nov 9, 2023
@EinKrebs @CBists
Fix release job (pg-sharding#247)
5a0cad0 
* Fix release

* Get tag in Makefile

Create coordinator.md (pg-sharding#225)

* Create coordinator.md

* update docs

* fix typo

---------

Co-authored-by: denchick <denchick@yandex-team.ru>

Run each feature in seperate job (pg-sharding#268)

Unite admin-console/psql-console pre-routing logic. (pg-sharding#259)

* Unite admin-console/psql-console pre-routing logic.

essentially fixes pg-sharding#255

* Fix tests

Update Router.md (pg-sharding#210)

* Update Router.md

resolves pg-sharding#201

* fix typo

---------

Co-authored-by: denchick <denchick@yandex-team.ru>

fix init.sql + backup.json logic (pg-sharding#262)

* fix init.sql + backup logic

* fix docs

* add new scenario

* fix spelling

Coordinator Show clients/pools/backend_connections funcs (pg-sharding#265)

* implement pools for coordinator
* `show pools` & `show backend connections ` now shows router names
* added units for coord pool
* added feature for show pools etc.

Show router status in `SHOW ROUTERS` command response (pg-sharding#266)

* Show router status in `SHOW ROUTERS` command responce

```

spqr-consolds=> show routers;
       show routers       | status
--------------------------+--------
 router r1-localhost:7010 | OPENED
(1 row)

```

* Linter fix

* Fix expected router out

* fixes

---------

Co-authored-by: denchick <denchick@yandex-team.ru>

Fix dataraces in memqdb (pg-sharding#260)

* fix datarace with memqdb.go?

* fix linter + ci

* add missing methods

* resolve conflicts

* fix deadlock: sharekey with unlockkey

* fix deadlock: dropkeyrange with unlockkeyrange

* fix Makefile

* refactor method

* fix comments

* fix error message

fix double lock key range on router when moving data (pg-sharding#271)

fix lock/unlock key range wrong namespace in qdb (pg-sharding#270)

* fix lock/unlock key range in coordinator

* fix e2e tests

test sync router after registration in coordinator (pg-sharding#267)

* test sync router after registration in coordinator

* check shard id in sync router test scenario

* fix sql result comparison in coordinator feature tests

Upload to existing release instead of creating a new one (pg-sharding#273)

add some checks to Split/Unite (pg-sharding#275)

* add some checks to UniteKeyRange in coordinator

* add some checks on bound to Split

* minor test scenarios titles fixes

Refactor QDB interfaces, add comments to code (pg-sharding#274)

check if sharding rule already exists, implement CheckKeyRangeLocked in etcdQDB (pg-sharding#278)

fix key range intersection checks (pg-sharding#276)

* fix moving key range to the same shardID

* fix adding key range with existing krid by splitting

* fix qdb key range intersection checks in qdb operations

* minor pr fixes

* fix tests

* extended tests

* cannot split range by lower or upper bound because range cannot be of 1 point

* fix comment

ping router before adding to qdb (pg-sharding#277)

* ping router before adding to qdb

* wrap ping router error

bump dependencies (pg-sharding#281)

fix coordinator crash when moving key range twice (pg-sharding#282)

Single coord (pg-sharding#280)

* locking coordinator

* made transactions atomic

* added tests

* fixed tests and linters

* fixed features

* enhansed coordinator lock

* fix

* requested changes

fix missing log level on router reload (pg-sharding#284)

Support X protocol in frontend interactions: Part 1 (pg-sharding#286)

* Support X protocol in frontend interactions

* Add mockgen and add test to Check TSA

* Add unit test to frontend function

* Fix linter

* Enhance makefile

* Bump lyx

* Comment frontend test as there is work to do, to make in work

Fix typo in spqr_test.go (pg-sharding#289)

Refactor Client, Relay and PoolMgr interfaces (pg-sharding#288)

* Refactor Client, Relay and PoolMgr interfaces

* Add test

* Add copy test

Parallelize scenarios (pg-sharding#285)

* Execute scenarios in feature in parallel
* not do second split

* cache images

from wal-g

* try without docker install

* don't build spqr-shard-image if docker/shard

doesn't changes

* reuse running jobs

* revert "cache images"

* refactor

add checks to move key range (pg-sharding#283)

* add checks to move key range

* change shardID from sh3 to non-existent in move feature test

Run regress tests on 12-15 Postgresql (pg-sharding#290)

* run regress on multiple version of postgres

* stop qdb after regress

* remove 16 and 17 version

Ignore feature tests artifacts (pg-sharding#293)

Added SCRAM auth feature tests (pg-sharding#292)

* Added "I run commandon host" step

* Added .feature file

* Some fixes

* Switch back to "regress" user

* Frontend auth test

* Added "Command output should match" step

* Added psql output check

* Added backend auth test

* Moved psql installation to Dockerfile

fix possible deadlock on big move (pg-sharding#291)

Bump lyx (pg-sharding#295)

* Bump lyx

* Regen

make gogen (pg-sharding#296)

Add gram rules for strart trace request (pg-sharding#297)

Run coordinator together with router (pg-sharding#287)

Dispatch coordinator goroutines with router binary start. Proxy all modifying requests to Coordinator via grpc.

Run admin console in proxy way for several type of queries:

    All modify requests to router console should be proxied to coordinator via particular gRPC-call. It is very dangerous to allow user to modify router metadata, like key-ranges or sharding rules, etc. This only should by done by coordinator or via coord. API.

    All show requests, like SHOW CLIENTS etc should be processed locally. In future, we can support installation-wide
    show requests processing using additional syntax, like SHOW CLIENTS is processed by router but SHOW GLOBAL CLIENTS to be proccesed by coordinator (coord. can process such queries already).

Regen yacc (pg-sharding#299)

Logproxy in router (pg-sharding#298)

Add workload recording infrastructure, workload recording API and bindings. Provide SQL interface to access new subsystem.

Replay test workload (pg-sharding#303)

Replay recorded workload in router.

revert config param (pg-sharding#300)

Update .gitignore (pg-sharding#304)

route begin transaction read-only to replica (pg-sharding#305)

* route begin transaction read-only to replica

* Bump lyx

* Bump

Bump lyx (pg-sharding#307)

bump lyx (pg-sharding#308)

Update main.go (pg-sharding#310)

Bump lyx (pg-sharding#311)

Xproto: extended query protocol interactions. (pg-sharding#302)

* Extended protocol interaction basic

* Fix

* Fix

* util+gomod

* Bump

* Fix

* Fix2

* Fix3

Bump lyx (pg-sharding#312)

* Bump luyx

* Fix

* Better

Added support for systemd notifications (pg-sharding#314)

* Added notifier type

* Added notifications to router instance

* Added alive notifications

* Added debug logging for systemd notifications

* Added reloading notifications support

* Added systemd notifier settings to config

* Fixed golangci-lint

Add client_encoding to router console (pg-sharding#315)

Add dataStyle to router console (pg-sharding#316)

Accept routing hints in comments (pg-sharding#317)

* Accept routing hints in comments

add dataspaces support

Change gram.y: Add DROP DATASPACE support to parser

Gen

Bugfix

Gen

delete DROP DATASPACE support

gen

Add show dataspace regress test

Small fix

fix regress tests and add tests to dataspace

Xproto: extended query protocol interactions. (pg-sharding#302)

* Extended protocol interaction basic

* Fix

* Fix

* util+gomod

* Bump

* Fix

* Fix2

* Fix3

Accept routing hints in comments (pg-sharding#317)

* Accept routing hints in comments

add dataspaces support

gen

hz

tests fix

add dataspaces support

Change gram.y: Add DROP DATASPACE support to parser

Gen

Bugfix

delete DROP DATASPACE support

fix regress tests and add tests to dataspace

Xproto: extended query protocol interactions. (pg-sharding#302)

* Extended protocol interaction basic

* Fix

* Fix

* util+gomod

* Bump

* Fix

* Fix2

* Fix3

Accept routing hints in comments (pg-sharding#317)

* Accept routing hints in comments

add dataspaces support

gen

tests fix

fix go.sum

gen

Delete .DS_Store

Delete .DS_Store files

add dataspaces support

Change gram.y: Add DROP DATASPACE support to parser

Gen

Bugfix

delete DROP DATASPACE support

Add show dataspace regress test

fix regress tests and add tests to dataspace

Xproto: extended query protocol interactions. (pg-sharding#302)

* Extended protocol interaction basic

* Fix

* Fix

* util+gomod

* Bump

* Fix

* Fix2

* Fix3

Accept routing hints in comments (pg-sharding#317)

* Accept routing hints in comments

add dataspaces support

gen

tests fix

Delete .DS_Store

gen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Denchick Denchick Denchick approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@diPhantxm @Denchick