Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: refactor scheduler test and move scheduler register #5957

Merged
merged 7 commits into from
Feb 14, 2023

Conversation

lhy1024
Copy link
Contributor

@lhy1024 lhy1024 commented Feb 9, 2023

Signed-off-by: lhy1024 admin@liudos.us

What problem does this PR solve?

Issue Number: Close #5956

What is changed and how does it work?

  1. refactor scheduler test by adding newTestCluster
  2. move scheduler register and reduce unnecessary register

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Release note

None.

Signed-off-by: lhy1024 <admin@liudos.us>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 9, 2023

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • nolouch
  • rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Feb 9, 2023
@lhy1024 lhy1024 mentioned this pull request Feb 9, 2023
5 tasks
@codecov
Copy link

codecov bot commented Feb 9, 2023

Codecov Report

Base: 75.15% // Head: 75.11% // Decreases project coverage by -0.05% ⚠️

Coverage data is based on head (097d873) compared to base (5e07877).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5957      +/-   ##
==========================================
- Coverage   75.15%   75.11%   -0.05%     
==========================================
  Files         362      362              
  Lines       36280    36281       +1     
==========================================
- Hits        27267    27251      -16     
- Misses       6641     6656      +15     
- Partials     2372     2374       +2     
Flag Coverage Δ
unittests 75.11% <100.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/testutil.go 90.62% <100.00%> (ø)
tests/cluster.go 79.27% <100.00%> (+0.06%) ⬆️
pkg/mcs/discovery/register.go 53.48% <0.00%> (-18.61%) ⬇️
client/resource_group/controller/limiter.go 61.25% <0.00%> (-6.25%) ⬇️
pkg/tso/local_allocator.go 64.86% <0.00%> (-4.06%) ⬇️
server/schedulers/shuffle_hot_region.go 56.66% <0.00%> (-3.34%) ⬇️
pkg/election/leadership.go 75.25% <0.00%> (-3.10%) ⬇️
pkg/election/lease.go 84.05% <0.00%> (-2.90%) ⬇️
pkg/encryption/key_manager.go 70.11% <0.00%> (-1.60%) ⬇️
pkg/tso/allocator_manager.go 63.38% <0.00%> (-0.81%) ⬇️
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Signed-off-by: lhy1024 <admin@liudos.us>
@lhy1024
Copy link
Contributor Author

lhy1024 commented Feb 9, 2023

except for #5953

# testing all pkgs...
CGO_ENABLED=1 go test -tags tso_function_test,deadlock -timeout 20m -race -cover github.com/tikv/pd/pkg/audit github.com/tikv/pd/pkg/autoscaling github.com/tikv/pd/pkg/btree github.com/tikv/pd/pkg/cache github.com/tikv/pd/pkg/cgroup github.com/tikv/pd/pkg/codec github.com/tikv/pd/pkg/core github.com/tikv/pd/pkg/core/rangetree github.com/tikv/pd/pkg/core/storelimit github.com/tikv/pd/pkg/dashboard/adapter github.com/tikv/pd/pkg/election github.com/tikv/pd/pkg/encryption github.com/tikv/pd/pkg/errs github.com/tikv/pd/pkg/gctuner github.com/tikv/pd/pkg/id github.com/tikv/pd/pkg/mcs/discovery github.com/tikv/pd/pkg/mcs/resource_manager/server github.com/tikv/pd/pkg/memory github.com/tikv/pd/pkg/mock/mockhbstream github.com/tikv/pd/pkg/movingaverage github.com/tikv/pd/pkg/progress github.com/tikv/pd/pkg/ratelimit github.com/tikv/pd/pkg/slice github.com/tikv/pd/pkg/storage github.com/tikv/pd/pkg/storage/endpoint github.com/tikv/pd/pkg/storage/kv github.com/tikv/pd/pkg/systimemon github.com/tikv/pd/pkg/utils/apiutil github.com/tikv/pd/pkg/utils/assertutil github.com/tikv/pd/pkg/utils/etcdutil github.com/tikv/pd/pkg/utils/grpcutil github.com/tikv/pd/pkg/utils/jsonutil github.com/tikv/pd/pkg/utils/keyutil github.com/tikv/pd/pkg/utils/logutil github.com/tikv/pd/pkg/utils/metricutil github.com/tikv/pd/pkg/utils/netutil github.com/tikv/pd/pkg/utils/reflectutil github.com/tikv/pd/pkg/utils/requestutil github.com/tikv/pd/pkg/utils/syncutil github.com/tikv/pd/pkg/utils/typeutil github.com/tikv/pd/server github.com/tikv/pd/server/api github.com/tikv/pd/server/cluster github.com/tikv/pd/server/config github.com/tikv/pd/server/gc github.com/tikv/pd/server/join github.com/tikv/pd/server/keyspace github.com/tikv/pd/server/region_syncer github.com/tikv/pd/server/replication github.com/tikv/pd/server/schedule github.com/tikv/pd/server/schedule/checker github.com/tikv/pd/server/schedule/filter github.com/tikv/pd/server/schedule/labeler github.com/tikv/pd/server/schedule/operator github.com/tikv/pd/server/schedule/placement github.com/tikv/pd/server/schedule/rangelist github.com/tikv/pd/server/schedulers github.com/tikv/pd/server/statistics github.com/tikv/pd/server/statistics/buckets github.com/tikv/pd/tests/autoscaling github.com/tikv/pd/tests/compatibility github.com/tikv/pd/tests/dashboard github.com/tikv/pd/tests/pdbackup github.com/tikv/pd/tests/pdctl github.com/tikv/pd/tests/pdctl/cluster github.com/tikv/pd/tests/pdctl/completion github.com/tikv/pd/tests/pdctl/config github.com/tikv/pd/tests/pdctl/health github.com/tikv/pd/tests/pdctl/hot github.com/tikv/pd/tests/pdctl/label github.com/tikv/pd/tests/pdctl/log github.com/tikv/pd/tests/pdctl/member github.com/tikv/pd/tests/pdctl/operator github.com/tikv/pd/tests/pdctl/region github.com/tikv/pd/tests/pdctl/scheduler github.com/tikv/pd/tests/pdctl/store github.com/tikv/pd/tests/pdctl/tso github.com/tikv/pd/tests/pdctl/unsafe github.com/tikv/pd/tests/registry github.com/tikv/pd/tests/server github.com/tikv/pd/tests/server/api github.com/tikv/pd/tests/server/apiv2/handlers github.com/tikv/pd/tests/server/cluster github.com/tikv/pd/tests/server/config github.com/tikv/pd/tests/server/id github.com/tikv/pd/tests/server/join github.com/tikv/pd/tests/server/join/join_fail github.com/tikv/pd/tests/server/keyspace github.com/tikv/pd/tests/server/member github.com/tikv/pd/tests/server/region_syncer github.com/tikv/pd/tests/server/storage github.com/tikv/pd/tests/server/watch github.com/tikv/pd/tools/pd-analysis/analysis github.com/tikv/pd/tools/pd-backup/pdbackup github.com/tikv/pd/tools/pd-ctl/pdctl github.com/tikv/pd/tools/pd-simulator/simulator/simutil || { $(find $PWD/ -type d | grep -vE "\.git" | xargs failpoint-ctl disable); exit 1; }
ok  	github.com/tikv/pd/pkg/audit	(cached)	coverage: 90.0% of statements
ok  	github.com/tikv/pd/pkg/autoscaling	(cached)	coverage: 46.0% of statements
ok  	github.com/tikv/pd/pkg/btree	(cached)	coverage: 90.3% of statements
ok  	github.com/tikv/pd/pkg/cache	(cached)	coverage: 70.8% of statements
--- FAIL: TestGetCgroupCPU (0.00s)
    cgroup_cpu_test.go:45: 
        	Error Trace:	/home/lhy1024/pd/pkg/cgroup/cgroup_cpu_test.go:45
        	Error:      	Received unexpected error:
        	            	no cpu controller detected
        	            	github.com/tikv/pd/pkg/cgroup.getCgroupCPUHelper
        	            		/home/lhy1024/pd/pkg/cgroup/cgroup_cpu.go:33
        	            	github.com/tikv/pd/pkg/cgroup.GetCgroupCPU
        	            		/home/lhy1024/pd/pkg/cgroup/cgroup_cpu_linux.go:28
        	            	github.com/tikv/pd/pkg/cgroup.TestGetCgroupCPU
        	            		/home/lhy1024/pd/pkg/cgroup/cgroup_cpu_test.go:44
        	            	testing.tRunner
        	            		/usr/local/go/src/testing/testing.go:1446
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1594
        	Test:       	TestGetCgroupCPU
[2023/02/09 19:17:12.326 +08:00] [WARN] [cgroup_memory.go:84] ["no cgroup memory controller detected"]
[2023/02/09 19:17:12.341 +08:00] [WARN] [cgroup_memory.go:55] ["no cgroup memory controller detected"]
[2023/02/09 19:17:12.357 +08:00] [WARN] [cgroup_memory.go:115] ["no cgroup memory controller detected"]
FAIL
coverage: 76.5% of statements
FAIL	github.com/tikv/pd/pkg/cgroup	0.121s
ok  	github.com/tikv/pd/pkg/codec	(cached)	coverage: 63.0% of statements
ok  	github.com/tikv/pd/pkg/core	(cached)	coverage: 39.4% of statements
ok  	github.com/tikv/pd/pkg/core/rangetree	(cached)	coverage: 58.5% of statements
ok  	github.com/tikv/pd/pkg/core/storelimit	(cached)	coverage: 72.2% of statements
ok  	github.com/tikv/pd/pkg/dashboard/adapter	(cached)	coverage: 22.1% of statements
ok  	github.com/tikv/pd/pkg/election	16.594s	coverage: 63.0% of statements
ok  	github.com/tikv/pd/pkg/encryption	18.303s	coverage: 73.6% of statements
ok  	github.com/tikv/pd/pkg/errs	(cached)	coverage: 85.7% of statements
ok  	github.com/tikv/pd/pkg/gctuner	(cached)	coverage: 76.9% of statements
ok  	github.com/tikv/pd/pkg/id	2.377s	coverage: 72.3% of statements
ok  	github.com/tikv/pd/pkg/mcs/discovery	8.148s	coverage: 63.3% of statements
ok  	github.com/tikv/pd/pkg/mcs/resource_manager/server	(cached)	coverage: 29.7% of statements
ok  	github.com/tikv/pd/pkg/memory	(cached)	coverage: 32.6% of statements [no tests to run]
ok  	github.com/tikv/pd/pkg/mock/mockhbstream	(cached)	coverage: 85.7% of statements
ok  	github.com/tikv/pd/pkg/movingaverage	(cached)	coverage: 88.8% of statements
ok  	github.com/tikv/pd/pkg/progress	(cached)	coverage: 88.5% of statements
ok  	github.com/tikv/pd/pkg/ratelimit	(cached)	coverage: 95.4% of statements
ok  	github.com/tikv/pd/pkg/slice	(cached)	coverage: 66.7% of statements
ok  	github.com/tikv/pd/pkg/storage	2.026s	coverage: 47.2% of statements
ok  	github.com/tikv/pd/pkg/storage/endpoint	(cached)	coverage: 3.2% of statements
ok  	github.com/tikv/pd/pkg/storage/kv	1.289s	coverage: 71.9% of statements
ok  	github.com/tikv/pd/pkg/systimemon	(cached)	coverage: 100.0% of statements
ok  	github.com/tikv/pd/pkg/utils/apiutil	(cached)	coverage: 20.5% of statements
ok  	github.com/tikv/pd/pkg/utils/assertutil	(cached)	coverage: 36.4% of statements
ok  	github.com/tikv/pd/pkg/utils/etcdutil	6.571s	coverage: 60.9% of statements
ok  	github.com/tikv/pd/pkg/utils/grpcutil	(cached)	coverage: 43.9% of statements
ok  	github.com/tikv/pd/pkg/utils/jsonutil	(cached)	coverage: 80.0% of statements
ok  	github.com/tikv/pd/pkg/utils/keyutil	(cached)	coverage: 14.3% of statements
ok  	github.com/tikv/pd/pkg/utils/logutil	(cached)	coverage: 64.0% of statements
ok  	github.com/tikv/pd/pkg/utils/metricutil	(cached)	coverage: 89.7% of statements
ok  	github.com/tikv/pd/pkg/utils/netutil	(cached)	coverage: 85.0% of statements
ok  	github.com/tikv/pd/pkg/utils/reflectutil	(cached)	coverage: 90.3% of statements
ok  	github.com/tikv/pd/pkg/utils/requestutil	(cached)	coverage: 30.0% of statements
ok  	github.com/tikv/pd/pkg/utils/syncutil	(cached)	coverage: 87.0% of statements
ok  	github.com/tikv/pd/pkg/utils/typeutil	(cached)	coverage: 77.1% of statements
ok  	github.com/tikv/pd/server	21.224s	coverage: 15.1% of statements
ok  	github.com/tikv/pd/server/api	217.952s	coverage: 63.2% of statements
ok  	github.com/tikv/pd/server/cluster	(cached)	coverage: 58.6% of statements
ok  	github.com/tikv/pd/server/config	(cached)	coverage: 47.3% of statements
ok  	github.com/tikv/pd/server/gc	(cached)	coverage: 95.8% of statements
ok  	github.com/tikv/pd/server/join	0.680s	coverage: 4.7% of statements
ok  	github.com/tikv/pd/server/keyspace	(cached)	coverage: 71.7% of statements
ok  	github.com/tikv/pd/server/region_syncer	(cached)	coverage: 29.7% of statements
ok  	github.com/tikv/pd/server/replication	(cached)	coverage: 79.3% of statements
ok  	github.com/tikv/pd/server/schedule	(cached)	coverage: 68.9% of statements
ok  	github.com/tikv/pd/server/schedule/checker	(cached)	coverage: 73.6% of statements
ok  	github.com/tikv/pd/server/schedule/filter	(cached)	coverage: 59.2% of statements
ok  	github.com/tikv/pd/server/schedule/labeler	(cached)	coverage: 86.6% of statements
ok  	github.com/tikv/pd/server/schedule/operator	(cached)	coverage: 72.2% of statements
ok  	github.com/tikv/pd/server/schedule/placement	(cached)	coverage: 77.2% of statements
ok  	github.com/tikv/pd/server/schedule/rangelist	(cached)	coverage: 100.0% of statements
ok  	github.com/tikv/pd/server/schedulers	73.113s	coverage: 69.4% of statements
ok  	github.com/tikv/pd/server/statistics	(cached)	coverage: 52.7% of statements
ok  	github.com/tikv/pd/server/statistics/buckets	(cached)	coverage: 94.0% of statements
ok  	github.com/tikv/pd/tests/autoscaling	3.801s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/compatibility	10.354s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/dashboard	22.906s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdbackup	3.327s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl	1.621s	coverage: 11.6% of statements
ok  	github.com/tikv/pd/tests/pdctl/cluster	1.943s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/completion	(cached)	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/config	16.250s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/health	7.719s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/hot	15.296s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/label	3.490s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/log	15.229s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/member	11.877s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/operator	4.696s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/region	6.312s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/scheduler	7.440s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/store	11.774s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/tso	(cached)	coverage: [no statements]
ok  	github.com/tikv/pd/tests/pdctl/unsafe	4.262s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/registry	3.142s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server	44.518s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/api	97.398s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/apiv2/handlers	9.787s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/cluster	112.219s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/config	12.164s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/id	36.214s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/join	77.010s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/join/join_fail	4.888s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/keyspace	54.111s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/member	134.116s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/region_syncer	60.309s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/storage	12.981s	coverage: [no statements]
ok  	github.com/tikv/pd/tests/server/watch	30.133s	coverage: [no statements]
ok  	github.com/tikv/pd/tools/pd-analysis/analysis	0.273s	coverage: 75.5% of statements
ok  	github.com/tikv/pd/tools/pd-backup/pdbackup	1.603s	coverage: 77.8% of statements
ok  	github.com/tikv/pd/tools/pd-ctl/pdctl	(cached)	coverage: 12.1% of statements
ok  	github.com/tikv/pd/tools/pd-simulator/simulator/simutil	(cached)	coverage: 85.5% of statements
FAIL

Signed-off-by: lhy1024 <admin@liudos.us>
Signed-off-by: lhy1024 <admin@liudos.us>
server/schedulers/utils.go Outdated Show resolved Hide resolved
Copy link
Member

@rleungx rleungx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 13, 2023
Signed-off-by: lhy1024 <admin@liudos.us>
Signed-off-by: lhy1024 <admin@liudos.us>
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Feb 14, 2023
@lhy1024
Copy link
Contributor Author

lhy1024 commented Feb 14, 2023

/merge

@ti-chi-bot
Copy link
Member

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 9468cb6

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 14, 2023
@ti-chi-bot
Copy link
Member

@lhy1024: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit 58dbc3f into tikv:master Feb 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

scheduler: move init function
4 participants