Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests/realcluster: support local real cluster test #7578

Merged
merged 9 commits into from
Dec 21, 2023
Merged

tests/realcluster: support local real cluster test #7578

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c1c0566
support local test
HuSharp Dec 19, 2023
23685b9
rename tiup
HuSharp Dec 19, 2023
e94b1a6
Merge remote-tracking branch 'upstream/master' into support_local_test
HuSharp Dec 21, 2023
93fcfa1
check all components
HuSharp Dec 21, 2023
69ea471
add annotation
HuSharp Dec 21, 2023
7a77e32
add comment
HuSharp Dec 21, 2023
786c681
Merge branch 'support_local_test' of github.com:HuSharp/pd into suppo…
HuSharp Dec 21, 2023
87095d6
fix test
HuSharp Dec 21, 2023
6f35718
Merge branch 'master' into support_local_test
ti-chi-bot[bot] Dec 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 8 additions & 8 deletions tests/integrations/realcluster/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,24 +30,24 @@ tidy:
git diff go.mod go.sum | cat
git diff --quiet go.mod go.sum

check: deploy test kill_tiup
check: deploy test kill_cluster

deploy: kill_tiup
deploy: kill_cluster
@ echo "deploying..."
./deploy.sh
@ echo "wait tiup cluster ready..."
@ echo "wait cluster ready..."
./wait_tiup.sh 15 20
@ echo "check cluster status..."
@ pid=$$(ps -ef | grep 'tiup' | grep -v grep | awk '{print $$2}' | head -n 1); \
@ pid=$$(ps -ef | grep 'playground' | grep -v grep | awk '{print $$2}' | head -n 1); \
echo $$pid;

kill_tiup:
@ echo "kill tiup..."
@ pid=$$(ps -ef | grep 'tiup' | grep -v grep | awk '{print $$2}' | head -n 1); \
kill_cluster:
@ echo "kill cluster..."
@ pid=$$(ps -ef | grep 'playground' | grep -v grep | awk '{print $$2}' | head -n 1); \
if [ ! -z "$$pid" ]; then \
echo $$pid; \
kill $$pid; \
echo "waiting for tiup to exit..."; \
echo "waiting for cluster to exit..."; \
sleep 10; \
fi

Expand Down
15 changes: 11 additions & 4 deletions tests/integrations/realcluster/deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,16 @@ curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh
$TIUP_BIN_DIR update playground

cd ../../..
# Run TiUP
$TIUP_BIN_DIR playground nightly --kv 3 --tiflash 1 --db 1 --pd 3 --without-monitor \
--pd.binpath ./bin/pd-server --kv.binpath ./bin/tikv-server --db.binpath ./bin/tidb-server --tiflash.binpath ./bin/tiflash --tag pd_test \
> $CUR_PATH/playground.log 2>&1 &
if [ ! -d "bin" ] || [ ! -e "bin/tikv-server" ] && [ ! -e "bin/tidb-server" ] && [ ! -e "bin/pd-server" ] && [ ! -e "bin/tiflash" ]; then
color-green "downloading binaries..."
color-green "this may take a few minutes, you can also download them manually and put them in the bin directory."
$TIUP_BIN_DIR playground nightly --kv 3 --tiflash 1 --db 1 --pd 3 --without-monitor --tag pd_test \
> $CUR_PATH/playground.log 2>&1 &
else
color-green "using existing binaries..."
$TIUP_BIN_DIR playground nightly --kv 3 --tiflash 1 --db 1 --pd 3 --without-monitor \
--pd.binpath ./bin/pd-server --kv.binpath ./bin/tikv-server --db.binpath ./bin/tidb-server --tiflash.binpath ./bin/tiflash --tag pd_test \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the user need to move tidb/tikv/tiflash binaries to the current bin path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, we download the binaries via the ci repo https://github.com/PingCAP-QE/ci/blob/main/pipelines/tikv/pd/latest/pull_integration_realcluster_test.groovy#L57- L65 in pd/bin to download the binary.

This approach reduces the cost of learning for developers. However, these packages can only be downloaded by the ci, which means that it is not possible to run locally.

To solve this problem, I would like to load the packages via tiup instead of downloading them by self.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Maybe we provider some flags to specify bin paths for other components?

Copy link
Member Author

@HuSharp HuSharp Dec 21, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether we need to support this feature
I think both methods start the test in essentially the same way?

  • cp binary file to directory
  • specific binpath flags

cc @JmPotato

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, It's both OK to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could keep the current implementation since it's most used in the CI environment.

> $CUR_PATH/playground.log 2>&1 &
fi

cd $CUR_PATH
5 changes: 4 additions & 1 deletion tests/integrations/realcluster/reboot_pd_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,13 @@ func TestReloadLabel(t *testing.T) {

resp, _ := pdHTTPCli.GetStores(ctx)
setStore := resp.Stores[0]
re.Empty(setStore.Store.Labels, nil)
// TiFlash labels will be ["engine": "tiflash"]
storeLabel := map[string]string{
"zone": "zone1",
}
for _, label := range setStore.Store.Labels {
storeLabel[label.Key] = label.Value
}
err := pdHTTPCli.SetStoreLabels(ctx, setStore.Store.ID, storeLabel)
re.NoError(err)

Expand Down