Fix checkall when shard is large #11105

hfu94 · 2023-12-18T20:38:55Z

the begin key has to be updated for checkall command when a shard
is large, this PR makes this change.

Testing: upload fdbcli binary to ck5-p210-wu-stateless-1:/var/dynamic-conf, with name fdbcli_c2.

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

The PR has a description, explaining both the problem and the solution.
The description mentions which forms of testing were done and the testing seems reasonable.
Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

hfu94 · 2023-12-18T20:39:55Z

@jzhou77 could you please share the ckperf host and the shard you checked, so that I can make test it?

foundationdb-ci · 2023-12-18T20:47:08Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 8e37da6
Duration 0:08:00
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-12-18T20:47:32Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 8e37da6
Duration 0:08:26
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-18T20:47:54Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 8e37da6
Duration 0:08:47
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-18T20:54:30Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 8e37da6
Duration 0:15:22
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-18T21:21:55Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 8e37da6
Duration 0:42:10
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-18T23:34:13Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 62caf96
Duration 0:07:40
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-12-18T23:35:03Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 62caf96
Duration 0:08:29
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-18T23:35:35Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 62caf96
Duration 0:09:03
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-18T23:42:00Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 62caf96
Duration 0:15:26
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T00:19:57Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 62caf96
Duration 0:53:24
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T01:59:36Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 17c54c5
Duration 0:07:39
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-12-19T02:00:17Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 17c54c5
Duration 0:08:21
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T02:00:58Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 17c54c5
Duration 0:09:03
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T02:08:01Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 17c54c5
Duration 0:16:07
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T02:43:45Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 17c54c5
Duration 0:51:47
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

the begin key has to be updated for checkall command when a shard is large, this PR makes this change.

hfu94 · 2023-12-19T17:24:05Z

Fixed the previous issue, now a sample output @jzhou77

fdb> checkall \x158\x19\x02\xb9\x8a\xe4s\x15s\x00\x15\x02\x15\x08\x17Z\xc3Z\x17\x11\xea \x158\x19\x02\xb9\x8c\x1e\x18\x15/\x00\x15\x01\x15=\x0258
Start checking range: \x158\x19\x02\xb9\x8a\xe4s\x15s\x00\x15\x02\x15\x08\x17Z\xc3Z\x17\x11\xea - \x158\x19\x02\xb9\x8c\x1e\x18\x15/\x00\x15\x01\x15=\x0258
Key range: \x158\x19\x02\xb9\x8a\xe4s\x15s\x00\x15\x02\x15\x08\x17Z\xc3Z\x17\x11\xea - \x158\x19\x02\xb9\x8c\x1e\x18\x15/\x00\x15\x01\x15=\x0258
  172.23.198.199:4502:tls
  172.28.9.175:4500:tls
  172.23.132.45:4502:tls
  172.29.216.175:4500:tls
  172.29.218.163:4502:tls
  172.23.204.44:4500:tls
round 0, begin Key: 0+firstGreaterOrEqual(\\x158\\x19\\x02\\xb9\\x8a\\xe4s\\x15s\\x00\\x15\\x02\\x15\\x08\\x17Z\\xc3Z\\x17\\x11\\xea)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8a\xf4B\x15\xb8\x00\x15\x09\x15\x0d\x15\x01
AllSame 1, hasMore 1, checkAll 0
round 1, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8a\\xf4B\\x15\\xb8\\x00\\x15\\x09\\x15\\x0d\\x15\\x01)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\x09Q\x15"\x00\x15\x02\x15\x0d\x15\x10\x02CONTACT\x00\x02RjI2MDc1RTgtOTFDQS00MDg2LTkwM0MtMjkzRkYzNDkxMkQw\x00
AllSame 1, hasMore 1, checkAll 0
round 2, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\x09Q\\x15"\\x00\\x15\\x02\\x15\\x0d\\x15\\x10\\x02CONTACT\\x00\\x02RjI2MDc1RTgtOTFDQS00MDg2LTkwM0MtMjkzRkYzNDkxMkQw\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b!\xf7\x15"\x00\x15\x02\x15\x03\x15\x10
AllSame 1, hasMore 1, checkAll 0

WARNING: Long delay (Ctrl-C to interrupt)
round 3, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b!\\xf7\\x15"\\x00\\x15\\x02\\x15\\x03\\x15\\x10)

The database is available, but has issues (type 'status' for more information).

Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b,\x8e\x16\x03v\x00\x15\x02\x15\x06\x15%!\xc1\xc1T\x15i\x03\xd7\x0a\x02cc3b530f-c314-44bb-b886-465ec636daa8\x00
AllSame 1, hasMore 1, checkAll 0
round 4, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b,\\x8e\\x16\\x03v\\x00\\x15\\x02\\x15\\x06\\x15%!\\xc1\\xc1T\\x15i\\x03\\xd7\\x0a\\x02cc3b530f-c314-44bb-b886-465ec636daa8\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8bA>\x15"\x00\x15\x02\x15+\x15\x10\x00\x02QTI1MzY1RkEtQTE5Ni00NURELTlDOTUtRDk4Q0JEOTNGMjcx\x00
AllSame 1, hasMore 1, checkAll 0
round 5, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8bA>\\x15"\\x00\\x15\\x02\\x15+\\x15\\x10\\x00\\x02QTI1MzY1RkEtQTE5Ni00NURELTlDOTUtRDk4Q0JEOTNGMjcx\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8bH\xcf\x16\x03\xb8\x00\x15\x09\x15\x01\x15\x04\x01\x00\xff\x00
AllSame 1, hasMore 1, checkAll 0
round 6, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8bH\\xcf\\x16\\x03\\xb8\\x00\\x15\\x09\\x15\\x01\\x15\\x04\\x01\\x00\\xff\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8bPv\x16\x02\x00\x00\x15\x02\x15\x0a\x15/\x025FDBCADF-E513-4DBD-8ED5-E0ECAA7156F2\x00
AllSame 1, hasMore 1, checkAll 0
round 7, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8bPv\\x16\\x02\\x00\\x00\\x15\\x02\\x15\\x0a\\x15/\\x025FDBCADF-E513-4DBD-8ED5-E0ECAA7156F2\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8bpl\x15\x01\x00\x15\x02\x15\x01\x15P\x16\x0c\xa6\x15P\x02syncAnchor\x00
AllSame 1, hasMore 1, checkAll 0
round 8, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8bpl\\x15\\x01\\x00\\x15\\x02\\x15\\x01\\x15P\\x16\\x0c\\xa6\\x15P\\x02syncAnchor\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8bs\x8c\x15\x01\x00\x15\x02\x15\x01\x15\x04\x164\x9f\x15\x04\x02flBDUy1pQ2xvdWREcml2ZS10b21i\x00
AllSame 1, hasMore 1, checkAll 0
round 9, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8bs\\x8c\\x15\\x01\\x00\\x15\\x02\\x15\\x01\\x15\\x04\\x164\\x9f\\x15\\x04\\x02flBDUy1pQ2xvdWREcml2ZS10b21i\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\x82[\x15-\x00\x15\x02\x15Q\x00\x02normalizedsubject\x00\x021xncj1l\x00\x15\x96
AllSame 1, hasMore 1, checkAll 0
round 10, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\x82[\\x15-\\x00\\x15\\x02\\x15Q\\x00\\x02normalizedsubject\\x00\\x021xncj1l\\x00\\x15\\x96)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\x83\x16\x15s\x00\x15\x02\x15\x0a\x17\xe3\xd1\xff\x02/device/isLocked\x00!\xc1\xc5\x90\xf1\xc0\x00\x9a\xbb\x023FD73E67-BDEE-417E-B565-70B9D85ECF51\x00
AllSame 1, hasMore 1, checkAll 0
round 11, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\x83\\x16\\x15s\\x00\\x15\\x02\\x15\\x0a\\x17\\xe3\\xd1\\xff\\x02/device/isLocked\\x00!\\xc1\\xc5\\x90\\xf1\\xc0\\x00\\x9a\\xbb\\x023FD73E67-BDEE-417E-B565-70B9D85ECF51\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\x8e\x8c\x15"\x00\x15\x01\x15\x10\x02FB7115DA-81DD-4C5E-BD68-91E5E4489374\x00\x13\xfe
AllSame 1, hasMore 1, checkAll 0
round 12, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\x8e\\x8c\\x15"\\x00\\x15\\x01\\x15\\x10\\x02FB7115DA-81DD-4C5E-BD68-91E5E4489374\\x00\\x13\\xfe)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\x8f\xa1\x15-\x00\x15\x02\x15\x87\x00\x0231\x00\x14\x1b\x01J\x1fM\xa7\xe1\x01\x159\x022e484ccf-8238-4fdd-b796-fc563b97c40d\x00
AllSame 1, hasMore 1, checkAll 0
round 13, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\x8f\\xa1\\x15-\\x00\\x15\\x02\\x15\\x87\\x00\\x0231\\x00\\x14\\x1b\\x01J\\x1fM\\xa7\\xe1\\x01\\x159\\x022e484ccf-8238-4fdd-b796-fc563b97c40d\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\x9fj\x15r\x00\x15\x01\x159\x02FE4B6C64-D4E8-4ECB-AEB5-A8F5B171C881\x00\x13\xfe
AllSame 1, hasMore 1, checkAll 0
round 14, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\x9fj\\x15r\\x00\\x15\\x01\\x159\\x02FE4B6C64-D4E8-4ECB-AEB5-A8F5B171C881\\x00\\x13\\xfe)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\xb3\x94\x15=\x00\x15\x02\x15(\x15\x0b\x15\x013\x00\x00'3~,\xe4@\x00\x01\x00\x08\x15\x0b\x02430333C6-37D0-4055-BD53-1DAEAE7AEE4B18964\x00
AllSame 1, hasMore 1, checkAll 0
round 15, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\xb3\\x94\\x15=\\x00\\x15\\x02\\x15(\\x15\\x0b\\x15\\x013\\x00\\x00'3~,\\xe4@\\x00\\x01\\x00\\x08\\x15\\x0b\\x02430333C6-37D0-4055-BD53-1DAEAE7AEE4B18964\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\xcf\x16\x15"\x00\x15\x02\x15\x0d\x15\x10\x02CONTACT\x00\x0291710F7B-A71E-4817-95A5-D3A8CAD2119E\x00
AllSame 1, hasMore 1, checkAll 0
round 16, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\xcf\\x16\\x15"\\x00\\x15\\x02\\x15\\x0d\\x15\\x10\\x02CONTACT\\x00\\x0291710F7B-A71E-4817-95A5-D3A8CAD2119E\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\xdf\x7f\x15\x89\x00\x15\x02\x15\x0e\x15\x1e\x15\x033\x00\x005rMD\x84\x80\x00\x01\x00\x07\x15\x1e\x02meDeviceIDSIdentifier\x00
AllSame 1, hasMore 1, checkAll 0
round 17, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\xdf\\x7f\\x15\\x89\\x00\\x15\\x02\\x15\\x0e\\x15\\x1e\\x15\\x033\\x00\\x005rMD\\x84\\x80\\x00\\x01\\x00\\x07\\x15\\x1e\\x02meDeviceIDSIdentifier\\x00)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\xe8\x0e\x15H\x15>\x15\x02\x15\x02\x15\x10
AllSame 1, hasMore 1, checkAll 0
round 18, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\xe8\\x0e\\x15H\\x15>\\x15\\x02\\x15\\x02\\x15\\x10)
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8b\xfd\xa9\x15H\x00\x15\x01\x15\x1e\x02classA\x00\x14
AllSame 1, hasMore 1, checkAll 0
round 19, begin Key: 0+firstGreaterThan(\\x158\\x19\\x02\\xb9\\x8b\\xfd\\xa9\\x15H\\x00\\x15\\x01\\x15\\x1e\\x02classA\\x00\\x14)
#0  server: 172.23.198.199:4502:tls  key count: 100000, cached: 0, more: 1
#2  server: 172.23.132.45:4502:tls  key count: 100000, cached: 0, more: 1n #0 CurrentI: 92184 ReferenceI: 92184 Unique key: \x158\x19\x02\xb9\x8c\x13\x13\x15\x01\x15\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ple.security.cloudkeychainproxy3\x00\x02\x00\x15\x033\x00\x005\x91\x1b/&\x94\x00\x00\x00\x01\x01\x03\x8fU\x89\xe4\x11KE\xb1S\xdf\xa3\xae\x16\xb81\x00
 #2 CurrentI: 99999 ReferenceI: 100000 Unique key: \x158\x19\x02\xb9\x8c\x13\x13\x15\x86\x15>\x15\x01\x15\x07\x01ZQ-f\xc2\xc8G`\x9a\xd7\xa5\xb55\x1d\xfa\x09\x00\x13\xfe
#0  server: 172.23.198.199:4502:tls  key count: 100000, cached: 0, more: 1
#3  server: 172.29.216.175:4500:tls  key count: 100000, cached: 0, more: 1n #0 CurrentI: 92184 ReferenceI: 92184 Unique key: \x158\x19\x02\xb9\x8c\x13\x13\x15\x01\x15\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ple.security.cloudkeychainproxy3\x00\x02\x00\x15\x033\x00\x005\x91\x1b/&\x94\x00\x00\x00\x01\x01\x03\x8fU\x89\xe4\x11KE\xb1S\xdf\xa3\xae\x16\xb81\x00
 #3 CurrentI: 99999 ReferenceI: 100000 Unique key: \x158\x19\x02\xb9\x8c\x13\x13\x15\x86\x15>\x15\x01\x15\x07\x01ZQ-f\xc2\xc8G`\x9a\xd7\xa5\xb55\x1d\xfa\x09\x00\x13\xfe
#0  server: 172.23.198.199:4502:tls  key count: 100000, cached: 0, more: 1
#4  server: 172.29.218.163:4502:tls  key count: 100000, cached: 0, more: 1n #0 CurrentI: 92184 ReferenceI: 92184 Unique key: \x158\x19\x02\xb9\x8c\x13\x13\x15\x01\x15\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ple.security.cloudkeychainproxy3\x00\x02\x00\x15\x033\x00\x005\x91\x1b/&\x94\x00\x00\x00\x01\x01\x03\x8fU\x89\xe4\x11KE\xb1S\xdf\xa3\xae\x16\xb81\x00
 #4 CurrentI: 99999 ReferenceI: 100000 Unique key: \x158\x19\x02\xb9\x8c\x13\x13\x15\x86\x15>\x15\x01\x15\x07\x01ZQ-f\xc2\xc8G`\x9a\xd7\xa5\xb55\x1d\xfa\x09\x00\x13\xfe
Warning: Consistency check was incomplete, last key of server 0 that was checked: \x158\x19\x02\xb9\x8c\x13\x13\x15\x86\x15>\x15\x01\x15\x04\x02CF0EF528E563F8EBC9B286C794E43B6F71D51A328D750BD4C97712F881A713D4\x00\x14
AllSame 0, hasMore 1, checkAll 0

hfu94 · 2023-12-19T17:25:14Z

fdbcli/DebugCommands.actor.cpp

 		const VectorRef<KeyValueRef>& result = replies[firstValidServer].get().get().data;
 		printf("Warning: Consistency check was incomplete, last key of server %d that was checked: %s\n",
 		       firstValidServer,
 		       printable(result[result.size() - 1].key).c_str());
-		begin = firstGreaterThan(result[result.size() - 1].key);
+		lastEnd = result[result.size() - 1].key; // store to a standalone, otherwise memory will be invalid


@jzhou77 this is needed, because result is from replies, and replies will be clear at the begin of each round, so I fix it this way, please let me know if you have a better idea

foundationdb-ci · 2023-12-19T17:31:13Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 83dbc5e
Duration 0:07:34
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-12-19T17:31:38Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 83dbc5e
Duration 0:08:01
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T17:32:35Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 83dbc5e
Duration 0:09:02
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T17:39:17Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 83dbc5e
Duration 0:15:43
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T18:15:52Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 83dbc5e
Duration 0:52:18
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

liquid-helium

LGTM overall, thanks for the fix!

fdbcli/DebugCommands.actor.cpp

foundationdb-ci · 2023-12-19T20:40:09Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 8761ee4
Duration 0:08:24
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-12-19T20:40:12Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 8761ee4
Duration 0:08:26
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T20:40:56Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 8761ee4
Duration 0:09:11
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T20:48:43Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 8761ee4
Duration 0:16:56
Result: ❌ FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-12-19T21:26:53Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 8761ee4
Duration 0:55:09
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

hfu94 force-pushed the fixCheck branch from 8e37da6 to 62caf96 Compare December 18, 2023 23:26

hfu94 force-pushed the fixCheck branch from 62caf96 to 17c54c5 Compare December 19, 2023 01:51

Fix checkall when shard is large

83dbc5e

the begin key has to be updated for checkall command when a shard is large, this PR makes this change.

hfu94 force-pushed the fixCheck branch from 17c54c5 to 83dbc5e Compare December 19, 2023 17:23

hfu94 commented Dec 19, 2023

View reviewed changes

hfu94 marked this pull request as ready for review December 19, 2023 17:26

jzhou77 previously approved these changes Dec 19, 2023

View reviewed changes

jzhou77 requested a review from neethuhaneesha December 19, 2023 18:55

jzhou77 requested a review from liquid-helium December 19, 2023 18:55

liquid-helium reviewed Dec 19, 2023

View reviewed changes

fdbcli/DebugCommands.actor.cpp Show resolved Hide resolved

Change comment

8761ee4

hfu94 dismissed jzhou77’s stale review via 8761ee4 December 19, 2023 20:31

jzhou77 approved these changes Dec 19, 2023

View reviewed changes

liquid-helium approved these changes Dec 19, 2023

View reviewed changes

jzhou77 merged commit 393fef7 into apple:release-7.1 Dec 20, 2023
1 of 5 checks passed

hfu94 deleted the fixCheck branch January 10, 2024 18:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix checkall when shard is large #11105

Fix checkall when shard is large #11105

hfu94 commented Dec 18, 2023 •

edited

hfu94 commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

hfu94 commented Dec 19, 2023

hfu94 Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

liquid-helium left a comment

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

foundationdb-ci commented Dec 19, 2023

Fix checkall when shard is large #11105

Fix checkall when shard is large #11105

Conversation

hfu94 commented Dec 18, 2023 • edited

Code-Reviewer Section

For Release-Branches

hfu94 commented Dec 18, 2023

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr on Linux CentOS 7

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-macos on macOS Ventura 13.x

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr on Linux CentOS 7

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-macos on macOS Ventura 13.x

foundationdb-ci commented Dec 18, 2023

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-macos on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

hfu94 commented Dec 19, 2023

hfu94 Dec 19, 2023

Choose a reason for hiding this comment

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-macos on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

liquid-helium left a comment

Choose a reason for hiding this comment

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr on Linux CentOS 7

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-macos on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

foundationdb-ci commented Dec 19, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

hfu94 commented Dec 18, 2023 •

edited