refactor(sharding): fix critical rebalancing issues and add docs - #562
Merged
Conversation
donhardman
commented
Jun 11, 2025
donhardman
commented
Member
- Refactor sharding rebalancing logic for failed and new nodes
- Add state management to prevent concurrent operations
- Improve error handling and recovery mechanisms
- Introduce detailed docs for core sharding components and workflows
- Enhance queue system for command dependencies and async execution
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
apt-get install -y jq 1> /dev/null 2>&1 ; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='5' rf='3' timeout='300'" >/dev/null
––– output –––
OK
––– input –––
grep 'started v' /var/log/manticore-{1,2,3,4,5}/searchd.log | cut -d\' -f3 | cut -d: -f3 | wc -l
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do echo "#{$i}"; mysql -h0 -P${i}306 -e "show create table t option force=1\G" | grep 'Create Table'; done
––– output –––
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
|
clt❌ CLT tests in test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "not found"
+ null
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
OK
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
+ | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
+ +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
donhardman
force-pushed
the
fix/auto-sharding-rebalance
branch
from
June 11, 2025 05:02
3ca0a84 to
3a498e2
Compare
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
apt-get install -y jq 1> /dev/null 2>&1 ; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='5' rf='3' timeout='300'" >/dev/null
––– output –––
OK
––– input –––
grep 'started v' /var/log/manticore-{1,2,3,4,5}/searchd.log | cut -d\' -f3 | cut -d: -f3 | wc -l
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do echo "#{$i}"; mysql -h0 -P${i}306 -e "show create table t option force=1\G" | grep 'Create Table'; done
––– output –––
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
|
clt❌ CLT tests in test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "not found"
+ null
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "cluster 'e3d112439806feecd6da4d4cb4544210' is not ready, not primary state (synced)",
+ "table": "system.test2_s2",
+ "index": "e3d112439806feecd6da4d4cb4544210:test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
donhardman
force-pushed
the
fix/auto-sharding-rebalance
branch
from
June 13, 2025 05:11
3a498e2 to
dca55e4
Compare
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
apt-get install -y jq 1> /dev/null 2>&1 ; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='5' rf='3' timeout='300'" >/dev/null
––– output –––
OK
––– input –––
grep 'started v' /var/log/manticore-{1,2,3,4,5}/searchd.log | cut -d\' -f3 | cut -d: -f3 | wc -l
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do echo "#{$i}"; mysql -h0 -P${i}306 -e "show create table t option force=1\G" | grep 'Create Table'; done
––– output –––
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2'
+ Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
|
clt❌ CLT tests in test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "not found"
+ null
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
OK
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
+ | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
+ +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/sharding/replication/create-table-3-nodes-6-shards.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done && cat /tmp/status_1306.log
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='6' rf='2'"
––– output –––
OK
––– input –––
for i in 1 2 3; do mysql -h0 -P${i}306 -e "show tables from system\G"; done | grep 'Table: system.t' | sort -V
––– output –––
OK
––– input –––
for i in 1 2 3; do mysql -h0 -P${i}306 -e "desc t\G"; done
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
- Field: id
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
- Type: bigint
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
- Properties:
- *************************** 1. row ***************************
- Field: id
- Type: bigint
- Properties:
- *************************** 1. row ***************************
- Field: id
- Type: bigint
- Properties:
––– input –––
for i in 1 2 3; do mysql -h0 -P${i}306 -e "show tables from system\G" | grep 'Table: system.t' | wc -l; done
––– output –––
OKtest/clt-tests/sharding/replication/create-table-2-nodes-4-shards.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done && cat /tmp/status_1306.log
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='4' rf='2'"
––– output –––
OK
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -e "show tables from system\G"; done | grep 'Table: system.t' | sort -V
––– output –––
OK
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -e "desc t\G"; done
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
- Field: id
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
- Type: bigint
- Properties:
- *************************** 1. row ***************************
- Field: id
- Type: bigint
- Properties:
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -e "show tables from system\G" | grep 'Table: system.t' | wc -l; done
––– output –––
OKtest/clt-tests/sharding/replication/create-table-5-nodes-60-shards.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306 3306 4306 5306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done && cat /tmp/status_1306.log
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='60' rf='3' timeout='60'"
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do mysql -h0 -P${i}306 -e "show tables from system\G"; done | grep 'Table: system.t' | sort -V
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do mysql -h0 -P${i}306 -e "desc t\G" | grep 'Create Table'; done
––– output –––
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
––– input –––
for i in 1 2 3 4 5; do mysql -h0 -P${i}306 -e "show tables from system\G" | grep 'Table: system.t' | wc -l; done
––– output –––
OKtest/clt-tests/sharding/replication/test-multi-node-sharding-and-replication.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
apt-get install -y jq 1> /dev/null 2>&1 ; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='5' rf='3' timeout='300'" >/dev/null
––– output –––
OK
––– input –––
grep 'started v' /var/log/manticore-{1,2,3,4,5}/searchd.log | cut -d\' -f3 | cut -d: -f3 | wc -l
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do echo "#{$i}"; mysql -h0 -P${i}306 -e "show create table t option force=1\G" | grep 'Create Table'; done
––– output –––
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
|
clt❌ CLT tests in test/clt-tests/core/crash-mre.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, name TEXT) SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show create table test2 option force=1"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"name": "iPhone 13 Pro"}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"name": "iPhone 14 Pro"}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT count(*) FROM test2;"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
OK
––– input –––
cp /var/log/manticore-2/query.log /tmp/query.log
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"name": "iPhone 15 Pro"}}'
––– output –––
OK
––– input –––
tail -n +$(wc -l /tmp/query.log | awk '{print $1}') /var/log/manticore-2/query.log
––– output –––
+ /* Tue Jun 17 07:19:00.567 2025 conn 117 (127.0.0.1:38346) real 0.000 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_e3d112439806feecd6da4d4cb4544210_status
- Value: primary
––– input –––
sleep 1; grep -ia 'FATAL' -A1000 /var/log/manticore-2/searchd.log
––– output –––
+ ------- FATAL: CRASH DUMP -------
+ [Tue Jun 17 07:19:00.251 2025] [ 94]
+ --- crashed SphinxAPI request dump ---
+ ABABAQAAAFl7ImNsdXN0ZXIiOiJyZXBsaWNhdGlvbiIsImlkIjoyLCJkb2MiOnsibmFtZSI6ImlQaG9uZSAxNSBQ
+ cm8ifSwidGFibGUiOiJzeXN0ZW0udGVzdDJfczAifQ==
+ --- request dump end ---
+ --- local index:�@*��desc
+ Manticore 0.0.0 f89b797f9@25061306 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)
+ Handling signal 11
+ -------------- backtrace begins here ---------------
+ Program compiled with Clang 16.0.6
+ Configured with flags: Configured with these definitions: -DDISTR_BUILD=jammy -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_JIEBA=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD=1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmysqlclient.so.21 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
+ Built on Linux x86_64 (jammy) (cross-compiled)
+ [Tue Jun 17 07:19:01.191 2025] [93] watchdog: main process 94 killed dirtily with signal 11, core dumped, will be restarted
+ [Tue Jun 17 07:19:01.191 2025] [93] watchdog: main process 170 forked ok
+ [Tue Jun 17 07:19:01.195 2025] [170] Using local time zone 'UTC'
+ [Tue Jun 17 07:19:01.196 2025] [170] starting daemon version '0.0.0 f89b797f9@25061306 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)' ...
+ [Tue Jun 17 07:19:01.196 2025] [170] listening on 127.0.0.1:2306 for mysql
+ [Tue Jun 17 07:19:01.196 2025] [170] listening on 127.0.0.1:2312 for sphinx and http(s)
+ [Tue Jun 17 07:19:01.196 2025] [170] listening on 127.0.0.1:2308 for sphinx and http(s)
+ [Tue Jun 17 07:19:01.198 2025] [176] WARNING: table 'system.test2_s0': disabled at the JSON config
+ [Tue Jun 17 07:19:01.198 2025] [176] WARNING: table 'system.test2_s0': prealloc: failed to open /var/log/manticore-2/system.test2_s0/system.test2_s0.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 07:19:01.198 2025] [176] WARNING: table 'system.test2_s1': disabled at the JSON config
+ [Tue Jun 17 07:19:01.198 2025] [176] WARNING: table 'system.test2_s1': prealloc: failed to open /var/log/manticore-2/system.test2_s1/system.test2_s1.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 07:19:01.199 2025] [176] WARNING: table 'system.test2_s2': disabled at the JSON config
+ [Tue Jun 17 07:19:01.199 2025] [176] WARNING: table 'system.test2_s2': prealloc: failed to open /var/log/manticore-2/system.test2_s2/system.test2_s2.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 07:19:01.199 2025] [178] binlog: replaying log /var/log/manticore-2/binlog/binlog.0000
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: table system.sharding_state: recovered from tid 0 to tid 11
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: replay stats: 11 commits, 1 tables
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0000; 0.0 MB in 0.000 sec
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: replaying log /var/log/manticore-2/binlog/binlog.0001
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: table system.sharding_table: recovered from tid 0 to tid 2
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: replay stats: 2 commits, 1 tables
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0001; 0.0 MB in 0.000 sec
+ [Tue Jun 17 07:19:01.200 2025] [182] binlog: replaying log /var/log/manticore-2/binlog/binlog.0002
+ [Tue Jun 17 07:19:01.200 2025] [180] binlog: table system.sharding_queue: recovered from tid 0 to tid 24
+ [Tue Jun 17 07:19:01.200 2025] [180] binlog: replay stats: 24 commits, 1 tables
+ [Tue Jun 17 07:19:01.200 2025] [180] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0002; 0.0 MB in 0.000 sec
+ [Tue Jun 17 07:19:01.201 2025] [180] binlog: replaying log /var/log/manticore-2/binlog/binlog.0003
+ [Tue Jun 17 07:19:01.201 2025] [180] binlog: table system.test2_s2: recovered from tid 0 to tid 2
+ [Tue Jun 17 07:19:01.201 2025] [180] binlog: replay stats: 2 commits, 1 tables
+ [Tue Jun 17 07:19:01.201 2025] [180] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0003; 0.0 MB in 0.000 sec
+ [Tue Jun 17 07:19:01.202 2025] [176] prereading 3 tables
+ [Tue Jun 17 07:19:01.202 2025] [176] preread 3 tables in 0.000 sec
+ [Tue Jun 17 07:19:01.202 2025] [180] WARNING: '127.0.0.1:1312': receiving failure (errno=111, msg=Connection refused)
+ [Tue Jun 17 07:19:01.202 2025] [178] WARNING: last message repeated 1 times
+ [Tue Jun 17 07:19:01.202 2025] [178] WARNING: unknown table, or wrong type of table 'system.test2_s0', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:01.202 2025] [178] WARNING: unknown table, or wrong type of table 'system.test2_s1', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:01.202 2025] [178] WARNING: unknown table, or wrong type of table 'system.test2_s2', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:03.305 2025] [176] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 07:19:03.306 2025] [176] WARNING: cluster 'replication', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 07:19:05.608 2025] [180] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 07:19:05.608 2025] [180] WARNING: cluster 'e3d112439806feecd6da4d4cb4544210', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 07:19:05.608 2025] [180] WARNING: no clusters to start
+ [Tue Jun 17 07:19:05.608 2025] [180] WARNING: 'replication' cluster [127.0.0.1:33554], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 07:19:05.608 2025] [178] WARNING: 'replication' cluster [127.0.0.1:33556], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 07:19:05.609 2025] [176] WARNING: 'replication' cluster [127.0.0.1:33590], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 07:19:05.609 2025] [178] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:33624], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:05.609 2025] [180] WARNING: 'replication' cluster [127.0.0.1:33612], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 07:19:05.609 2025] [176] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:33638], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:05.609 2025] [178] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:33648], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:05.610 2025] [182] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:33666], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 07:19:05.614 2025] [170] accepting connections
+ [Tue Jun 17 07:19:05.671 2025] [178] [BUDDY] started v3.30.2-2-g11903e 'manticore-executor /usr/share/manticore/modules/manticore-buddy/src/main.php --listen=http://127.0.0.1:2312 --threads=4 ' at http://127.0.0.1:39261
+ [Tue Jun 17 07:19:05.671 2025] [178] [BUDDY] Loaded plugins:
+ [Tue Jun 17 07:19:05.671 2025] [178] [BUDDY] core: empty-string, backup, emulate-elastic, fuzzy, create-table, create-cluster, drop, insert, alias, select, show, plugin, test, alter-column, alter-distributed-table, alter-rename-table, modify-table, knn, replace, queue, sharding, update, autocomplete, cli-table, distributed-insert, truncate, metrics
+ [Tue Jun 17 07:19:05.671 2025] [178] [BUDDY] local:
+ [Tue Jun 17 07:19:05.671 2025] [178] [BUDDY] extra:
––– input –––
cat /var/log/manticore-2/query.log
––– output –––
+ /* Tue Jun 17 07:18:56.401 2025 conn 5 (127.0.0.1:48660) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:18:56.401 2025 conn 5 (127.0.0.1:48660) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:18:56.401 2025 conn 5 (127.0.0.1:48660) real 0.000 */ JOIN CLUSTER replication AT '127.0.0.1:1312';
+ /* Tue Jun 17 07:18:57.415 2025 conn 5 (127.0.0.1:48660) real 1.014 */ /* DONE */;
+ /* Tue Jun 17 07:18:57.425 2025 conn 10 (127.0.0.1:48666) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:18:57.425 2025 conn 10 (127.0.0.1:48666) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:18:57.426 2025 conn 10 (127.0.0.1:48666) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_status';
+ /* Tue Jun 17 07:18:57.426 2025 conn 10 (127.0.0.1:48666) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:18:57.444 2025 conn 11 (127.0.0.1:48670) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:18:57.444 2025 conn 11 (127.0.0.1:48670) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:18:57.444 2025 conn 11 (127.0.0.1:48670) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_node_state';
+ /* Tue Jun 17 07:18:57.444 2025 conn 11 (127.0.0.1:48670) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:00.493 2025 conn 111 (127.0.0.1:38324) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:19:00.493 2025 conn 111 (127.0.0.1:38324) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:00.493 2025 conn 111 (127.0.0.1:38324) real 0.000 */ SHOW TABLES FROM SYSTEM;
+ /* Tue Jun 17 07:19:00.493 2025 conn 111 (127.0.0.1:38324) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:00.534 2025 conn 112 (127.0.0.1:33472) real 0.000 wall 0.000 found 0 */ SELECT * FROM system.test2_s1 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 07:19:00.534 2025 conn 113 (127.0.0.1:33482) real 0.000 wall 0.000 found 2 */ SELECT * FROM system.test2_s2 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 07:19:00.546 2025 conn 115 (127.0.0.1:33500) real 0.000 wall 0.000 found 0 */ SELECT count(*) FROM system.test2_s0 LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 07:19:00.556 2025 conn 116 (127.0.0.1:38330) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:19:00.556 2025 conn 116 (127.0.0.1:38330) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:00.556 2025 conn 116 (127.0.0.1:38330) real 0.000 */ DESC system.test2_s2 OPTION force=1;
+ /* Tue Jun 17 07:19:00.556 2025 conn 116 (127.0.0.1:38330) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:00.567 2025 conn 117 (127.0.0.1:38346) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:19:00.567 2025 conn 117 (127.0.0.1:38346) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:00.567 2025 conn 117 (127.0.0.1:38346) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Tue Jun 17 07:19:00.567 2025 conn 117 (127.0.0.1:38346) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:01.215 2025 conn 4 (127.0.0.1:38350) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 07:19:01.215 2025 conn 4 (127.0.0.1:38350) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 07:19:01.215 2025 conn 4 (127.0.0.1:38350) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Tue Jun 17 07:19:05.608 2025 conn 4 (127.0.0.1:38350) real 4.393 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
- +-------+--------+----------------+
+ ERROR 1064 (42000) at line 1: no such table 'system.test2_s2'
- | Field | Type | Properties |
- +-------+--------+----------------+
- | id | bigint | |
- | name | text | indexed stored |
- +-------+--------+----------------+
test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
- +------------------+--------------+----------------+
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards
- | Field | Type | Properties |
- +------------------+--------------+----------------+
- | id | bigint | |
- | model | text | indexed stored |
- | storage_capacity | uint | |
- | color | string | |
- | release_year | uint | |
- | price | float | |
- | discounted_price | float | |
- | sold | bool | |
- | date_added | timestamp | |
- | product_codes | mva | |
- | values | mva64 | |
- | additional_info | json | |
- | vector | float_vector | knn |
- +------------------+--------------+----------------+
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
+ ERROR 2003 (HY000): Can't connect to MySQL server on '0:1306' (111)
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/sharding/replication/create-table-3-nodes-6-shards.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306 3306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done && cat /tmp/status_1306.log
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='6' rf='2'"
––– output –––
OK
––– input –––
for i in 1 2 3; do mysql -h0 -P${i}306 -e "show tables from system\G"; done | grep 'Table: system.t' | sort -V
––– output –––
OK
––– input –––
for i in 1 2 3; do mysql -h0 -P${i}306 -e "desc t\G"; done
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
- Field: id
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
- Type: bigint
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
- Properties:
- *************************** 1. row ***************************
- Field: id
- Type: bigint
- Properties:
- *************************** 1. row ***************************
- Field: id
- Type: bigint
- Properties:
––– input –––
for i in 1 2 3; do mysql -h0 -P${i}306 -e "show tables from system\G" | grep 'Table: system.t' | wc -l; done
––– output –––
OKtest/clt-tests/sharding/replication/create-table-2-nodes-4-shards.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done && cat /tmp/status_1306.log
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='4' rf='2'"
––– output –––
OK
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -e "show tables from system\G"; done | grep 'Table: system.t' | sort -V
––– output –––
OK
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -e "desc t\G"; done
––– output –––
- *************************** 1. row ***************************
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
- Field: id
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
- Type: bigint
- Properties:
- *************************** 1. row ***************************
- Field: id
- Type: bigint
- Properties:
––– input –––
for i in 1 2; do mysql -h0 -P${i}306 -e "show tables from system\G" | grep 'Table: system.t' | wc -l; done
––– output –––
OKtest/clt-tests/sharding/replication/create-table-5-nodes-60-shards.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306 3306 4306 5306; do timeout 30 mysql -h0 -P$port -e "SHOW STATUS LIKE 'cluster_c_status'\G" > /tmp/status_$port.log && grep -q "Value: primary" /tmp/status_$port.log && echo "Port $port: Node synced"; done && cat /tmp/status_1306.log
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='60' rf='3' timeout='60'"
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do mysql -h0 -P${i}306 -e "show tables from system\G"; done | grep 'Table: system.t' | sort -V
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do mysql -h0 -P${i}306 -e "desc t\G" | grep 'Create Table'; done
––– output –––
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
––– input –––
for i in 1 2 3 4 5; do mysql -h0 -P${i}306 -e "show tables from system\G" | grep 'Table: system.t' | wc -l; done
––– output –––
OKtest/clt-tests/sharding/replication/test-multi-node-sharding-and-replication.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
apt-get install -y jq 1> /dev/null 2>&1 ; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='5' rf='3' timeout='300'" >/dev/null
––– output –––
OK
––– input –––
grep 'started v' /var/log/manticore-{1,2,3,4,5}/searchd.log | cut -d\' -f3 | cut -d: -f3 | wc -l
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do echo "#{$i}"; mysql -h0 -P${i}306 -e "show create table t option force=1\G" | grep 'Create Table'; done
––– output –––
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
|
clt❌ CLT tests in test/clt-tests/core/crash-mre.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, name TEXT) SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show create table test2 option force=1"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"name": "iPhone 13 Pro"}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"name": "iPhone 14 Pro"}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT count(*) FROM test2;"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
OK
––– input –––
cp /var/log/manticore-2/query.log /tmp/query.log
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"name": "iPhone 15 Pro"}}'
––– output –––
OK
––– input –––
tail -n +$(wc -l /tmp/query.log | awk '{print $1}') /var/log/manticore-2/query.log
––– output –––
+ /* Tue Jun 17 08:56:45.545 2025 conn 117 (127.0.0.1:41470) real 0.000 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_e3d112439806feecd6da4d4cb4544210_status
- Value: primary
––– input –––
sleep 1; grep -ia 'FATAL' -A1000 /var/log/manticore-2/searchd.log
––– output –––
+ ------- FATAL: CRASH DUMP -------
+ [Tue Jun 17 08:56:45.221 2025] [ 94]
+ --- crashed SphinxAPI request dump ---
+ ABABAQAAAFl7ImNsdXN0ZXIiOiJyZXBsaWNhdGlvbiIsImlkIjoyLCJkb2MiOnsibmFtZSI6ImlQaG9uZSAxNSBQ
+ cm8ifSwidGFibGUiOiJzeXN0ZW0udGVzdDJfczAifQ==
+ --- request dump end ---
+ --- local index:�K�!V� desc
+ Manticore 0.0.0 61de091f5@25061707 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)
+ Handling signal 11
+ -------------- backtrace begins here ---------------
+ Program compiled with Clang 16.0.6
+ Configured with flags: Configured with these definitions: -DDISTR_BUILD=jammy -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_JIEBA=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD=1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmysqlclient.so.21 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
+ Built on Linux x86_64 (jammy) (cross-compiled)
+ Stack bottom = 0x7f51cc063fc0, thread stack size = 0x20000
+ Trying manual backtrace:
+ Something wrong with thread stack, manual backtrace may be incorrect (fp=0x20000)
+ Wrong stack limit or frame pointer, manual backtrace failed (fp=0x20000, stack=0x7f51cc070000, stacksize=0x20000)
+ Trying system backtrace:
+ begin of system symbols:
+ [Tue Jun 17 08:56:46.110 2025] [93] watchdog: main process 94 killed dirtily with signal 11, core dumped, will be restarted
+ [Tue Jun 17 08:56:46.111 2025] [93] watchdog: main process 170 forked ok
+ [Tue Jun 17 08:56:46.114 2025] [170] Using local time zone 'UTC'
+ [Tue Jun 17 08:56:46.114 2025] [170] starting daemon version '0.0.0 61de091f5@25061707 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)' ...
+ [Tue Jun 17 08:56:46.114 2025] [170] listening on 127.0.0.1:2306 for mysql
+ [Tue Jun 17 08:56:46.114 2025] [170] listening on 127.0.0.1:2312 for sphinx and http(s)
+ [Tue Jun 17 08:56:46.114 2025] [170] listening on 127.0.0.1:2308 for sphinx and http(s)
+ [Tue Jun 17 08:56:46.118 2025] [178] WARNING: table 'system.test2_s0': disabled at the JSON config
+ [Tue Jun 17 08:56:46.118 2025] [178] WARNING: table 'system.test2_s0': prealloc: failed to open /var/log/manticore-2/system.test2_s0/system.test2_s0.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 08:56:46.118 2025] [178] WARNING: table 'system.test2_s1': disabled at the JSON config
+ [Tue Jun 17 08:56:46.118 2025] [178] WARNING: table 'system.test2_s1': prealloc: failed to open /var/log/manticore-2/system.test2_s1/system.test2_s1.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 08:56:46.118 2025] [178] WARNING: table 'system.test2_s2': disabled at the JSON config
+ [Tue Jun 17 08:56:46.118 2025] [178] WARNING: table 'system.test2_s2': prealloc: failed to open /var/log/manticore-2/system.test2_s2/system.test2_s2.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0000
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: table system.sharding_state: recovered from tid 0 to tid 11
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: replay stats: 11 commits, 1 tables
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0000; 0.0 MB in 0.000 sec
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0001
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: table system.sharding_table: recovered from tid 0 to tid 2
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: replay stats: 2 commits, 1 tables
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0001; 0.0 MB in 0.000 sec
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0002
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: table system.sharding_queue: recovered from tid 0 to tid 24
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: replay stats: 24 commits, 1 tables
+ [Tue Jun 17 08:56:46.119 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0002; 0.0 MB in 0.000 sec
+ [Tue Jun 17 08:56:46.120 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0003
+ [Tue Jun 17 08:56:46.120 2025] [177] binlog: table system.test2_s2: recovered from tid 0 to tid 2
+ [Tue Jun 17 08:56:46.120 2025] [177] binlog: replay stats: 2 commits, 1 tables
+ [Tue Jun 17 08:56:46.120 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0003; 0.0 MB in 0.000 sec
+ [Tue Jun 17 08:56:46.121 2025] [175] prereading 3 tables
+ [Tue Jun 17 08:56:46.121 2025] [175] preread 3 tables in 0.000 sec
+ [Tue Jun 17 08:56:46.122 2025] [176] WARNING: '127.0.0.1:1312': receiving failure (errno=111, msg=Connection refused)
+ [Tue Jun 17 08:56:46.122 2025] [177] WARNING: last message repeated 1 times
+ [Tue Jun 17 08:56:46.122 2025] [177] WARNING: unknown table, or wrong type of table 'system.test2_s0', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:46.122 2025] [177] WARNING: unknown table, or wrong type of table 'system.test2_s1', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:46.122 2025] [177] WARNING: unknown table, or wrong type of table 'system.test2_s2', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:48.225 2025] [176] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 08:56:48.225 2025] [176] WARNING: cluster 'replication', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 08:56:50.528 2025] [176] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 08:56:50.528 2025] [176] WARNING: cluster 'e3d112439806feecd6da4d4cb4544210', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 08:56:50.528 2025] [176] WARNING: no clusters to start
+ [Tue Jun 17 08:56:50.528 2025] [178] WARNING: 'replication' cluster [127.0.0.1:36580], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 08:56:50.528 2025] [175] WARNING: 'replication' cluster [127.0.0.1:36596], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 08:56:50.528 2025] [177] WARNING: 'replication' cluster [127.0.0.1:36620], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 08:56:50.528 2025] [178] WARNING: 'replication' cluster [127.0.0.1:36636], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 08:56:50.528 2025] [175] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36644], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:50.528 2025] [177] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36668], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:50.528 2025] [178] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36672], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:50.530 2025] [177] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36688], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 08:56:50.534 2025] [170] accepting connections
+ [Tue Jun 17 08:56:50.590 2025] [178] [BUDDY] started v3.30.2-3-g785d6c 'manticore-executor /usr/share/manticore/modules/manticore-buddy/src/main.php --listen=http://127.0.0.1:2312 --threads=4 ' at http://127.0.0.1:45397
+ [Tue Jun 17 08:56:50.590 2025] [178] [BUDDY] Loaded plugins:
+ [Tue Jun 17 08:56:50.590 2025] [178] [BUDDY] core: empty-string, backup, emulate-elastic, fuzzy, create-table, create-cluster, drop, insert, alias, select, show, plugin, test, alter-column, alter-distributed-table, alter-rename-table, modify-table, knn, replace, queue, sharding, update, autocomplete, cli-table, distributed-insert, truncate, metrics
+ [Tue Jun 17 08:56:50.590 2025] [178] [BUDDY] local:
+ [Tue Jun 17 08:56:50.590 2025] [178] [BUDDY] extra:
––– input –––
cat /var/log/manticore-2/query.log
––– output –––
+ /* Tue Jun 17 08:56:41.373 2025 conn 5 (127.0.0.1:56912) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:41.373 2025 conn 5 (127.0.0.1:56912) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:41.373 2025 conn 5 (127.0.0.1:56912) real 0.000 */ JOIN CLUSTER replication AT '127.0.0.1:1312';
+ /* Tue Jun 17 08:56:42.388 2025 conn 5 (127.0.0.1:56912) real 1.015 */ /* DONE */;
+ /* Tue Jun 17 08:56:42.400 2025 conn 10 (127.0.0.1:41440) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:42.400 2025 conn 10 (127.0.0.1:41440) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:42.400 2025 conn 10 (127.0.0.1:41440) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_status';
+ /* Tue Jun 17 08:56:42.400 2025 conn 10 (127.0.0.1:41440) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:42.419 2025 conn 11 (127.0.0.1:41448) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:42.419 2025 conn 11 (127.0.0.1:41448) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:42.420 2025 conn 11 (127.0.0.1:41448) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_node_state';
+ /* Tue Jun 17 08:56:42.420 2025 conn 11 (127.0.0.1:41448) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:45.470 2025 conn 111 (127.0.0.1:41458) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:45.470 2025 conn 111 (127.0.0.1:41458) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:45.470 2025 conn 111 (127.0.0.1:41458) real 0.000 */ SHOW TABLES FROM SYSTEM;
+ /* Tue Jun 17 08:56:45.470 2025 conn 111 (127.0.0.1:41458) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:45.512 2025 conn 112 (127.0.0.1:36530) real 0.000 wall 0.000 found 0 */ SELECT * FROM system.test2_s1 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 08:56:45.513 2025 conn 113 (127.0.0.1:36546) real 0.000 wall 0.000 found 2 */ SELECT * FROM system.test2_s2 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 08:56:45.524 2025 conn 115 (127.0.0.1:36556) real 0.000 wall 0.000 found 0 */ SELECT count(*) FROM system.test2_s0 LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 08:56:45.535 2025 conn 116 (127.0.0.1:41464) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:45.535 2025 conn 116 (127.0.0.1:41464) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:45.535 2025 conn 116 (127.0.0.1:41464) real 0.000 */ DESC system.test2_s2 OPTION force=1;
+ /* Tue Jun 17 08:56:45.535 2025 conn 116 (127.0.0.1:41464) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:45.545 2025 conn 117 (127.0.0.1:41470) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:45.545 2025 conn 117 (127.0.0.1:41470) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:45.545 2025 conn 117 (127.0.0.1:41470) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Tue Jun 17 08:56:45.545 2025 conn 117 (127.0.0.1:41470) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:46.136 2025 conn 4 (127.0.0.1:41472) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 08:56:46.136 2025 conn 4 (127.0.0.1:41472) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 08:56:46.136 2025 conn 4 (127.0.0.1:41472) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Tue Jun 17 08:56:50.528 2025 conn 4 (127.0.0.1:41472) real 4.391 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
- +-------+--------+----------------+
+ ERROR 1064 (42000) at line 1: no such table 'system.test2_s2'
- | Field | Type | Properties |
- +-------+--------+----------------+
- | id | bigint | |
- | name | text | indexed stored |
- +-------+--------+----------------+
test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
- +------------------+--------------+----------------+
+ ERROR 1064 (42000) at line 1: Failed to find structure from local shards or agents
- | Field | Type | Properties |
- +------------------+--------------+----------------+
- | id | bigint | |
- | model | text | indexed stored |
- | storage_capacity | uint | |
- | color | string | |
- | release_year | uint | |
- | price | float | |
- | discounted_price | float | |
- | sold | bool | |
- | date_added | timestamp | |
- | product_codes | mva | |
- | values | mva64 | |
- | additional_info | json | |
- | vector | float_vector | knn |
- +------------------+--------------+----------------+
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/sharding/replication/test-multi-node-sharding-and-replication.rec––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=3
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=4
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export INSTANCE=5
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log; fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create cluster ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
export CLUSTER_NAME=c
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "join cluster ${CLUSTER_NAME} at '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "show status like 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
apt-get install -y jq 1> /dev/null 2>&1 ; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "create table c:t(id bigint) shards='5' rf='3' timeout='300'" >/dev/null
––– output –––
OK
––– input –––
grep 'started v' /var/log/manticore-{1,2,3,4,5}/searchd.log | cut -d\' -f3 | cut -d: -f3 | wc -l
––– output –––
OK
––– input –––
for i in 1 2 3 4 5; do echo "#{$i}"; mysql -h0 -P${i}306 -e "show create table t option force=1\G" | grep 'Create Table'; done
––– output –––
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s1' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s2' local='system.t_s4' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s1' local='system.t_s2' local='system.t_s3' agent='127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
- Create Table: CREATE TABLE t type='distributed' local='system.t_s0' local='system.t_s3' local='system.t_s4' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4' agent='127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3'
+ Create Table: CREATE TABLE t type='distributed' agent='127.0.0.1:1312:system.t_s0|127.0.0.1:3312:system.t_s0|127.0.0.1:5312:system.t_s0' agent='127.0.0.1:1312:system.t_s1|127.0.0.1:2312:system.t_s1|127.0.0.1:4312:system.t_s1' agent='127.0.0.1:1312:system.t_s3|127.0.0.1:4312:system.t_s3|127.0.0.1:5312:system.t_s3' agent='127.0.0.1:2312:system.t_s2|127.0.0.1:3312:system.t_s2|127.0.0.1:4312:system.t_s2' agent='127.0.0.1:2312:system.t_s4|127.0.0.1:3312:system.t_s4|127.0.0.1:5312:system.t_s4'
|
clt❌ CLT tests in test/clt-tests/core/crash-mre.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, name TEXT) SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show create table test2 option force=1"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"name": "iPhone 13 Pro"}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"name": "iPhone 14 Pro"}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT count(*) FROM test2;"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
OK
––– input –––
cp /var/log/manticore-2/query.log /tmp/query.log
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"name": "iPhone 15 Pro"}}'
––– output –––
OK
––– input –––
tail -n +$(wc -l /tmp/query.log | awk '{print $1}') /var/log/manticore-2/query.log
––– output –––
+ /* Tue Jun 17 10:34:44.196 2025 conn 117 (127.0.0.1:43290) real 0.000 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_e3d112439806feecd6da4d4cb4544210_status
- Value: primary
––– input –––
sleep 1; grep -ia 'FATAL' -A1000 /var/log/manticore-2/searchd.log
––– output –––
+ ------- FATAL: CRASH DUMP -------
+ [Tue Jun 17 10:34:43.869 2025] [ 94]
+ --- crashed SphinxAPI request dump ---
+ ABABAQAAAFl7ImNsdXN0ZXIiOiJyZXBsaWNhdGlvbiIsImlkIjoyLCJkb2MiOnsibmFtZSI6ImlQaG9uZSAxNSBQ
+ cm8ifSwidGFibGUiOiJzeXN0ZW0udGVzdDJfczAifQ==
+ --- request dump end ---
+ --- local index:� #t��
+ Manticore 0.0.0 61de091f5@25061707 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)
+ Handling signal 11
+ -------------- backtrace begins here ---------------
+ Program compiled with Clang 16.0.6
+ [Tue Jun 17 10:34:44.850 2025] [93] watchdog: main process 94 killed dirtily with signal 11, core dumped, will be restarted
+ [Tue Jun 17 10:34:44.850 2025] [93] watchdog: main process 170 forked ok
+ [Tue Jun 17 10:34:44.854 2025] [170] Using local time zone 'UTC'
+ [Tue Jun 17 10:34:44.854 2025] [170] starting daemon version '0.0.0 61de091f5@25061707 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)' ...
+ [Tue Jun 17 10:34:44.854 2025] [170] listening on 127.0.0.1:2306 for mysql
+ [Tue Jun 17 10:34:44.854 2025] [170] listening on 127.0.0.1:2312 for sphinx and http(s)
+ [Tue Jun 17 10:34:44.854 2025] [170] listening on 127.0.0.1:2308 for sphinx and http(s)
+ [Tue Jun 17 10:34:44.857 2025] [178] WARNING: table 'system.test2_s0': disabled at the JSON config
+ [Tue Jun 17 10:34:44.857 2025] [178] WARNING: table 'system.test2_s0': prealloc: failed to open /var/log/manticore-2/system.test2_s0/system.test2_s0.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 10:34:44.857 2025] [178] WARNING: table 'system.test2_s1': disabled at the JSON config
+ [Tue Jun 17 10:34:44.857 2025] [178] WARNING: table 'system.test2_s1': prealloc: failed to open /var/log/manticore-2/system.test2_s1/system.test2_s1.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 10:34:44.857 2025] [178] WARNING: table 'system.test2_s2': disabled at the JSON config
+ [Tue Jun 17 10:34:44.857 2025] [178] WARNING: table 'system.test2_s2': prealloc: failed to open /var/log/manticore-2/system.test2_s2/system.test2_s2.lock: No such file or directory - NOT SERVING
+ [Tue Jun 17 10:34:44.857 2025] [176] binlog: replaying log /var/log/manticore-2/binlog/binlog.0000
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: table system.sharding_state: recovered from tid 0 to tid 11
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: replay stats: 11 commits, 1 tables
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0000; 0.0 MB in 0.000 sec
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: replaying log /var/log/manticore-2/binlog/binlog.0001
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: table system.sharding_table: recovered from tid 0 to tid 2
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: replay stats: 2 commits, 1 tables
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0001; 0.0 MB in 0.000 sec
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: replaying log /var/log/manticore-2/binlog/binlog.0002
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: table system.sharding_queue: recovered from tid 0 to tid 24
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: replay stats: 24 commits, 1 tables
+ [Tue Jun 17 10:34:44.858 2025] [176] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0002; 0.0 MB in 0.000 sec
+ [Tue Jun 17 10:34:44.859 2025] [176] binlog: replaying log /var/log/manticore-2/binlog/binlog.0003
+ [Tue Jun 17 10:34:44.859 2025] [176] binlog: table system.test2_s2: recovered from tid 0 to tid 2
+ [Tue Jun 17 10:34:44.859 2025] [176] binlog: replay stats: 2 commits, 1 tables
+ [Tue Jun 17 10:34:44.859 2025] [176] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0003; 0.0 MB in 0.000 sec
+ [Tue Jun 17 10:34:44.860 2025] [177] prereading 3 tables
+ [Tue Jun 17 10:34:44.860 2025] [177] preread 3 tables in 0.000 sec
+ [Tue Jun 17 10:34:44.860 2025] [176] WARNING: '127.0.0.1:1312': receiving failure (errno=111, msg=Connection refused)
+ [Tue Jun 17 10:34:44.861 2025] [175] WARNING: last message repeated 1 times
+ [Tue Jun 17 10:34:44.861 2025] [175] WARNING: unknown table, or wrong type of table 'system.test2_s0', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:44.861 2025] [175] WARNING: unknown table, or wrong type of table 'system.test2_s1', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:44.861 2025] [175] WARNING: unknown table, or wrong type of table 'system.test2_s2', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:46.964 2025] [175] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 10:34:46.964 2025] [175] WARNING: cluster 'replication', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 10:34:49.268 2025] [178] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 10:34:49.268 2025] [178] WARNING: cluster 'e3d112439806feecd6da4d4cb4544210', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Tue Jun 17 10:34:49.268 2025] [178] WARNING: no clusters to start
+ [Tue Jun 17 10:34:49.268 2025] [178] WARNING: 'replication' cluster [127.0.0.1:32804], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 10:34:49.268 2025] [176] WARNING: 'replication' cluster [127.0.0.1:55450], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 10:34:49.268 2025] [175] WARNING: 'replication' cluster [127.0.0.1:55476], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 10:34:49.268 2025] [176] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:55488], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:49.268 2025] [178] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:55506], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:49.268 2025] [175] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:55516], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:49.268 2025] [177] WARNING: 'replication' cluster [127.0.0.1:32788], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Tue Jun 17 10:34:49.271 2025] [178] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:55524], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Tue Jun 17 10:34:49.274 2025] [170] accepting connections
+ [Tue Jun 17 10:34:49.330 2025] [175] [BUDDY] started v3.30.2-4-g1686f4 'manticore-executor /usr/share/manticore/modules/manticore-buddy/src/main.php --listen=http://127.0.0.1:2312 --threads=4 ' at http://127.0.0.1:33313
+ [Tue Jun 17 10:34:49.330 2025] [175] [BUDDY] Loaded plugins:
+ [Tue Jun 17 10:34:49.330 2025] [175] [BUDDY] core: empty-string, backup, emulate-elastic, fuzzy, create-table, create-cluster, drop, insert, alias, select, show, plugin, test, alter-column, alter-distributed-table, alter-rename-table, modify-table, knn, replace, queue, sharding, update, autocomplete, cli-table, distributed-insert, truncate, metrics
+ [Tue Jun 17 10:34:49.330 2025] [175] [BUDDY] local:
+ [Tue Jun 17 10:34:49.330 2025] [175] [BUDDY] extra:
––– input –––
cat /var/log/manticore-2/query.log
––– output –––
+ /* Tue Jun 17 10:34:40.021 2025 conn 5 (127.0.0.1:43236) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:40.021 2025 conn 5 (127.0.0.1:43236) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:40.021 2025 conn 5 (127.0.0.1:43236) real 0.000 */ JOIN CLUSTER replication AT '127.0.0.1:1312';
+ /* Tue Jun 17 10:34:41.037 2025 conn 5 (127.0.0.1:43236) real 1.015 */ /* DONE */;
+ /* Tue Jun 17 10:34:41.048 2025 conn 10 (127.0.0.1:43246) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:41.048 2025 conn 10 (127.0.0.1:43246) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:41.048 2025 conn 10 (127.0.0.1:43246) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_status';
+ /* Tue Jun 17 10:34:41.048 2025 conn 10 (127.0.0.1:43246) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:41.067 2025 conn 11 (127.0.0.1:43254) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:41.067 2025 conn 11 (127.0.0.1:43254) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:41.067 2025 conn 11 (127.0.0.1:43254) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_node_state';
+ /* Tue Jun 17 10:34:41.067 2025 conn 11 (127.0.0.1:43254) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.120 2025 conn 111 (127.0.0.1:43264) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:44.120 2025 conn 111 (127.0.0.1:43264) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.120 2025 conn 111 (127.0.0.1:43264) real 0.000 */ SHOW TABLES FROM SYSTEM;
+ /* Tue Jun 17 10:34:44.120 2025 conn 111 (127.0.0.1:43264) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.163 2025 conn 112 (127.0.0.1:60954) real 0.000 wall 0.000 found 0 */ SELECT * FROM system.test2_s1 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 10:34:44.163 2025 conn 113 (127.0.0.1:60962) real 0.000 wall 0.000 found 2 */ SELECT * FROM system.test2_s2 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 10:34:44.174 2025 conn 115 (127.0.0.1:60974) real 0.000 wall 0.000 found 0 */ SELECT count(*) FROM system.test2_s0 LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Tue Jun 17 10:34:44.185 2025 conn 116 (127.0.0.1:43274) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:44.185 2025 conn 116 (127.0.0.1:43274) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.185 2025 conn 116 (127.0.0.1:43274) real 0.000 */ DESC system.test2_s2 OPTION force=1;
+ /* Tue Jun 17 10:34:44.185 2025 conn 116 (127.0.0.1:43274) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.195 2025 conn 117 (127.0.0.1:43290) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:44.195 2025 conn 117 (127.0.0.1:43290) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.196 2025 conn 117 (127.0.0.1:43290) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Tue Jun 17 10:34:44.196 2025 conn 117 (127.0.0.1:43290) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.875 2025 conn 4 (127.0.0.1:43296) real 0.000 */ select @@version_comment limit 1;
+ /* Tue Jun 17 10:34:44.875 2025 conn 4 (127.0.0.1:43296) real 0.000 */ /* DONE */;
+ /* Tue Jun 17 10:34:44.875 2025 conn 4 (127.0.0.1:43296) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Tue Jun 17 10:34:49.268 2025 conn 4 (127.0.0.1:43296) real 4.392 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
- +-------+--------+----------------+
+ ERROR 1064 (42000) at line 1: no such table 'system.test2_s2'
- | Field | Type | Properties |
- +-------+--------+----------------+
- | id | bigint | |
- | name | text | indexed stored |
- +-------+--------+----------------+
test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
Member
Author
|
BLOCKED |
6 tasks
Nick-S-2018
approved these changes
Jun 18, 2025
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/join/test-avg-select-join.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
manticore-load --drop --batch-size=1000 --threads=5 --total=10000 --init="CREATE TABLE products(id bigint, name text, category multi)" --load="INSERT INTO products(id, name, category) VALUES(<increment>, '<text/2/10>', (<array/1/3/1/1000>))" --together --drop --batch-size=10000 --threads=5 --total=100000 --init="create table reviews(product_id bigint, user_id int, rating int, comment text)" --load="insert into reviews(product_id, user_id, rating, comment) values(<int/1/10000>, <int/1/1000>, <int/1/100>, '<text/20/100>')" --quiet
––– output –––
- 5 ; 1000 ; %{NUMBER}:%{NUMBER} ; 10000 ; %{NUMBER} ; 0 ; 0 ; 0 ; 0 ; 0 ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ;
+ 5 ; 1000 ; 00:02 ; 10000 ; 4542 ; 0 ; 0 ; 0 ; 0 ; 0 ; 113.5 ; 19.5 ; 495.0 ; 495.0 ;
- 5 ; 10000 ; %{NUMBER}:%{NUMBER} ; 100000 ; %{NUMBER} ; 0 ; 0 ; 0 ; 0 ; 0 ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ;
+ 5 ; 10000 ; 00:01 ; 100000 ; 68313 ; 10 ; 10 ; 10 ; 10 ; 10 ; 641.1 ; 575.0 ; 975.0 ; 975.0 ;
––– input –––
mysql -P9306 -h0 -e "select id, count(*), avg(reviews.rating) avg_rating from products left join reviews on reviews.product_id = products.id group by id order by count(*) desc; select id, count(*) from products left join reviews on reviews.product_id = products.id group by id order by count(*) desc"
––– output –––
- | 370 | 23 | 42.73913043 |
+ | 1589 | 23 | 58.65217391 |
- | 499 | 23 | 49.86956522 |
+ | 477 | 22 | 59.81818182 |
- | 2553 | 23 | 52.26086957 |
+ | 5584 | 22 | 59.63636364 |
- | 2083 | 22 | 53.13636364 |
+ | 2632 | 22 | 38.54545455 |
- | 3197 | 22 | 65.000000 |
+ | 805 | 22 | 57.86363636 |
- | 3538 | 22 | 54.72727273 |
+ | 4982 | 22 | 50.27272727 |
- | 4848 | 22 | 49.54545455 |
+ | 2031 | 21 | 60.19047619 |
- | 2850 | 22 | 59.40909091 |
+ | 6057 | 21 | 46.66666667 |
- | 9903 | 22 | 40.90909091 |
+ | 5077 | 21 | 37.38095238 |
- | 7947 | 22 | 56.77272727 |
+ | 214 | 21 | 63.19047619 |
- | 5986 | 22 | 45.09090909 |
+ | 366 | 21 | 51.61904762 |
- | 1145 | 21 | 51.28571429 |
+ | 1498 | 21 | 55.23809524 |
- | 9245 | 21 | 46.28571429 |
+ | 5667 | 21 | 48.90476190 |
- | 9432 | 21 | 48.14285714 |
+ | 5682 | 21 | 37.14285714 |
- | 4445 | 21 | 47.28571429 |
+ | 6721 | 21 | 49.14285714 |
- | 559 | 21 | 66.61904762 |
+ | 3993 | 21 | 49.76190476 |
- | 677 | 21 | 57.47619048 |
+ | 1014 | 20 | 58.250000 |
- | 6688 | 21 | 47.47619048 |
+ | 79 | 20 | 43.100000 |
- | 5714 | 21 | 59.90476190 |
+ | 7158 | 20 | 62.400000 |
- | 124 | 20 | 48.500000 |
+ | 5162 | 20 | 57.600000 |
- | 370 | 23 |
+ | 1589 | 23 |
- | 499 | 23 |
+ | 477 | 22 |
- | 2553 | 23 |
+ | 5584 | 22 |
- | 2083 | 22 |
+ | 2632 | 22 |
- | 3197 | 22 |
+ | 805 | 22 |
- | 3538 | 22 |
+ | 4982 | 22 |
- | 4848 | 22 |
+ | 2031 | 21 |
- | 2850 | 22 |
+ | 6057 | 21 |
- | 9903 | 22 |
+ | 5077 | 21 |
- | 7947 | 22 |
+ | 214 | 21 |
- | 5986 | 22 |
+ | 366 | 21 |
- | 1145 | 21 |
+ | 1498 | 21 |
- | 9245 | 21 |
+ | 5667 | 21 |
- | 9432 | 21 |
+ | 5682 | 21 |
- | 4445 | 21 |
+ | 6721 | 21 |
- | 559 | 21 |
+ | 3993 | 21 |
- | 677 | 21 |
+ | 1014 | 20 |
- | 6688 | 21 |
+ | 79 | 20 |
- | 5714 | 21 |
+ | 7158 | 20 |
- | 124 | 20 |
+ | 5162 | 20 |
––– input –––
mysql -P9306 -h0 -e "SELECT COUNT(*) FROM products; SELECT COUNT(*) FROM reviews;"
––– output –––
OK |
clt❌ CLT tests in test/clt-tests/core/crash-mre.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, value integer) SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show create table test2 option force=1"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"value":1}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"value":2}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT count(*) FROM test2;"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
OK
––– input –––
cp /var/log/manticore-2/query.log /tmp/query.log
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"value": 5}}'
––– output –––
OK
––– input –––
tail -n +$(wc -l /tmp/query.log | awk '{print $1}') /var/log/manticore-2/query.log
––– output –––
+ /* Thu Jun 26 09:29:53.315 2025 conn 116 (127.0.0.1:50178) real 0.000 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_e3d112439806feecd6da4d4cb4544210_status
- Value: primary
––– input –––
sleep 1; grep -ia 'FATAL' -A1000 /var/log/manticore-2/searchd.log
––– output –––
+ ------- FATAL: CRASH DUMP -------
+ [Thu Jun 26 09:29:52.983 2025] [ 94]
+ --- crashed SphinxAPI request dump ---
+ ABABAQAAAEx7ImNsdXN0ZXIiOiJyZXBsaWNhdGlvbiIsImlkIjoyLCJkb2MiOnsidmFsdWUiOjV9LCJ0YWJsZSI6
+ InN5c3RlbS50ZXN0Ml9zMiJ9
+ --- request dump end ---
+ --- local index:�Xm��� desc
+ Manticore 0.0.0 bb1a8c2b6@25061710 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)
+ Handling signal 11
+ -------------- backtrace begins here ---------------
+ Program compiled with Clang 16.0.6
+ Configured with flags: Configured with these definitions: -DDISTR_BUILD=jammy -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_JIEBA=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD=1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmysqlclient.so.21 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
+ Built on Linux x86_64 (jammy) (cross-compiled)
+ Stack bottom = 0x7fd7100a3fe0, thread stack size = 0x20000
+ Trying manual backtrace:
+ Something wrong with thread stack, manual backtrace may be incorrect (fp=0x20000)
+ Wrong stack limit or frame pointer, manual backtrace failed (fp=0x20000, stack=0x7fd7100b0000, stacksize=0x20000)
+ Trying system backtrace:
+ [Thu Jun 26 09:29:53.918 2025] [93] watchdog: main process 94 killed dirtily with signal 11, core dumped, will be restarted
+ [Thu Jun 26 09:29:53.918 2025] [93] watchdog: main process 170 forked ok
+ [Thu Jun 26 09:29:53.922 2025] [170] Using local time zone 'UTC'
+ [Thu Jun 26 09:29:53.923 2025] [170] starting daemon version '0.0.0 bb1a8c2b6@25061710 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)' ...
+ [Thu Jun 26 09:29:53.923 2025] [170] listening on 127.0.0.1:2306 for mysql
+ [Thu Jun 26 09:29:53.923 2025] [170] listening on 127.0.0.1:2312 for sphinx and http(s)
+ [Thu Jun 26 09:29:53.923 2025] [170] listening on 127.0.0.1:2308 for sphinx and http(s)
+ [Thu Jun 26 09:29:53.926 2025] [177] WARNING: table 'system.test2_s0': disabled at the JSON config
+ [Thu Jun 26 09:29:53.926 2025] [177] WARNING: table 'system.test2_s0': prealloc: failed to open /var/log/manticore-2/system.test2_s0/system.test2_s0.lock: No such file or directory - NOT SERVING
+ [Thu Jun 26 09:29:53.926 2025] [177] WARNING: table 'system.test2_s1': disabled at the JSON config
+ [Thu Jun 26 09:29:53.926 2025] [177] WARNING: table 'system.test2_s1': prealloc: failed to open /var/log/manticore-2/system.test2_s1/system.test2_s1.lock: No such file or directory - NOT SERVING
+ [Thu Jun 26 09:29:53.926 2025] [177] WARNING: table 'system.test2_s2': disabled at the JSON config
+ [Thu Jun 26 09:29:53.926 2025] [177] WARNING: table 'system.test2_s2': prealloc: failed to open /var/log/manticore-2/system.test2_s2/system.test2_s2.lock: No such file or directory - NOT SERVING
+ [Thu Jun 26 09:29:53.926 2025] [179] binlog: replaying log /var/log/manticore-2/binlog/binlog.0000
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: table system.sharding_state: recovered from tid 0 to tid 11
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: replay stats: 11 commits, 1 tables
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0000; 0.0 MB in 0.000 sec
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: replaying log /var/log/manticore-2/binlog/binlog.0001
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: table system.sharding_table: recovered from tid 0 to tid 2
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: replay stats: 2 commits, 1 tables
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0001; 0.0 MB in 0.000 sec
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: replaying log /var/log/manticore-2/binlog/binlog.0002
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: table system.sharding_queue: recovered from tid 0 to tid 24
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: replay stats: 24 commits, 1 tables
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0002; 0.0 MB in 0.000 sec
+ [Thu Jun 26 09:29:53.927 2025] [179] binlog: replaying log /var/log/manticore-2/binlog/binlog.0003
+ [Thu Jun 26 09:29:53.928 2025] [179] binlog: table system.test2_s2: recovered from tid 0 to tid 2
+ [Thu Jun 26 09:29:53.928 2025] [179] binlog: replay stats: 2 commits, 1 tables
+ [Thu Jun 26 09:29:53.928 2025] [179] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0003; 0.0 MB in 0.000 sec
+ [Thu Jun 26 09:29:53.929 2025] [177] prereading 3 tables
+ [Thu Jun 26 09:29:53.929 2025] [177] preread 3 tables in 0.000 sec
+ [Thu Jun 26 09:29:53.930 2025] [177] WARNING: '127.0.0.1:1312': receiving failure (errno=111, msg=Connection refused)
+ [Thu Jun 26 09:29:53.930 2025] [178] WARNING: last message repeated 1 times
+ [Thu Jun 26 09:29:53.930 2025] [178] WARNING: unknown table, or wrong type of table 'system.test2_s0', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:53.930 2025] [178] WARNING: unknown table, or wrong type of table 'system.test2_s1', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:53.930 2025] [178] WARNING: unknown table, or wrong type of table 'system.test2_s2', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:56.034 2025] [178] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Thu Jun 26 09:29:56.034 2025] [178] WARNING: cluster 'replication', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Thu Jun 26 09:29:58.337 2025] [178] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Thu Jun 26 09:29:58.337 2025] [178] WARNING: cluster 'e3d112439806feecd6da4d4cb4544210', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Thu Jun 26 09:29:58.337 2025] [178] WARNING: no clusters to start
+ [Thu Jun 26 09:29:58.337 2025] [179] WARNING: 'replication' cluster [127.0.0.1:49546], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Thu Jun 26 09:29:58.337 2025] [176] WARNING: 'replication' cluster [127.0.0.1:49560], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Thu Jun 26 09:29:58.337 2025] [179] WARNING: 'replication' cluster [127.0.0.1:49596], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Thu Jun 26 09:29:58.337 2025] [177] WARNING: 'replication' cluster [127.0.0.1:49638], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Thu Jun 26 09:29:58.337 2025] [176] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:49642], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:58.337 2025] [179] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36150], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:58.337 2025] [176] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36166], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:58.338 2025] [179] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:36208], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Thu Jun 26 09:29:58.342 2025] [170] accepting connections
+ [Thu Jun 26 09:29:58.399 2025] [176] [BUDDY] started v3.30.2-5-g4c08a4 'manticore-executor /usr/share/manticore/modules/manticore-buddy/src/main.php --listen=http://127.0.0.1:2312 --threads=4 ' at http://127.0.0.1:33679
+ [Thu Jun 26 09:29:58.399 2025] [176] [BUDDY] Loaded plugins:
+ [Thu Jun 26 09:29:58.399 2025] [176] [BUDDY] core: empty-string, backup, emulate-elastic, fuzzy, create-table, create-cluster, drop, insert, alias, select, show, plugin, test, alter-column, alter-distributed-table, alter-rename-table, modify-table, knn, replace, queue, sharding, update, autocomplete, cli-table, distributed-insert, truncate, metrics
+ [Thu Jun 26 09:29:58.399 2025] [176] [BUDDY] local:
+ [Thu Jun 26 09:29:58.399 2025] [176] [BUDDY] extra:
––– input –––
cat /var/log/manticore-2/query.log
––– output –––
+ /* Thu Jun 26 09:29:49.137 2025 conn 5 (127.0.0.1:50126) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:49.137 2025 conn 5 (127.0.0.1:50126) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:49.137 2025 conn 5 (127.0.0.1:50126) real 0.000 */ JOIN CLUSTER replication AT '127.0.0.1:1312';
+ /* Thu Jun 26 09:29:50.152 2025 conn 5 (127.0.0.1:50126) real 1.015 */ /* DONE */;
+ /* Thu Jun 26 09:29:50.164 2025 conn 10 (127.0.0.1:50128) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:50.164 2025 conn 10 (127.0.0.1:50128) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:50.164 2025 conn 10 (127.0.0.1:50128) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_status';
+ /* Thu Jun 26 09:29:50.164 2025 conn 10 (127.0.0.1:50128) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:50.185 2025 conn 11 (127.0.0.1:50136) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:50.185 2025 conn 11 (127.0.0.1:50136) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:50.185 2025 conn 11 (127.0.0.1:50136) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_node_state';
+ /* Thu Jun 26 09:29:50.185 2025 conn 11 (127.0.0.1:50136) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.238 2025 conn 111 (127.0.0.1:50150) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:53.238 2025 conn 111 (127.0.0.1:50150) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.238 2025 conn 111 (127.0.0.1:50150) real 0.000 */ SHOW TABLES FROM SYSTEM;
+ /* Thu Jun 26 09:29:53.238 2025 conn 111 (127.0.0.1:50150) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.281 2025 conn 112 (127.0.0.1:49470) real 0.000 wall 0.000 found 0 */ SELECT * FROM system.test2_s1 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Thu Jun 26 09:29:53.281 2025 conn 113 (127.0.0.1:49482) real 0.000 wall 0.000 found 2 */ SELECT * FROM system.test2_s2 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Thu Jun 26 09:29:53.293 2025 conn 114 (127.0.0.1:49488) real 0.000 wall 0.000 found 0 */ SELECT count(*) FROM system.test2_s0 LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Thu Jun 26 09:29:53.304 2025 conn 115 (127.0.0.1:50164) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:53.304 2025 conn 115 (127.0.0.1:50164) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.304 2025 conn 115 (127.0.0.1:50164) real 0.000 */ DESC system.test2_s2 OPTION force=1;
+ /* Thu Jun 26 09:29:53.305 2025 conn 115 (127.0.0.1:50164) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.315 2025 conn 116 (127.0.0.1:50178) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:53.315 2025 conn 116 (127.0.0.1:50178) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.315 2025 conn 116 (127.0.0.1:50178) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Thu Jun 26 09:29:53.315 2025 conn 116 (127.0.0.1:50178) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.943 2025 conn 4 (127.0.0.1:50182) real 0.000 */ select @@version_comment limit 1;
+ /* Thu Jun 26 09:29:53.943 2025 conn 4 (127.0.0.1:50182) real 0.000 */ /* DONE */;
+ /* Thu Jun 26 09:29:53.944 2025 conn 4 (127.0.0.1:50182) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Thu Jun 26 09:29:58.337 2025 conn 4 (127.0.0.1:50182) real 4.393 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
- +-------+--------+------------+
+ ERROR 1064 (42000) at line 1: no such table 'system.test2_s2'
- | Field | Type | Properties |
- +-------+--------+------------+
- | id | bigint | |
- | value | uint | |
- +-------+--------+------------+
test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/join/test-avg-select-join.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
manticore-load --drop --batch-size=1000 --threads=5 --total=10000 --init="CREATE TABLE products(id bigint, name text, category multi)" --load="INSERT INTO products(id, name, category) VALUES(<increment>, '<text/2/10>', (<array/1/3/1/1000>))" --together --drop --batch-size=10000 --threads=5 --total=100000 --init="create table reviews(product_id bigint, user_id int, rating int, comment text)" --load="insert into reviews(product_id, user_id, rating, comment) values(<int/1/10000>, <int/1/1000>, <int/1/100>, '<text/20/100>')" --quiet
––– output –––
- 5 ; 1000 ; %{NUMBER}:%{NUMBER} ; 10000 ; %{NUMBER} ; 0 ; 0 ; 0 ; 0 ; 0 ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ;
+ 5 ; 1000 ; 00:02 ; 10000 ; 4473 ; 0 ; 0 ; 0 ; 0 ; 0 ; 112.6 ; 18.5 ; 495.0 ; 495.0 ;
- 5 ; 10000 ; %{NUMBER}:%{NUMBER} ; 100000 ; %{NUMBER} ; 0 ; 0 ; 0 ; 0 ; 0 ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ;
+ 5 ; 10000 ; 00:01 ; 100000 ; 67832 ; 10 ; 10 ; 10 ; 10 ; 10 ; 650.9 ; 585.0 ; 1050.0 ; 1050.0 ;
––– input –––
mysql -P9306 -h0 -e "select id, count(*), avg(reviews.rating) avg_rating from products left join reviews on reviews.product_id = products.id group by id order by count(*) desc; select id, count(*) from products left join reviews on reviews.product_id = products.id group by id order by count(*) desc"
––– output –––
- | 370 | 23 | 42.73913043 |
+ | 1589 | 23 | 58.65217391 |
- | 499 | 23 | 49.86956522 |
+ | 477 | 22 | 59.81818182 |
- | 2553 | 23 | 52.26086957 |
+ | 5584 | 22 | 59.63636364 |
- | 2083 | 22 | 53.13636364 |
+ | 2632 | 22 | 38.54545455 |
- | 3197 | 22 | 65.000000 |
+ | 805 | 22 | 57.86363636 |
- | 3538 | 22 | 54.72727273 |
+ | 4982 | 22 | 50.27272727 |
- | 4848 | 22 | 49.54545455 |
+ | 2031 | 21 | 60.19047619 |
- | 2850 | 22 | 59.40909091 |
+ | 6057 | 21 | 46.66666667 |
- | 9903 | 22 | 40.90909091 |
+ | 5077 | 21 | 37.38095238 |
- | 7947 | 22 | 56.77272727 |
+ | 214 | 21 | 63.19047619 |
- | 5986 | 22 | 45.09090909 |
+ | 366 | 21 | 51.61904762 |
- | 1145 | 21 | 51.28571429 |
+ | 1498 | 21 | 55.23809524 |
- | 9245 | 21 | 46.28571429 |
+ | 5667 | 21 | 48.90476190 |
- | 9432 | 21 | 48.14285714 |
+ | 5682 | 21 | 37.14285714 |
- | 4445 | 21 | 47.28571429 |
+ | 6721 | 21 | 49.14285714 |
- | 559 | 21 | 66.61904762 |
+ | 3993 | 21 | 49.76190476 |
- | 677 | 21 | 57.47619048 |
+ | 1014 | 20 | 58.250000 |
- | 6688 | 21 | 47.47619048 |
+ | 79 | 20 | 43.100000 |
- | 5714 | 21 | 59.90476190 |
+ | 7158 | 20 | 62.400000 |
- | 124 | 20 | 48.500000 |
+ | 5162 | 20 | 57.600000 |
- | 370 | 23 |
+ | 1589 | 23 |
- | 499 | 23 |
+ | 477 | 22 |
- | 2553 | 23 |
+ | 5584 | 22 |
- | 2083 | 22 |
+ | 2632 | 22 |
- | 3197 | 22 |
+ | 805 | 22 |
- | 3538 | 22 |
+ | 4982 | 22 |
- | 4848 | 22 |
+ | 2031 | 21 |
- | 2850 | 22 |
+ | 6057 | 21 |
- | 9903 | 22 |
+ | 5077 | 21 |
- | 7947 | 22 |
+ | 214 | 21 |
- | 5986 | 22 |
+ | 366 | 21 |
- | 1145 | 21 |
+ | 1498 | 21 |
- | 9245 | 21 |
+ | 5667 | 21 |
- | 9432 | 21 |
+ | 5682 | 21 |
- | 4445 | 21 |
+ | 6721 | 21 |
- | 559 | 21 |
+ | 3993 | 21 |
- | 677 | 21 |
+ | 1014 | 20 |
- | 6688 | 21 |
+ | 79 | 20 |
- | 5714 | 21 |
+ | 7158 | 20 |
- | 124 | 20 |
+ | 5162 | 20 |
––– input –––
mysql -P9306 -h0 -e "SELECT COUNT(*) FROM products; SELECT COUNT(*) FROM reviews;"
––– output –––
OK |
clt❌ CLT tests in test/clt-tests/core/crash-mre.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, value integer) SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "show create table test2 option force=1"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G" | grep Table | cut -d' ' -f2
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"value":1}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"value":2}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT count(*) FROM test2;"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
OK
––– input –––
cp /var/log/manticore-2/query.log /tmp/query.log
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"value": 5}}'
––– output –––
OK
––– input –––
tail -n +$(wc -l /tmp/query.log | awk '{print $1}') /var/log/manticore-2/query.log
––– output –––
+ /* Mon Jun 30 05:15:59.087 2025 conn 116 (127.0.0.1:54576) real 0.000 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status'\G"
––– output –––
- *************************** 1. row ***************************
- Counter: cluster_e3d112439806feecd6da4d4cb4544210_status
- Value: primary
––– input –––
sleep 1; grep -ia 'FATAL' -A1000 /var/log/manticore-2/searchd.log
––– output –––
+ ------- FATAL: CRASH DUMP -------
+ [Mon Jun 30 05:15:58.764 2025] [ 94]
+ --- crashed SphinxAPI request dump ---
+ ABABAQAAAEx7ImNsdXN0ZXIiOiJyZXBsaWNhdGlvbiIsImlkIjoyLCJkb2MiOnsidmFsdWUiOjV9LCJ0YWJsZSI6
+ InN5c3RlbS50ZXN0Ml9zMiJ9
+ --- request dump end ---
+ --- local index:%0��#�
+ Manticore 0.0.0 bb1a8c2b6@25061710 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)
+ Handling signal 11
+ -------------- backtrace begins here ---------------
+ Program compiled with Clang 16.0.6
+ Configured with flags: Configured with these definitions: -DDISTR_BUILD=jammy -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_JIEBA=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD=1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmysqlclient.so.21 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
+ Built on Linux x86_64 (jammy) (cross-compiled)
+ Stack bottom = 0x7fbdc00a3fe0, thread stack size = 0x20000
+ Trying manual backtrace:
+ Something wrong with thread stack, manual backtrace may be incorrect (fp=0x20000)
+ Wrong stack limit or frame pointer, manual backtrace failed (fp=0x20000, stack=0x7fbdc00b0000, stacksize=0x20000)
+ Trying system backtrace:
+ begin of system symbols:
+ [Mon Jun 30 05:15:59.715 2025] [93] watchdog: main process 94 killed dirtily with signal 11, core dumped, will be restarted
+ [Mon Jun 30 05:15:59.716 2025] [93] watchdog: main process 170 forked ok
+ [Mon Jun 30 05:15:59.719 2025] [170] Using local time zone 'UTC'
+ [Mon Jun 30 05:15:59.720 2025] [170] starting daemon version '0.0.0 bb1a8c2b6@25061710 (columnar 5.0.2 3813c56@25060612) (secondary 5.0.2 3813c56@25060612) (knn 5.0.2 3813c56@25060612) (embeddings 1.0.0)' ...
+ [Mon Jun 30 05:15:59.720 2025] [170] listening on 127.0.0.1:2306 for mysql
+ [Mon Jun 30 05:15:59.720 2025] [170] listening on 127.0.0.1:2312 for sphinx and http(s)
+ [Mon Jun 30 05:15:59.720 2025] [170] listening on 127.0.0.1:2308 for sphinx and http(s)
+ [Mon Jun 30 05:15:59.723 2025] [176] WARNING: table 'system.test2_s0': disabled at the JSON config
+ [Mon Jun 30 05:15:59.723 2025] [176] WARNING: table 'system.test2_s0': prealloc: failed to open /var/log/manticore-2/system.test2_s0/system.test2_s0.lock: No such file or directory - NOT SERVING
+ [Mon Jun 30 05:15:59.723 2025] [176] WARNING: table 'system.test2_s1': disabled at the JSON config
+ [Mon Jun 30 05:15:59.723 2025] [176] WARNING: table 'system.test2_s1': prealloc: failed to open /var/log/manticore-2/system.test2_s1/system.test2_s1.lock: No such file or directory - NOT SERVING
+ [Mon Jun 30 05:15:59.723 2025] [176] WARNING: table 'system.test2_s2': disabled at the JSON config
+ [Mon Jun 30 05:15:59.723 2025] [176] WARNING: table 'system.test2_s2': prealloc: failed to open /var/log/manticore-2/system.test2_s2/system.test2_s2.lock: No such file or directory - NOT SERVING
+ [Mon Jun 30 05:15:59.723 2025] [176] binlog: replaying log /var/log/manticore-2/binlog/binlog.0000
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: table system.sharding_state: recovered from tid 0 to tid 11
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replay stats: 11 commits, 1 tables
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0000; 0.0 MB in 0.000 sec
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0001
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: table system.sharding_table: recovered from tid 0 to tid 2
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replay stats: 2 commits, 1 tables
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0001; 0.0 MB in 0.000 sec
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0002
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: table system.sharding_queue: recovered from tid 0 to tid 24
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replay stats: 24 commits, 1 tables
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0002; 0.0 MB in 0.000 sec
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replaying log /var/log/manticore-2/binlog/binlog.0003
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: table system.test2_s2: recovered from tid 0 to tid 2
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: replay stats: 2 commits, 1 tables
+ [Mon Jun 30 05:15:59.724 2025] [177] binlog: finished replaying /var/log/manticore-2/binlog/binlog.0003; 0.0 MB in 0.000 sec
+ [Mon Jun 30 05:15:59.725 2025] [179] prereading 3 tables
+ [Mon Jun 30 05:15:59.725 2025] [179] preread 3 tables in 0.000 sec
+ [Mon Jun 30 05:15:59.726 2025] [177] WARNING: '127.0.0.1:1312': receiving failure (errno=111, msg=Connection refused)
+ [Mon Jun 30 05:15:59.726 2025] [179] WARNING: last message repeated 1 times
+ [Mon Jun 30 05:15:59.726 2025] [179] WARNING: unknown table, or wrong type of table 'system.test2_s0', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:15:59.726 2025] [179] WARNING: unknown table, or wrong type of table 'system.test2_s1', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:15:59.726 2025] [179] WARNING: unknown table, or wrong type of table 'system.test2_s2', removed from cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:16:01.829 2025] [177] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Mon Jun 30 05:16:01.829 2025] [177] WARNING: cluster 'replication', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Mon Jun 30 05:16:04.132 2025] [179] WARNING: '127.0.0.1:1312': retries limit exceeded
+ [Mon Jun 30 05:16:04.132 2025] [179] WARNING: cluster 'e3d112439806feecd6da4d4cb4544210', no nodes available(127.0.0.1:1312,127.0.0.1:2312), error: '127.0.0.1:1312': retries limit exceeded
+ [Mon Jun 30 05:16:04.132 2025] [179] WARNING: no clusters to start
+ [Mon Jun 30 05:16:04.132 2025] [176] WARNING: 'replication' cluster [127.0.0.1:46880], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Mon Jun 30 05:16:04.132 2025] [177] WARNING: 'replication' cluster [127.0.0.1:46884], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Mon Jun 30 05:16:04.132 2025] [178] WARNING: 'replication' cluster [127.0.0.1:46916], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Mon Jun 30 05:16:04.132 2025] [179] WARNING: 'replication' cluster [127.0.0.1:46948], cmd: get_nodes(6), error: unknown cluster 'replication'
+ [Mon Jun 30 05:16:04.132 2025] [176] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:46956], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:16:04.132 2025] [177] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:46990], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:16:04.132 2025] [178] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:46974], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:16:04.134 2025] [179] WARNING: 'e3d112439806feecd6da4d4cb4544210' cluster [127.0.0.1:47006], cmd: get_nodes(6), error: unknown cluster 'e3d112439806feecd6da4d4cb4544210'
+ [Mon Jun 30 05:16:04.138 2025] [170] accepting connections
+ [Mon Jun 30 05:16:04.194 2025] [178] [BUDDY] started v3.30.2-6-g055605 'manticore-executor /usr/share/manticore/modules/manticore-buddy/src/main.php --listen=http://127.0.0.1:2312 --threads=4 ' at http://127.0.0.1:42445
+ [Mon Jun 30 05:16:04.194 2025] [178] [BUDDY] Loaded plugins:
+ [Mon Jun 30 05:16:04.194 2025] [178] [BUDDY] core: empty-string, backup, emulate-elastic, fuzzy, create-table, create-cluster, drop, insert, alias, select, show, plugin, test, alter-column, alter-distributed-table, alter-rename-table, modify-table, knn, replace, queue, sharding, update, autocomplete, cli-table, distributed-insert, truncate, metrics
+ [Mon Jun 30 05:16:04.194 2025] [178] [BUDDY] local:
+ [Mon Jun 30 05:16:04.194 2025] [178] [BUDDY] extra:
––– input –––
cat /var/log/manticore-2/query.log
––– output –––
+ /* Mon Jun 30 05:15:54.916 2025 conn 5 (127.0.0.1:54540) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:54.916 2025 conn 5 (127.0.0.1:54540) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:54.916 2025 conn 5 (127.0.0.1:54540) real 0.000 */ JOIN CLUSTER replication AT '127.0.0.1:1312';
+ /* Mon Jun 30 05:15:55.931 2025 conn 5 (127.0.0.1:54540) real 1.015 */ /* DONE */;
+ /* Mon Jun 30 05:15:55.942 2025 conn 10 (127.0.0.1:54542) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:55.942 2025 conn 10 (127.0.0.1:54542) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:55.942 2025 conn 10 (127.0.0.1:54542) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_status';
+ /* Mon Jun 30 05:15:55.943 2025 conn 10 (127.0.0.1:54542) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:55.962 2025 conn 11 (127.0.0.1:54550) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:55.962 2025 conn 11 (127.0.0.1:54550) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:55.962 2025 conn 11 (127.0.0.1:54550) real 0.000 */ SHOW STATUS LIKE 'cluster_replication_node_state';
+ /* Mon Jun 30 05:15:55.962 2025 conn 11 (127.0.0.1:54550) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.013 2025 conn 111 (127.0.0.1:54562) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:59.013 2025 conn 111 (127.0.0.1:54562) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.013 2025 conn 111 (127.0.0.1:54562) real 0.000 */ SHOW TABLES FROM SYSTEM;
+ /* Mon Jun 30 05:15:59.013 2025 conn 111 (127.0.0.1:54562) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.054 2025 conn 112 (127.0.0.1:46814) real 0.000 wall 0.000 found 0 */ SELECT * FROM system.test2_s1 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Mon Jun 30 05:15:59.054 2025 conn 113 (127.0.0.1:46828) real 0.000 wall 0.000 found 2 */ SELECT * FROM system.test2_s2 ORDER BY id asc LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Mon Jun 30 05:15:59.066 2025 conn 114 (127.0.0.1:46830) real 0.000 wall 0.000 found 0 */ SELECT count(*) FROM system.test2_s0 LIMIT 1000 OPTION retry_count=0, retry_delay=0;
+ /* Mon Jun 30 05:15:59.076 2025 conn 115 (127.0.0.1:54572) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:59.076 2025 conn 115 (127.0.0.1:54572) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.076 2025 conn 115 (127.0.0.1:54572) real 0.000 */ DESC system.test2_s2 OPTION force=1;
+ /* Mon Jun 30 05:15:59.076 2025 conn 115 (127.0.0.1:54572) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.087 2025 conn 116 (127.0.0.1:54576) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:59.087 2025 conn 116 (127.0.0.1:54576) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.087 2025 conn 116 (127.0.0.1:54576) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Mon Jun 30 05:15:59.087 2025 conn 116 (127.0.0.1:54576) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.740 2025 conn 4 (127.0.0.1:54592) real 0.000 */ select @@version_comment limit 1;
+ /* Mon Jun 30 05:15:59.740 2025 conn 4 (127.0.0.1:54592) real 0.000 */ /* DONE */;
+ /* Mon Jun 30 05:15:59.740 2025 conn 4 (127.0.0.1:54592) real 0.000 */ SHOW STATUS LIKE 'cluster_e3d112439806feecd6da4d4cb4544210_status';
+ /* Mon Jun 30 05:16:04.132 2025 conn 4 (127.0.0.1:54592) real 4.391 */ /* DONE */;
––– input –––
mysql -h0 -P2306 -e 'DESC system.test2_s2 OPTION force=1'
––– output –––
- +-------+--------+------------+
+ ERROR 1064 (42000) at line 1: no such table 'system.test2_s2'
- | Field | Type | Properties |
- +-------+--------+------------+
- | id | bigint | |
- | value | uint | |
- +-------+--------+------------+
test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/bugs/3481-http-update-distributed-table-wrong-cluster-crash.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd --logdebugv > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Manticore started!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -v -P9306 -h0 -e "drop table if exists t; create table t(f int); drop table if exists d; CREATE TABLE d type='distributed' agent='127.0.0.1:9312:t'"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:9312/update -d '{"cluster": "unknown_cluster", "table": "d", "id": 2, "doc": {"f": 5}}'
––– output –––
- {"table":"d","id":2,"result":"noop"}
+ {"error":{"type":"action_request_validation_exception","reason":"","table":"d"},"status":409}
––– input –––
sleep 2; cat /var/log/manticore/searchd.log | grep 'response data' | cut -d' ' -f11-
––– output –––
- {"version":3,"type":"json response","message":{"error":{"type":"action_request_validation_exception","reason":"table t: table 't' is not in any cluster, use just 't'","table":"t"},"status":409},"meta":null,"error_code":0}
+ data: {"version":3,"type":"json response","message":{"error":{"type":"action_request_validation_exception","reason":"","table":"d"},"status":409},"meta":null,"error_code":0}
|
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/core/test-knn-id.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "create table test ( title text, image_vector float_vector knn_type='hnsw' knn_dims='4' hnsw_similarity='l2' );"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "insert into test values ( 1, 'yellow bag', (0.653448,0.192478,0.017971,0.339821) ), ( 2, 'white bag', (-0.148894,0.748278,0.091892,-0.095406) );"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select id, knn_dist() from test where knn ( image_vector, 5, (0.286569,-0.031816,0.066684,0.032926), {ef=2000} );"
––– output –––
- +------+------------+
+ ERROR 1064 (42000) at line 1: P01: syntax error, unexpected '{', expecting integer or '-' near '{ef=2000} )'
- | id | knn_dist() |
- +------+------------+
- | 1 | 0.28146550 |
- | 2 | 0.81527930 |
- +------+------------+
––– input –––
mysql -h0 -P9306 -e "select id, knn_dist() from test where knn ( image_vector, 5, 1 );"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select * from test where knn ( image_vector, 5, 1 );"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select count(*) from test where knn ( image_vector, 5, 1 );"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE knn_test (id BIGINT, model TEXT, color TEXT, sold BOOL, date_added BIGINT, storage_capacity UINT, release_year UINT, price FLOAT, discounted_price FLOAT, product_codes JSON, values JSON, additional_info JSON, vector float_vector knn_type='hnsw' knn_dims='4' hnsw_similarity='l2');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO knn_test (id, model, color, sold, date_added, storage_capacity, release_year, price, discounted_price, product_codes, values, additional_info, vector) VALUES (1, 'Model_1', 'Color_1', 1, 1620590570251, 512, 2022, 654.19, 855.34, '[81,56,60]', '[96,389,27]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.1, 0.2, 0.3, 0.4)), (2, 'Model_2', 'Color_2', 1, 1689374130160, 128, 2020, 663.70, 767.99, '[12,39,30]', '[226,305,123]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.286569, -0.031816, 0.066684, 0.032926)), (3, 'Model_3', 'Color_3', 0, 1640782021529, 512, 2020, 676.46, 847.08, '[62,13,4]', '[707,601,608]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (-0.5, 0.7, -0.1, 0.2)), (4, 'Model_4', 'Color_4', 0, 1615319697581, 128, 2020, 424.23, 883.90, '[98,96,89]', '[166,252,510]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.7, -0.8, 0.1, -0.2)), (5, 'Model_5', 'Color_5', 1, 1671157942660, 128, 2021, 1173.29, 977.33, '[27,12,37]', '[988,87,21]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.3, 0.4, 0.5, 0.6)), (6, 'Model_6', 'Color_6', 0, 1609876342571, 256, 2021, 755.99, 665.49, '[5,15,25]', '[15,100,500]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (-0.2, 0.4, -0.6, 0.8)), (7, 'Model_7', 'Color_7', 1, 1648765432160, 64, 2019, 299.99, 199.99, '[77,88,99]', '[99,888,555]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.5, -0.3, 0.9, -0.1)), (8, 'Model_8', 'Color_8', 0, 1654876543130, 256, 2019, 800.99, 699.99, '[22,33,44]', '[66,333,999]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.8, 0.6, 0.2, -0.4)), (9, 'Model_9', 'Color_9', 1, 1631567882541, 128, 2022, 499.99, 399.99, '[11,22,33]', '[111,222,333]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (-0.1, 0.3, -0.5, 0.7)), (10, 'Model_10', 'Color_10', 0, 1678956543120, 512, 2022, 1299.99, 1099.99, '[9,8,7]', '[555,444,333]', '{\"features\": [\"Feature_1\", \"Feature_2\", \"Feature_3\"]}', (0.9, 0.1, 0.8, -0.2));"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, knn_dist() FROM knn_test WHERE knn(vector, 5, 1);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, knn_dist() FROM knn_test WHERE knn(vector, 5, 1);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM knn_test WHERE knn(vector, 5, 1);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT count(*) FROM knn_test WHERE knn(vector, 5, 1);"
––– output –––
OKtest/clt-tests/core/tets-insert-knn-errors.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE knn_test (id BIGINT, model TEXT, vector float_vector)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO knn_test (id, model, vector) VALUES (1, 'Model_1', '[0.9012, 0.2126, 0.2879, 0.7552]')"; echo $?
––– output –––
- ERROR 1064 (42000) at line 1: row 1, column 3: incompatible value specified for a float vector column
+ 0
- 1
––– input –––
mysql -h0 -P9306 -e "SELECT count(*) FROM knn_test WHERE knn(vector, 10, 1)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE knn_test"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE knn_test2 (id BIGINT, model TEXT, vector float_vector)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO knn_test2 (id, model, vector) VALUES (1, 'Model_1', (0.286569,-0.031816,0.066684,0.032926))"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT count(*) FROM knn_test2 WHERE knn(vector, 10, 1)"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE knn_test2"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE knn_test3 (id BIGINT, model TEXT, vector float_vector)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO knn_test3 (id, model, vector) VALUES (1, 'Model_1', '[0.9012, 0.2126, 0.2879, 0.7552]')"; echo $?
––– output –––
- ERROR 1064 (42000) at line 1: row 1, column 3: incompatible value specified for a float vector column
+ 0
- 1
––– input –––
mysql -h0 -P9306 -e "SELECT count(*) FROM knn_test3 WHERE knn(vector, 10, 1)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE knn_test3"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE knn_test4 (id BIGINT, model TEXT, vector int)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO knn_test4 (id, model, vector) VALUES (1, 'Model_1', '[0.9012, 0.2126, 0.2879, 0.7552]')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT count(*) FROM knn_test4 WHERE knn(vector, 10, 1)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE knn_test4"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE knn_test5 (id BIGINT, model TEXT, vector float_vector knn_type='hnsw' knn_dims='4' hnsw_similarity='l2')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO knn_test5 (id, model, vector) VALUES (1, 'Model_1', (0.9012, 0.2126, 0.2879, 0.7552))"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT count(*) FROM knn_test5 WHERE knn(vector, 10, 1)"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE knn_test5"
––– output –––
OKtest/clt-tests/core/show-threads.rec––– input –––
stdbuf -oL searchd --cpustats > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW THREADS\G;"
––– output –––
- CPU activity: %{NUMBER}%
+ CPU activity: 9.89%
test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
- Refactor sharding rebalancing logic for failed and new nodes - Add state management to prevent concurrent operations - Improve error handling and recovery mechanisms - Introduce detailed docs for core sharding components and workflows - Enhance queue system for command dependencies and async execution
- Return empty local table set when replication factor > 1 to avoid incorrect local entries - Include all replicas in agent entries for replicated shards to ensure proper mirroring - Exclude current node from agents only when replication factor = 1 to prevent self-referencing - Pass replication factor to local table builder for accurate behavior based on replication setup
donhardman
force-pushed
the
fix/auto-sharding-rebalance
branch
from
July 7, 2025 07:31
0556050 to
1862d08
Compare
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/buddy/test-system-variables-error-handling.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE IF NOT EXISTS tbl (id bigint, value int); INSERT INTO tbl VALUES (1, 100), (2, 200);"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@session.auto_increment_increment;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@character_set_client;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@character_set_connection;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@max_allowed_packet;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@lower_case_table_names;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@session.last_insert_id;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocommit;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@tx_isolation_eahhahfefhehfehf;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@nonexistent_variable;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@random_name;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocomm;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@character_set;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@version;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocommit, @@nonexistent_variable;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@character_set_client, @@tx_isolation_fake, @@max_allowed_packet;"
––– output –––
OK
––– input –––
mysqldump -h0 -P9306 manticore tbl --no-data --skip-comments --skip-opt --no-tablespaces > /tmp/dump.sql 2> >(grep -E -v "Warning: column statistics|Warning: version string returned by server is incorrect." >&2)
––– output –––
OK
––– input –––
test -s /tmp/dump.sql && echo "Dump created successfully" || (echo "Dump is empty or failed!" && cat /tmp/dump.sql)
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DROP TABLE tbl;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 manticore < /tmp/dump.sql; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE tbl;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @user_variable;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @nonexistent_user_var;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@123;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'SELECT @@`quoted_name`;'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@very_long_variable_name_that_exceeds_normal_length_limits;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@AUTOCOMMIT;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@AutoCommit;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocommit;"
––– output –––
OK
––– input –––
stdbuf -oL searchd --stopwait > /dev/null
––– output –––
OK
––– input –––
echo -e "searchd {\nlisten = 9306:mysql\nlisten = 9308:http\nlog = /var/log/manticore/searchd.log\nquery_log = /var/log/manticore/query.log\npid_file = /var/run/manticore/searchd.pid\ndata_dir = /var/lib/manticore\nbuddy_path = \n}" > /tmp/manticore_without_buddy.conf; echo $?
––– output –––
OK
––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd -c /tmp/manticore_without_buddy.conf > /dev/null; if timeout 10 grep -qm1 'accepting connections' /var/log/manticore/searchd.log; then echo 'Searchd started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@tx_isolation_eahhahfefhehfehf;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@nonexistent_variable;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@random_name;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocomm;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@character_set;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@version;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocommit, @@nonexistent_variable;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@character_set_client, @@tx_isolation_fake, @@max_allowed_packet;"
––– output –––
OK
––– input –––
mysqldump -h0 -P9306 manticore tbl --no-data --skip-comments --skip-opt --no-tablespaces > /tmp/dump.sql 2> >(grep -E -v "Warning: column statistics|Warning: version string returned by server is incorrect." >&2)
––– output –––
+ mysqldump: Couldn't execute 'SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"') FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = 'manticore' AND TABLE_NAME = 'tbl';': No such table information_schema.COLUMN_STATISTICS (1064)
––– input –––
test -s /tmp/dump.sql && echo "Dump created successfully" || (echo "Dump is empty or failed!" && cat /tmp/dump.sql)
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DROP TABLE tbl;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 manticore < /tmp/dump.sql; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE tbl;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @user_variable;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @nonexistent_user_var;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@123;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e 'SELECT @@`quoted_name`;'
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@very_long_variable_name_that_exceeds_normal_length_limits;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@AUTOCOMMIT;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@AutoCommit;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT @@autocommit;"
––– output –––
OK |
clt❌ CLT tests in test/clt-tests/core/test-distributed-inserts-with-replication.rec––– input –––
apt-get install -y jq > /dev/null; echo $?
––– output –––
OK
––– input –––
export INSTANCE=1
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export INSTANCE=2
––– output –––
OK
––– input –––
mkdir -p /var/{run,lib,log}/manticore-${INSTANCE}
––– output –––
OK
––– input –––
stdbuf -oL searchd -c test/clt-tests/base/searchd-with-flexible-ports.conf > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore-${INSTANCE}/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore-${INSTANCE}/searchd.log;fi
––– output –––
OK
––– input –––
export CLUSTER_NAME=replication
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE CLUSTER ${CLUSTER_NAME}"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for n in `seq 2 $INSTANCE`; do mysql -h0 -P${n}306 -e "JOIN CLUSTER ${CLUSTER_NAME} AT '127.0.0.1:1312'"; done;
––– output –––
OK
––– input –––
mysql -h0 -P${INSTANCE}306 -e "SHOW STATUS LIKE 'cluster_${CLUSTER_NAME}_status'\G"
––– output –––
OK
––– input –––
for port in 1306 2306; do echo "Checking status for port $port:"; timeout 10 bash -c "while ! mysql -h0 -P$port -e \"SHOW STATUS LIKE 'cluster_replication_node_state'\G\" | grep -q 'Value: synced'; do sleep 1; done" && echo "Port $port: Node is synced." || echo "Port $port: Node is not synced (Value: closed or other)."; done
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "DROP TABLE IF EXISTS test2;"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "CREATE TABLE replication:test2 (id BIGINT, model TEXT, storage_capacity INTEGER, color STRING, release_year INTEGER, price FLOAT, discounted_price FLOAT, sold BOOL, date_added TIMESTAMP, product_codes MULTI, values MULTI64, additional_info JSON, vector FLOAT_VECTOR KNN_TYPE='hnsw' KNN_DIMS='4' HNSW_SIMILARITY='l2') SHARDS='3' RF='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_replication_node_state'\G"
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -h0 -P2306 -e "SHOW TABLES FROM SYSTEM\G"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P2306 -h0 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -P1306 -h0 -e "DESCRIBE test2;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "iPhone 13 Pro", "storage_capacity": 256, "color": "silver", "release_year": 2021, "price": 1099.99, "discounted_price": 989.99, "sold": 1, "date_added": 1591362342000, "product_codes": [1,2,3], "values": [523456764345678976], "additional_info": {"features": ["ProMotion display", "A15 Bionic chip"]}, "vector": [0.773448,0.312478,0.137971,0.459821]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"model": "Galaxy S21 Ultra", "storage_capacity": 128, "color": "black", "release_year": 2021, "price": 1199.99, "discounted_price": 1099.99, "sold": 1, "date_added": 1609459200000, "product_codes": [4,5,6], "values": [1234567890123456789], "additional_info": {"features": ["Dynamic AMOLED 2X", "Exynos 2100"]}, "vector": [0.5,0.4,0.3,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 3, "doc": {"model": "Pixel 6", "storage_capacity": 128, "color": "white", "release_year": 2021, "price": 599.99, "discounted_price": 549.99, "sold": false, "date_added": 1630454400000, "product_codes": [7,8,9], "values": [987654321987654321], "additional_info": {"features": ["Smooth display", "Google Tensor chip"]}, "vector": [0.8,0.6,0.4,0.2]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/replace -d '{"cluster": "replication", "table": "test2", "id": 1, "doc": {"model": "Updated Model", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1399.99, "discounted_price": 1299.99, "sold": false, "date_added": 1630454400000, "product_codes": [10,11,12], "values": [987654321987654321], "additional_info": {"features": ["New feature 1", "New feature 2"]}, "vector": [0.7,0.8,0.9,1.0]}}' | jq '.result'
––– output –––
OK
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/update -d '{"cluster": "replication", "table": "test2", "id": 2, "doc": {"price": 1099.99, "discounted_price": 999.99}}' | jq '.result'
––– output –––
- "updated"
+ null
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
- "deleted"
+ "not found"
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
+ | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
+ | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1199.989990 | 1099.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+--------------------------------------------------+-------------------------------------+
+ | 3 | Pixel 6 | 128 | white | 2021 | 599.989990 | 549.989990 | 0 | 2661794816 | 7,8,9 | 987654321987654321 | {"features":["Smooth display","Google Tensor chip"]} | 0.800000,0.600000,0.400000,0.200000 |
+ +------+------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+------------------------------------------------------+-------------------------------------+
––– input –––
curl -s -X POST http://localhost:1308/delete -d '{"cluster": "replication", "table": "test2", "id": 3}' | jq '.result'
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": 4, "doc": {"model": "Updated iPhone 14", "storage_capacity": 512, "color": "black", "release_year": 2022, "price": 1299.99, "discounted_price": 1199.99, "sold": 1, "date_added": 1661990400, "product_codes": [19,20,21], "values": [1234567890123456789],"additional_info": {"features": ["A16 Bionic","Dynamic Island","Improved camera"]},"vector": [0.1,0.2,0.3,0.4]}}' | jq -e '.error' && echo "Duplicate ID test passed!" || echo "Duplicate ID test failed!"
––– output –––
- null
+ {
- Duplicate ID test failed!
+ "type": "action_request_validation_exception",
+ "reason": "table 'test2' does not support INSERT",
+ "table": "test2"
+ }
+ Duplicate ID test passed!
––– input –––
(for i in {1..10}; do curl -s -X POST http://localhost:1308/insert -d '{"cluster": "replication", "table": "test2", "id": '$((10 + $i))', "doc": {"model": "Device '$i'", "storage_capacity": 64, "color": "black", "release_year": 2023, "price": 499.99, "discounted_price": 449.99, "sold": 0, "date_added": 1672531200, "product_codes": [1,2,3], "values": [1234567890123456789], "additional_info": {"features": ["Feature 1","Feature 2"]}, "vector": [0.1,0.2,0.3,0.4]}}' & done; wait) | jq -s 'map(select(.created == true)) | length == 10' > /dev/null && echo "Parallel insert test passed!" || echo "Parallel insert test failed!"
––– output –––
OK
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 ORDER BY id ASC;"
––– output –––
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 1 | Updated Model | 512 | black | 2022 | 1399.989990 | 1299.989990 | 0 | 2661794816 | 10,11,12 | 987654321987654321 | {"features":["New feature 1","New feature 2"]} | 0.700000,0.800000,0.900000,1.000000 |
- | 2 | Galaxy S21 Ultra | 128 | black | 2021 | 1099.989990 | 999.989990 | 1 | 3141431296 | 4,5,6 | 1234567890123456789 | {"features":["Dynamic AMOLED 2X","Exynos 2100"]} | 0.500000,0.400000,0.300000,0.200000 |
- | 4 | Updated iPhone 14 | 512 | black | 2022 | 1299.989990 | 1199.989990 | 1 | 1661990400 | 19,20,21 | 1234567890123456789 | {"features":["A16 Bionic","Dynamic Island","Improved camera"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-------------------+------------------+-------+--------------+-------------+------------------+------+------------+---------------+---------------------+----------------------------------------------------------------+-------------------------------------+
––– input –––
sleep 2; mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id BETWEEN 11 AND 20 ORDER BY id ASC;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
- | 11 | Device 1 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 12 | Device 2 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 13 | Device 3 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 14 | Device 4 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 15 | Device 5 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 16 | Device 6 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 17 | Device 7 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 18 | Device 8 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 19 | Device 9 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
mysql -h0 -P1306 -e "SELECT * FROM test2 WHERE id = 20;"
––– output –––
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ ERROR 1064 (42000) at line 1: table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s0' in search request;
- | id | model | storage_capacity | color | release_year | price | discounted_price | sold | date_added | product_codes | values | additional_info | vector |
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s1' in search request;
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
+ table test2: agent 127.0.0.1:2312: remote query error: unknown local table(s) 'system.test2_s2' in search request
- | 20 | Device 10 | 64 | black | 2023 | 499.989990 | 449.989990 | 0 | 1672531200 | 1,2,3 | 1234567890123456789 | {"features":["Feature 1","Feature 2"]} | 0.100000,0.200000,0.300000,0.400000 |
- +------+-----------+------------------+-------+--------------+------------+------------------+------+------------+---------------+---------------------+----------------------------------------+-------------------------------------+
––– input –––
rm -f bulk.json bulk_insert.json bulk_update.json bulk_delete.json bulk_operations.json; if [[ ! -f bulk.json && ! -f bulk_insert.json && ! -f bulk_update.json && ! -f bulk_delete.json && ! -f bulk_operations.json ]]; then echo "All temporary files deleted successfully."; else echo "Error: Some temporary files were not deleted."; fi
––– output –––
OK |
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
clt❌ CLT tests in test/clt-tests/bugs/3481-http-update-distributed-table-wrong-cluster-crash.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd --logdebugv > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Manticore started!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -v -P9306 -h0 -e "drop table if exists t; create table t(f int); drop table if exists d; CREATE TABLE d type='distributed' agent='127.0.0.1:9312:t'"
––– output –––
OK
––– input –––
curl -s -X POST http://localhost:9312/update -d '{"cluster": "unknown_cluster", "table": "d", "id": 2, "doc": {"f": 5}}'
––– output –––
- {"error":{"type":"action_request_validation_exception","reason":"","table":"d"},"status":409}
+ {"error":{"type":"action_request_validation_exception","reason":"table t: table t: table 't' is not in any cluster, use just 't'","table":"d"},"status":409}
––– input –––
sleep 2; cat /var/log/manticore/searchd.log | grep 'response data' | cut -d' ' -f12-
––– output –––
- {"version":3,"type":"json response","message":{"error":{"type":"action_request_validation_exception","reason":"","table":"d"},"status":409},"meta":null,"error_code":0}
+ {"version":3,"type":"json response","message":{"error":{"type":"action_request_validation_exception","reason":"table t: table 't' is not in any cluster, use just 't'","table":"t"},"status":409},"meta":null,"error_code":0}
+ {"version":3,"type":"json response","message":{"error":{"type":"action_request_validation_exception","reason":"table t: table t: table 't' is not in any cluster, use just 't'","table":"d"},"status":409},"meta":null,"error_code":0}
|
🐳 Docker Images PushedBuddy Test-Kit Images:
Base image used: These images contain the buddy code from this PR and can be used for testing. |
sanikolaev
pushed a commit
to manticoresoftware/manticoresearch
that referenced
this pull request
Jul 10, 2025
This increases buddy version from 3.32.0 to 3.32.1 which provides refactoring of auto-sharding Related PR manticoresoftware/manticoresearch-buddy#562 Related PR #3558 Related issue #3481
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.