Skip to content

[Feat]: admin-server and protocol-test support#24

Merged
vincent-scitix merged 44 commits into
mainfrom
vincent-dev
Apr 13, 2026
Merged

[Feat]: admin-server and protocol-test support#24
vincent-scitix merged 44 commits into
mainfrom
vincent-dev

Conversation

@vincent-scitix

Copy link
Copy Markdown
Collaborator
  • Admin server
    AdminServer is a UDS (Unix Domain Socket) based admin interface embedded in SiMM components. It provides a local channel for querying and configuring process-internal state — gflags, trace, node/shard routing, heartbeat status, and more — without going through the RDMA-based SiCL RPC stack.

  • Protocol test
    A pytest-based end-to-end distributed protocol test framework for SiMM's cluster management protocols (node join/leave, heartbeat, shard rebalancing, deferred reshard). Tests run against real CM and DS binaries — no mocking. Fault injection via process signals, state observation via admin UDS queries (simmctl).

- CallbackNode: move resp declaration before lambda so delete works
- CallbackShard: same restructure for resp lifetime
- CallbackTrace: delete resp on Call() failure path
…t tests

- Rename tests/cluster_integration/ to tests/protocol_test/
- Update comments in conftest.py to reflect new naming
- Add missing DS gflag injection: cm_hb_tolerance_count, cm_connect_retry_interval_sec
  in process_manager.py and cluster.py
- Add 65 unit tests for framework modules:
  test_config (25), test_port_allocator (8), test_log_parser (10),
  test_admin_client (9), test_scenario_runner (13)
…ication

Log-parsing tests were fragile (depend on log keywords that change often).
The same protocol behavior is already verified via admin UDS queries
(node list, shard list, DS status). Also clean up unused LogParser
references from cluster_observer and cluster.
Comment thread tests/protocol_test/conftest.py Fixed
Comment thread tests/protocol_test/tests/test_cm_restart.py Fixed
Comment thread tests/protocol_test/tests/test_deferred_reshard.py Fixed
Comment thread tests/protocol_test/tests/test_deferred_reshard.py Fixed
Comment thread tests/protocol_test/framework/ssh_executor.py Fixed
Comment thread tests/protocol_test/conftest.py Fixed
Comment thread tests/protocol_test/framework/cluster_observer.py Fixed
Comment thread tests/protocol_test/conftest.py Fixed
Comment thread tests/protocol_test/framework/process_manager.py Fixed
Comment thread tests/protocol_test/framework/scenario_runner.py Fixed
@Sebastian-Keith Sebastian-Keith changed the title admin-server and protocol-test support [Feat]: admin-server and protocol-test support Apr 9, 2026
Comment thread src/data_server/kv_server_main.cc Outdated
Comment thread src/data_server/kv_server_main.cc Outdated
Comment thread src/common/admin/admin_msg_types.h
Comment thread src/cluster_manager/cm_main.cc Outdated
Comment thread tools/simm_ctl_admin.cc
vincent-scitix and others added 6 commits April 11, 2026 15:02
# Conflicts:
#	tools/simm_ctl_admin.cc
… a call'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Comment thread tests/protocol_test/conftest.py Fixed
Comment thread tests/protocol_test/framework/tests/test_config.py Fixed
Comment thread tests/protocol_test/framework/tests/test_config.py Fixed
Comment thread tests/protocol_test/framework/tests/test_config.py Fixed
Comment thread tests/protocol_test/framework/tests/test_config.py Fixed
Comment thread tests/protocol_test/tests/test_deferred_reshard.py Fixed
vincent-scitix and others added 2 commits April 11, 2026 15:44
…mplicit (fall through) returns'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Comment thread tests/protocol_test/conftest.py Fixed
Comment thread tests/protocol_test/tests/test_graceful_shutdown.py Dismissed
vincent-scitix and others added 3 commits April 11, 2026 15:48
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Comment thread tests/protocol_test/tests/test_multi_failure.py Dismissed
Comment thread tests/protocol_test/tests/test_node_join.py Fixed
Comment thread tests/protocol_test/tests/test_node_rejoin.py Fixed
vincent-scitix and others added 6 commits April 11, 2026 17:46
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
… a call'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Comment thread tests/protocol_test/conftest.py Fixed
Comment thread tests/protocol_test/framework/tests/test_config.py Fixed
Comment thread tests/protocol_test/framework/tests/test_config.py Dismissed
Comment thread tests/protocol_test/framework/tests/test_scenario_runner.py Dismissed
- Define kCmAdminUdsBasePath / kDsAdminUdsBasePath in admin_msg_types.h,
  replace hardcoded "/run/simm/admin_cm" and "/run/simm/admin_ds" in
  cm_main.cc, kv_server_main.cc, and simm_ctl_admin.cc
- simmctl: remove duplicated AdminMsgType enum, use shared header
- Add test_insufficient_nodes_keeps_deferred_state for deferred reshard
- Fix conftest cluster_medium parameter name (grace_period_sec ->
  cm_cluster_init_grace_period_inSecs), add UT coverage
- Fix unused variables in test_deferred_reshard.py, test_cm_restart.py
- Add debug logging to silent exception handlers in ssh_executor.py
  and cluster_observer.py
- Remove unused imports across framework and test files
- Update admin_tool.md with UDS mode (--pid/--proc), cm/ds status,
  node summary/stat, trace subcommands
# Conflicts:
#	tests/protocol_test/conftest.py
#	tests/protocol_test/framework/ssh_executor.py
#	tests/protocol_test/tests/test_deferred_reshard.py
@vincent-scitix
vincent-scitix merged commit 31b9b5d into main Apr 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants