fix(scripts): M5a — --host 127.0.0.1 for --local mode#926
Conversation
E2E run of scripts/run-jepsen-m5-local.sh after PR #924 + #925 merge revealed a DNS-resolution failure during the workload's first setup!: DynamoDB :cognitect.anomalies/not-found: n5: nodename nor servname provided, or not known Root cause: the workload's open! resolves the DynamoDB client hostname via (or (:dynamo-host test) (name node)). When --local is set, :dynamo-host is nil (the cluster is local), so it falls through to (name node) where node is one of default-nodes ["n1" "n2" "n3" "n4" "n5"] — virtual labels, not real hostnames. DNS resolution fails. Fix: thread --host 127.0.0.1 through lein run. cli/common-cli-opts maps it to :host -> prepare-dynamo-opts copies to :dynamo-host -> make-ddb-client uses it as the endpoint hostname. All five nodes' clients now correctly dial the single-process loopback DynamoDB endpoint (PROC_ADDR=127.0.0.1:50051 / 63801). Verification: post-fix run no longer hits the n5 DNS failure. setup! now proceeds to verify-multi-group-routing! and create-all-tables!. Note: a separate issue surfaces after this fix — workers report ResourceNotFoundException for every txn, even though create-all-tables! reports success and ListRoutes shows the expected two-group catalog. The 5-parallel client setup may be racing on CreateTable in a way the server reports as ACTIVE but the table-meta key is not actually durably visible to the subsequent invoke!. That's a separate investigation outside this fix.
|
Warning Review limit reached
More reviews will be available in 15 minutes and 51 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the scripts/run-jepsen-m5-local.sh script to explicitly pass the --host 127.0.0.1 argument to the Jepsen workload command. This prevents DNS resolution failures that occur when the workload attempts to resolve virtual node labels (e.g., 'n1', 'n2') as real hostnames, ensuring all node clients correctly connect to the local loopback DynamoDB endpoint. A detailed comment explaining this change has also been added. There are no review comments to address, and I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…#930 partial) Adding default range coverage for routing keys lexicographically smaller than T1_KEY. Without this, any table whose base64-encoded name sorts before 'amVwc2VuX2FwcGVuZF90MQ' (= base64('jepsen_append_t1')) returns 'no route for key' from ShardedCoordinator.dispatchTxn, which createTableWithRetry silently swallows as ACTIVE (issue #930 root cause #1). Topology consequence: the default range goes to group 1. Tables 1-2 share that group; tables 3-4 use group 2. NOTE — partial fix. E2E run after this commit STILL shows all workers reporting ResourceNotFoundException. The jepsen_append_t1..4 routing keys are inside the original [T1_KEY, +inf) coverage, so those should not benefit from this change — yet they're still failing. A second bug is present beyond the routing-coverage gap and needs separate investigation. See issue #930 for the next-step plan.
1 participant
Bug fix discovered during the M5a E2E run of
scripts/run-jepsen-m5-local.shafter PR #924 + #925 merge.症状
./scripts/run-jepsen-m5-local.sh実行時、最初の workload setup! が以下で失敗:根本原因
dynamodb_multi_table_workload.cljのopen!で:--local設定時、:dynamo-hostは nil なので(name node)(default-nodesの"n1".."n5") を hostname として使用 → DNS 解決失敗。Fix
lein runに--host 127.0.0.1を追加。cli/common-cli-opts経由で:host→:dynamo-host→make-ddb-clientに thread されて、全 5 nodes の client が同じループバック endpoint を dial する。関連する未解決問題 (本 PR スコープ外)
この修正後、setup! は
n5DNS failure を回避できるが、新たに workers が全 txn でResourceNotFoundException: table not foundを返す問題が surface した。:create-all-tables!は (5 並列 client で) 5 回成功 reportCreateTableを投げると HTTP 200 +"TableStatus":"ACTIVE"GetItem/TransactGetItemsはResourceNotFoundException5 client 並列での CreateTable race か、
adapter/dynamodb.goの internal sync 問題か。別 issue で調査予定。Test plan
bash -n scripts/run-jepsen-m5-local.sh構文OKn5: nodename nor servname providedエラーは出ないResourceNotFoundException別件解決後