Skip to content

fix(eks-lifecycle): 00-auth.sh kubectl reachability check inside admin subshell - #405

Merged
panicboat merged 1 commit into
mainfrom
fix/eks-00-auth-kubectl-check
May 16, 2026
Merged

fix(eks-lifecycle): 00-auth.sh kubectl reachability check inside admin subshell#405
panicboat merged 1 commit into
mainfrom
fix/eks-00-auth-kubectl-check

Conversation

@panicboat

Copy link
Copy Markdown
Owner

2026-05-16 cold-start recreate live run の Phase 9 で `60-flux-bootstrap.sh` が `Cluster not reachable. Setting CLUSTER_EXISTS=false` で誤判定し exit した bug の fix。

Symptom

`60-flux-bootstrap.sh` が cluster 起動済の状態で `CLUSTER_EXISTS=false` 判定 → `Cluster not reachable. Run make eks-recreate-aws ENV=production first.` で exit 1。 live run では script bypass で manual に hydrate / kubectl apply / Flux setup を実行する workaround を取った。

Root cause

`00-auth.sh` の cluster reachability check:

```bash
(
AWS_*_KEY = admin creds
aws eks update-kubeconfig ...
) && CLUSTER_REACHABLE="true" || CLUSTER_REACHABLE="false"

if [ "$CLUSTER_REACHABLE" = "true" ] && kubectl get nodes >/dev/null 2>&1; then
```

  • `aws eks update-kubeconfig` は subshell 内で admin creds 使用
  • `kubectl get nodes` は subshell で実行される
  • 親 shell の AWS env は operator IAM principal (= panicboat user 等) のまま
  • kubectl exec plugin (= `aws eks get-token`) は親 shell の env で動作 = operator creds で token 取得
  • operator が EKS aws-auth に未登録のケース (= `access_entries` に `eks-admin` role のみ登録) で 401
  • → `CLUSTER_REACHABLE=false` の誤判定

Fix

`aws eks update-kubeconfig` + `kubectl get nodes` を同 subshell に集約:

```bash
(
AWS_*_KEY = admin creds
if aws eks update-kubeconfig ... && \
kubectl get nodes >/dev/null 2>&1; then
exit 0
fi
exit 1
) && CLUSTER_REACHABLE="true" || CLUSTER_REACHABLE="false"

if [ "$CLUSTER_REACHABLE" = "true" ]; then
ok "Cluster reachable via admin role"
...
```

両者 admin creds で実行 + subshell 終了後 CLUSTER_REACHABLE 判定のみ親 shell。

Test plan

  • live run で manual bypass 検証済 (= admin role assume + kubectl get nodes が動作することを確認)
  • 次回 recreate live run で 60-flux-bootstrap.sh が CLUSTER_EXISTS=true で進むことを確認

Related

…n subshell

2026-05-16 cold-start recreate live run の Phase 9 で 60-flux-bootstrap.sh が
'Cluster not reachable. Setting CLUSTER_EXISTS=false' で誤判定し exit、
manual bypass で workaround した bug の正式 fix。

旧 code: aws eks update-kubeconfig は subshell 内で admin creds 使うが、
kubectl get nodes は subshell 外で実行され、 親 shell の operator IAM
principal (= panicboat user 等) で kubectl exec plugin (= aws eks get-token)
が動作。 operator が EKS aws-auth に未登録のケース (= access_entries に
eks-admin role のみ登録) で 401 → CLUSTER_REACHABLE=false の誤判定。

新 code: aws eks update-kubeconfig + kubectl get nodes を同 subshell に
集約 (= 両者 admin creds で実行)。 subshell 終了後 CLUSTER_REACHABLE 判定
のみ親 shell で実施。

Signed-off-by: panicboat <panicboat@gmail.com>
@coderabbitai

coderabbitai Bot commented May 16, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@panicboat has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 58 minutes and 34 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 67e13b04-7db2-4aed-8d6b-ac79932fb7ee

📥 Commits

Reviewing files that changed from the base of the PR and between fbc7c09 and 0a19293.

📒 Files selected for processing (1)
  • scripts/eks-lifecycle/lib/00-auth.sh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@panicboat
panicboat merged commit df26b79 into main May 16, 2026
7 checks passed
@panicboat
panicboat deleted the fix/eks-00-auth-kubectl-check branch May 16, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant