Skip to content

feat(e2e): 基于 Docker 的 e2e 测试框架 + 修复 3 个 Linux 数据面 bug - #2

Merged
kvmaker merged 23 commits into
masterfrom
e2e/impl
Jul 8, 2026
Merged

feat(e2e): 基于 Docker 的 e2e 测试框架 + 修复 3 个 Linux 数据面 bug#2
kvmaker merged 23 commits into
masterfrom
e2e/impl

Conversation

@kvmaker

@kvmaker kvmaker commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

基于 Docker 的 e2e 测试框架(T00-T08)+ e2e 发现并修复的 3 个 mesh Linux 数据面生产 bug。

e2e 框架

  • 3 容器拓扑(1 server + 2 client,privileged + /dev/net/tun + tc netem)
  • 3 场景:连通性/路由(P0)、性能/抖动(P0)、故障/重连/优雅退出(P1)
  • run.sh 一键入口(--quick/--all/--strict/--scenario)
  • CI workflow(--quick + if:always artifact)
  • 软+可选硬门槛(STRICT)

修复的 mesh bug(e2e 发现,此前从未在 Linux 跑通真实数据面)

Bug 修复前 修复后 commit
Linux TUN offset(数据面全断) ICMP/UDP/TCP 全断 全通 56a0a19
B00 TCP checksum(TCP 全断) TCP 0.0016 Mbps 877.64 Mbps 4b4a943
B01 main loop 死锁(不退出/不重连) SIGTERM >10s 116ms a3868b8

附带改进

  • MESH_TEST_TLS 开关(server 自签 + client --insecure,生产路径零影响)
  • entrypoint 幂等化(容器 restart 通用修复)

Test Plan

  • go test ./... 全绿
  • bash tests/e2e/run.sh --all → Overall PASS
  • macOS 不回归(tun_other.go + darwin offset 4)
  • CI e2e job(本 PR 触发)
  • coderabbit review

详见:

  • 设计 spec: docs/superpowers/specs/2026-07-08-e2e-docker-design.md
  • 实施计划: docs/superpowers/plans/2026-07-08-e2e-impl.md
  • bug 分析: docs/todo/bug/B00.md, B01.md

🤖 CodeRabbit Review Request

Summary by CodeRabbit

  • New Features

    • Added end-to-end test support with Docker-based server/client setup and quick/full/strict run modes.
    • Added a --insecure option for joining in test environments with self-signed certificates.
  • Bug Fixes

    • Improved client reconnect and shutdown behavior so connections recover more reliably and stop cleanly.
    • Fixed Linux networking setup to avoid packet corruption in TUN-based traffic.
  • Documentation

    • Expanded setup and usage docs for running end-to-end tests and reviewing results.
  • Tests

    • Added automated checks for connectivity, performance, failure recovery, and TLS behavior.

maxyu and others added 22 commits July 8, 2026 10:42
基于 spec 拆解:MESH_TEST_TLS 开关、Docker 编排、helpers/metrics、
netem 预设、连通性/性能/故障三场景、run.sh 聚合、CI workflow。

Co-Authored-By: Claude <noreply@anthropic.com>
- config 增加 TLSTestMode,由 MESH_TEST_TLS 环境变量控制
- server 在 test mode 用内存自签证书,不依赖 Let's Encrypt
- client join/up 支持 --insecure 跳过证书校验
- NewTunnelClient 增加 tlsConfig 参数

Co-Authored-By: Claude <noreply@anthropic.com>
- Default() 调 applyTestMode,配置回退路径也生效
- --insecure 改 bool flag,避免字符串陷阱
- NewTunnelClient 复用 http.Client,防止重连 fd 泄漏
- config test 补边界值(true/self/no/大小写/空格)
- 补 join_test.go(insecure 传输与持久化)
- peers.go 改用 cfg.InsecureTLS,修生产环境无条件跳过 TLS 校验的安全问题

Co-Authored-By: Claude <noreply@anthropic.com>
- Dockerfile.server / Dockerfile.client(多阶段构建,golang:1.25)
- docker-compose 3 容器(privileged + NET_ADMIN + /dev/net/tun)
- fixtures: meshd.yaml / entrypoint-server.sh(含 token 提取)/ entrypoint-client.sh

Co-Authored-By: Claude <noreply@anthropic.com>
- client 等 server 超时后 exit 1,避免混淆报错
- compose 加 hostname: client-a/client-b,注册名可读
- 加 .dockerignore,减小 build context
- server token 提取失败报错退出
- 去掉 server 镜像未用的 sqlite3

Co-Authored-By: Claude <noreply@anthropic.com>
- helpers: dex/wait_for_server/wait_for_client/get_token/dump_logs/assert
- metrics: rtt_stats/iperf_tcp/iperf_udp_loss/json_set

Co-Authored-By: Claude <noreply@anthropic.com>
正则跨过第一个斜杠值(min),取第二个(avg),符合规格与通用约定。

Co-Authored-By: Claude <noreply@anthropic.com>
- fixtures/netem.sh 封装 tc netem 预设
- Dockerfile.client 打进 /usr/local/bin/netem-preset.sh

Co-Authored-By: Claude <noreply@anthropic.com>
wireguard/tun 库在 Linux 以 IFF_TUN|IFF_NO_PI|IFF_VNET_HDR 打开 TUN 设备,
每帧前缀 10 字节 virtio net header。Write 路径的 handleGRO 会硬性校验
offset >= virtioNetHdrLen(10),传 0 直接返回 "invalid offset",导致所有从
网络收到的包都写不进 TUN(server 与 client 均受影响,mesh0 RX 恒为 0)。

Offset() 改为 Linux 返回 10、darwin 仍返回 4。所有 buffer 分配与读写偏移
均基于 Offset(),无需改动调用方。

Co-Authored-By: Claude <noreply@anthropic.com>
容器重启(docker compose stop/start、restart)时 /root/.mesh/config.json 仍在,
mesh join 会拒绝并报 "already registered; run 'mesh leave' first",导致
entrypoint 退出、容器无法自愈。改为检测到已注册则跳过 join 直接 mesh up。

这样容器重启可自愈,e2e reconnect 场景也能通过。

Co-Authored-By: Claude <noreply@anthropic.com>
- mesh0 路由注入、ping server/peer、不存在 IP 100% 丢包
- client-b 断开/重连、fping 并发

Co-Authored-By: Claude <noreply@anthropic.com>
- fping 空值兜底 exit 0→1,消除假阳性入口
- .gitignore 排除 tests/e2e/results/

Co-Authored-By: Claude <noreply@anthropic.com>
- iperf3 TCP 1流/4流、UDP 丢包
- baseline/wan netem 对比
- RTT 采集,可选 STRICT 硬门槛

Co-Authored-By: Claude <noreply@anthropic.com>
e2e T05 性能场景抓包定位:mesh 在 Linux 上 TCP over mesh 全断
(~0Mbps),UDP/ICMP 正常。根因是 wireguard/tun 库强制 IFF_VNET_HDR
启用 offload,kernel 对 TCP 包不软件算 checksum,mesh 转发不修正,
对端 kernel 校验失败丢弃。P0 生产 blocker,待 brainstorm 修复方案。

Co-Authored-By: Claude <noreply@anthropic.com>
- server kill→restart client 自动恢复
- UDP 长流中 server 重启连通恢复(TCP 因 B00 checksum bug 暂用 UDP)
- SIGTERM <=2s 优雅退出

Co-Authored-By: Claude <noreply@anthropic.com>
e2e T06 故障场景发现:无 TUN 背景流量时 SIGTERM 不退出(80s+)、
server 故障后不自动重连(死锁)。根因是 connect 主循环 select{ctx.Done;
default}+tun.Read 阻塞不响应 cancel。P1,修复方向是 ctx cancel 时
close TUN fd。T06 暂用主动 ping 产生流量 workaround。

Co-Authored-By: Claude <noreply@anthropic.com>
- 分步起容器(server→token→串行 client),避免 token 错/server recreate
- 跑场景、收日志、聚合 summary.txt
- 支持 --all/--quick/--strict/--scenario N
- trap 保证容器 always down

Co-Authored-By: Claude <noreply@anthropic.com>
- push/PR 触发,--quick 模式
- if: always() 保证 artifact 上传

Co-Authored-By: Claude <noreply@anthropic.com>
- metrics.sh rtt_stats 改 sed-E(POSIX),消除 PCRE 死代码歧义
- 删 02-performance.sh 的本地 rtt_stats 覆盖(库版已正确)
- README 加 e2e 测试运行说明

Co-Authored-By: Claude <noreply@anthropic.com>
wireguard/tun 库 CreateTUN 硬编码 IFF_VNET_HDR 启用 offload,kernel 对
TCP 包不软件算 checksum,mesh 转发不修正导致对端 kernel 丢弃所有 TCP
数据段(TCP over mesh 全断)。改为 mesh 自开 /dev/net/tun(IFF_TUN|
IFF_NO_PI,无 VNET_HDR)+ CreateTUNFromFile,vnetHdr=false 不 offload,
TCP checksum 由 kernel 软件算好。Offset() Linux 回 0(撤 T04 workaround)。

Co-Authored-By: Claude <noreply@anthropic.com>
TunnelClient main loop 阻塞在 tun.Read 时不响应 ctx cancel,导致无背景
流量时 SIGTERM 不退出、server 故障不自动重连。

改为在 Run 启动持久的 tunReadLoop goroutine 跨重连复用同一 TUN fd,
tun.Read 在此执行并通过 pktCh 投递包;connect 的 main loop 用
select{pktCh; ctx.Done} 监听,无流量时也能立即响应取消。

不采用 close TUN 方案:close 后 fd 永久失效,Run 的进程内重连会因
tun.Read 立即报错而无限失败。channel 解耦让 TUN fd 生命周期独立于
WS 连接,重连无需重建 TUN。

Co-Authored-By: Claude <noreply@anthropic.com>
- B00(commit 4b4a943):TCP checksum,方案 D,0.0016→877.64 Mbps
- B01(commit a3868b8):main loop 死锁,channel 方案,SIGTERM 116ms
- bug 类别 2/2 100%

Co-Authored-By: Claude <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@kvmaker, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 39 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6c01e3a4-1f46-418c-86b6-d1b1ad6aa065

📥 Commits

Reviewing files that changed from the base of the PR and between 15e6c52 and 06acfb3.

📒 Files selected for processing (10)
  • .github/workflows/e2e.yml
  • docs/superpowers/specs/2026-07-08-e2e-docker-design.md
  • internal/api/server.go
  • internal/config/config.go
  • internal/config/config_test.go
  • internal/tun/tun_linux.go
  • internal/tunnel/client.go
  • tests/e2e/fixtures/entrypoint-server.sh
  • tests/e2e/scenarios/01-connectivity.sh
  • tests/e2e/scenarios/02-performance.sh
📝 Walkthrough

Walkthrough

This PR adds an opt-in TLS test mode (self-signed certs on the server, insecure TLS skip on clients) via a new config flag and CLI --insecure option, refactors Linux TUN creation to drop IFF_VNET_HDR and decouples TUN reads from the tunnel client's WebSocket loop, and introduces a full Docker Compose-based e2e test suite with scenarios, helper libraries, CI workflow, and documentation.

Changes

TLS Test Mode and TUN Fix

Layer / File(s) Summary
Config TLSTestMode flag
internal/config/config.go, internal/config/config_test.go
Adds TLSTestMode field and applyTestMode() parsing MESH_TEST_TLS, called from Default(), with a table-driven test.
Server self-signed TLS
internal/api/server.go
ListenAndServeTLS branches to a new selfSignedTLSConfig helper generating an ECDSA cert when TLSTestMode is enabled, bypassing autocert.
Client insecure TLS flag
cmd/mesh/main.go, internal/client/config.go, internal/client/join.go, internal/client/join_test.go, internal/client/peers.go, internal/client/up.go
Adds --insecure flag, InsecureTLS config field, and conditional InsecureSkipVerify in Join, Peers, and Up; adds TestJoinInsecureTLS.
Tunnel client TLS + decoupled TUN reads
internal/tunnel/client.go
NewTunnelClient accepts tlsConfig; Run starts a persistent tunReadLoop feeding a packet channel consumed by connect.
Linux TUN creation without VNET_HDR
internal/tun/tun.go, internal/tun/tun_linux.go, internal/tun/tun_other.go
CreateTUN delegates to platform-specific createTUNNative; Linux opens /dev/net/tun with `IFF_TUN
Bug tracking docs
docs/todo/README.md, docs/todo/bug/*
Adds B00/B01 bug analysis docs and index, updates TODO progress table.

Estimated code review effort: 4 (Complex) | ~60 minutes

Docker E2E Test Suite

Layer / File(s) Summary
Docker images and compose topology
tests/e2e/Dockerfile.*, tests/e2e/docker-compose.yml, tests/e2e/fixtures/*, .dockerignore
Adds server/client Dockerfiles, entrypoint/config fixtures, netem presets, and a 3-container privileged compose topology.
Helper and metrics libraries
tests/e2e/lib/helpers.sh, tests/e2e/lib/metrics.sh
Adds container exec/readiness/log/assert helpers and RTT/iperf3 metric collection functions.
Test scenarios
tests/e2e/scenarios/*
Adds connectivity, performance, and failure/reconnect/shutdown scenario scripts writing JSON pass/fail results.
Runner and CI
tests/e2e/run.sh, .github/workflows/e2e.yml, .gitignore, README.md
Adds run.sh orchestrating compose/token/scenarios and summary.txt, plus a CI workflow uploading results.
Implementation plan doc
docs/superpowers/plans/2026-07-08-e2e-impl.md
Adds a detailed implementation plan document covering all e2e tasks.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RunSh as run.sh
  participant Compose as docker-compose
  participant Server as mesh-server
  participant ClientA as mesh-client-a
  participant ClientB as mesh-client-b

  RunSh->>Compose: up server (dummy token)
  Compose->>Server: start meshd (self-signed TLS)
  RunSh->>Server: wait_for_server / get_token
  Server-->>RunSh: real token
  RunSh->>Compose: up client-a, client-b (MESH_TOKEN)
  Compose->>ClientA: entrypoint-client.sh join+up
  Compose->>ClientB: entrypoint-client.sh join+up
  ClientA->>Server: mesh join / WebSocket connect
  ClientB->>Server: mesh join / WebSocket connect
  RunSh->>ClientA: run scenario scripts (01/02/03)
  ClientA->>ClientB: ping/iperf3 traffic via mesh0
  RunSh->>RunSh: aggregate JSON results into summary.txt
Loading
sequenceDiagram
  participant TunReadLoop as tunReadLoop
  participant PktCh as pktCh (channel)
  participant Connect as connect()
  participant WS as WebSocket

  TunReadLoop->>TunReadLoop: read packet from TUN device
  TunReadLoop->>PktCh: forward packet
  Connect->>PktCh: select and receive packet
  Connect->>WS: write packet, update TX counters
  Connect->>Connect: on ctx.Done(), exit loop
  TunReadLoop->>TunReadLoop: continue reading across reconnects
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed Title accurately summarizes the PR’s main additions: Docker-based e2e testing plus fixes for the Linux data-plane bugs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch e2e/impl

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

🧹 Nitpick comments (4)
internal/api/server.go (1)

67-71: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Consider adding ReadHeaderTimeout to the http.Server.

The server is constructed without read timeouts, leaving it vulnerable to slowloris-style resource exhaustion. This is pre-existing but the construction was restructured in this diff. A minimal fix is ReadHeaderTimeout.

♻️ Proposed addition
 	srv := &http.Server{
 		Addr:    s.cfg.ListenAddr,
 		Handler: s.Handler(),
+		ReadHeaderTimeout: 10 * time.Second,
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/api/server.go` around lines 67 - 71, The http.Server created in
ListenAndServeTLS is missing a read timeout, so add ReadHeaderTimeout to the
server construction alongside Addr and Handler. Use the existing Server setup in
ListenAndServeTLS and set a reasonable header-read timeout on the srv
initialization to protect against slowloris-style requests.
.github/workflows/e2e.yml (1)

3-8: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Consider adding a concurrency group to cancel stale workflow runs.

Without a concurrency group, pushing multiple commits to the same PR can queue redundant e2e runs. A concurrency group cancels in-progress runs when a new commit is pushed.

♻️ Optional: add concurrency group
 on:
   push:
     branches: [master]
   pull_request:
     branches: [master]

+concurrency:
+  group: e2e-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e.yml around lines 3 - 8, Add a concurrency group to the
e2e workflow so newer pushes cancel stale in-progress runs and avoid redundant
queued executions. Update the workflow in the e2e.yml job setup near the
existing on: trigger block by adding a concurrency configuration that groups
runs per PR/branch and enables cancel-in-progress. Use the workflow’s existing
e2e run context so the setting applies consistently to pull_request and push
events.
tests/e2e/fixtures/entrypoint-server.sh (1)

8-11: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Prefer mktemp over hardcoded /tmp/init.log.

Using a predictable /tmp path is vulnerable to symlink attacks (CWE-377). While risk is low in an ephemeral Docker container, mktemp is best practice and costs nothing.

🔒 Proposed fix
- meshd init 2>&1 | tee /tmp/init.log
- # meshd init 输出 "Initialized.\nToken: xxx",提取 token 写入文件供 e2e 读取
- # 用 sed 比 grep -oP 更可移植(不依赖 PCRE)
- sed -n 's/^Token: //p' /tmp/init.log > /etc/mesh/token 2>/dev/null || true
- if [ ! -s /etc/mesh/token ]; then
-   echo "ERROR: failed to extract token from init output" >&2
-   exit 1
- fi
+ init_log="$(mktemp)"
+ trap 'rm -f "$init_log"' EXIT
+ meshd init 2>&1 | tee "$init_log"
+ # meshd init 输出 "Initialized.\nToken: xxx",提取 token 写入文件供 e2e 读取
+ # 用 sed 比 grep -oP 更可移植(不依赖 PCRE)
+ sed -n 's/^Token: //p' "$init_log" > /etc/mesh/token 2>/dev/null || true
+ if [ ! -s /etc/mesh/token ]; then
+   echo "ERROR: failed to extract token from init output" >&2
+   exit 1
+ fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/fixtures/entrypoint-server.sh` around lines 8 - 11, The init log
handling in entrypoint-server.sh uses a predictable /tmp/init.log path, which
should be replaced with a unique temporary file created via mktemp. Update the
meshd init pipeline and the subsequent sed extraction to use that temp file name
consistently, and clean it up afterward if needed; keep the existing token
parsing flow intact.
tests/e2e/docker-compose.yml (1)

16-19: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Consider dropping privileged: true in favor of granular capabilities.

privileged: true grants full container access, making the cap_add entries redundant. With NET_ADMIN, NET_RAW, and /dev/net/tun device access already specified, privileged mode is likely unnecessary for TUN creation and tc netem. Removing it reduces attack surface per least-privilege principle.

Test without privileged: true first; if certain operations (e.g., sysctl, iptables) fail, add the specific capabilities or syscalls needed rather than re-enabling full privileged mode.

🔒 Proposed change (apply to all three services)
     container_name: mesh-server
-    privileged: true
     cap_add:
       - NET_ADMIN
       - NET_RAW

Also applies to: 37-41, 57-61

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/docker-compose.yml` around lines 16 - 19, Remove the unnecessary
privileged mode from the Docker Compose services and rely on the existing
granular permissions instead. Update the relevant service definitions in
docker-compose.yml so `privileged: true` is dropped while keeping the required
`cap_add` entries and `/dev/net/tun` access for the containers that need TUN and
`tc netem`. If any setup step used by these services (for example in the test
environment) fails afterward, add only the specific missing capability or device
permission rather than restoring full privileged access.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e.yml:
- Line 14: The checkout step in the e2e workflow is persisting GitHub
credentials unnecessarily, which leaves the token in git config after the
repository is fetched. Update the existing actions/checkout@v4 step to disable
credential persistence by setting persist-credentials to false, keeping the
change localized to this checkout configuration since the workflow only runs
tests and uploads artifacts.

In `@docs/superpowers/plans/2026-07-08-e2e-impl.md`:
- Around line 486-511: The client entrypoint currently runs mesh join
unconditionally in entrypoint-client.sh, which causes restart/start flows to
fail when the client is already registered. Update the entrypoint logic around
the mesh join call so it first detects an existing client config or registration
state and skips join when already enrolled, while still preserving the normal
join path for first-time startup. Keep the change localized to the entrypoint
script and the join/setup flow used before exec mesh up.
- Around line 1182-1194: The client startup flow still reuses the placeholder
MESH_TOKEN because `docker compose start` does not apply the
`MESH_TOKEN="$REAL_TOKEN"` prefix to existing containers. Update the
`wait_for_server` / `get_token` / `docker compose start` sequence so the real
token is obtained before the clients are created, or recreate `client-a` and
`client-b` with the final token instead of starting the old containers. Ensure
the final token is the one used by the client containers after `REAL_TOKEN` is
fetched.
- Around line 877-910: The mesh validation steps wrap several `dex` invocations
in `bash -c`, which breaks execution because `assert` already runs the command
directly and the new shell loses access to `dex`. Update the affected checks in
the mesh test block to call `dex` directly, removing the `bash -c` wrappers
around the route, unreachable-IP, and offline ping assertions while keeping the
existing `assert` names and command logic intact.

In `@internal/config/config.go`:
- Around line 27-33: The applyTestMode behavior in Config should stop treating
"off" as an enabled value for TLSTestMode, since it implies disabling test TLS
but currently turns it on. Update the switch in applyTestMode to remove "off"
from the accepted true-like cases, and adjust the corresponding config_test.go
expectation for the "off" MESH_TEST_TLS case to assert TLSTestMode stays false.

In `@internal/tun/tun_linux.go`:
- Around line 46-49: The CreateTUNFromFile path is leaking the passed-in file
descriptor on early failures. Update the TUN creation flow around
wgtun.CreateTUNFromFile and the surrounding CreateTUN / tun_linux logic so any
error return after taking ownership of file also closes it before returning. Use
the CreateTUNFromFile and file.Close paths as the main points to locate and fix
every error branch that can exit without releasing the fd.

In `@internal/tunnel/client.go`:
- Around line 107-111: The tunReadLoop in tc.tunReadLoop is swallowing all
tc.tun.Read errors, which hides unexpected TUN failures. Update the error branch
to distinguish normal shutdown/ctx cancellation from real read failures, and use
the existing client logging path to record unexpected errors before returning.
Keep the silent return only for expected Close()-driven exits; otherwise emit a
clear log message with the read error so packet flow failures are diagnosable.

In `@tests/e2e/fixtures/entrypoint-server.sh`:
- Around line 6-15: The entrypoint currently runs meshd init on every startup
and overwrites /etc/mesh/token, which regenerates the bootstrap token on
restart. Update entrypoint-server.sh to preserve the existing token by checking
for an already-initialized DB or existing token before calling meshd init, and
only extract/write the token when initialization is actually needed. Use the
existing meshd init, /tmp/init.log, and /etc/mesh/token flow as the main
locations to adjust.

In `@tests/e2e/scenarios/01-connectivity.sh`:
- Line 25: The MESH_TOKEN setup masks failures from get_token because export
succeeds even when the command substitution returns empty, so split the
declaration and assignment in the connectivity script and keep the get_token
call explicit so a failure is surfaced immediately under set -e. Update the
token initialization near the existing MESH_TOKEN export to first assign the
command output to MESH_TOKEN, then export it separately.

In `@tests/e2e/scenarios/02-performance.sh`:
- Around line 10-19: Add an EXIT cleanup trap in the performance scenario script
so netem is always restored even if a command fails after applying `netem wan`.
Update the `tests/e2e/scenarios/02-performance.sh` flow around the netem setup
and the existing cleanup section to register a trap that calls the same restore
logic used at the end, keyed off the scenario’s setup/teardown commands and the
`netem wan` application. The explicit cleanup block can remain if it is
idempotent, but the trap must guarantee baseline restoration on early exit
caused by `set -euo pipefail`.

---

Nitpick comments:
In @.github/workflows/e2e.yml:
- Around line 3-8: Add a concurrency group to the e2e workflow so newer pushes
cancel stale in-progress runs and avoid redundant queued executions. Update the
workflow in the e2e.yml job setup near the existing on: trigger block by adding
a concurrency configuration that groups runs per PR/branch and enables
cancel-in-progress. Use the workflow’s existing e2e run context so the setting
applies consistently to pull_request and push events.

In `@internal/api/server.go`:
- Around line 67-71: The http.Server created in ListenAndServeTLS is missing a
read timeout, so add ReadHeaderTimeout to the server construction alongside Addr
and Handler. Use the existing Server setup in ListenAndServeTLS and set a
reasonable header-read timeout on the srv initialization to protect against
slowloris-style requests.

In `@tests/e2e/docker-compose.yml`:
- Around line 16-19: Remove the unnecessary privileged mode from the Docker
Compose services and rely on the existing granular permissions instead. Update
the relevant service definitions in docker-compose.yml so `privileged: true` is
dropped while keeping the required `cap_add` entries and `/dev/net/tun` access
for the containers that need TUN and `tc netem`. If any setup step used by these
services (for example in the test environment) fails afterward, add only the
specific missing capability or device permission rather than restoring full
privileged access.

In `@tests/e2e/fixtures/entrypoint-server.sh`:
- Around line 8-11: The init log handling in entrypoint-server.sh uses a
predictable /tmp/init.log path, which should be replaced with a unique temporary
file created via mktemp. Update the meshd init pipeline and the subsequent sed
extraction to use that temp file name consistently, and clean it up afterward if
needed; keep the existing token parsing flow intact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fc02c7a2-84b4-4b88-bc95-2ab934dfd535

📥 Commits

Reviewing files that changed from the base of the PR and between a181dcf and 15e6c52.

📒 Files selected for processing (35)
  • .dockerignore
  • .github/workflows/e2e.yml
  • .gitignore
  • README.md
  • cmd/mesh/main.go
  • docs/superpowers/plans/2026-07-08-e2e-impl.md
  • docs/todo/README.md
  • docs/todo/bug/B00.md
  • docs/todo/bug/B01.md
  • docs/todo/bug/bug.md
  • internal/api/server.go
  • internal/client/config.go
  • internal/client/join.go
  • internal/client/join_test.go
  • internal/client/peers.go
  • internal/client/up.go
  • internal/config/config.go
  • internal/config/config_test.go
  • internal/tun/tun.go
  • internal/tun/tun_linux.go
  • internal/tun/tun_other.go
  • internal/tunnel/client.go
  • tests/e2e/Dockerfile.client
  • tests/e2e/Dockerfile.server
  • tests/e2e/docker-compose.yml
  • tests/e2e/fixtures/entrypoint-client.sh
  • tests/e2e/fixtures/entrypoint-server.sh
  • tests/e2e/fixtures/meshd.yaml
  • tests/e2e/fixtures/netem.sh
  • tests/e2e/lib/helpers.sh
  • tests/e2e/lib/metrics.sh
  • tests/e2e/run.sh
  • tests/e2e/scenarios/01-connectivity.sh
  • tests/e2e/scenarios/02-performance.sh
  • tests/e2e/scenarios/03-failure.sh

Comment thread .github/workflows/e2e.yml
Comment on lines +486 to +511
- [ ] **Step 5: 写 client entrypoint(占位,正式 join 逻辑在 T04)**

`tests/e2e/fixtures/entrypoint-client.sh`:

```bash
#!/usr/bin/env bash
set -euo pipefail

# 等待 server 起来
echo "waiting for server..."
for i in $(seq 1 60); do
if curl -kfsS https://server:443/api/devices >/dev/null 2>&1; then
echo "server is up"
break
fi
sleep 1
done

# 读取由 run.sh 注入的 TOKEN,注册本设备
: "${MESH_TOKEN:?MESH_TOKEN must be set}"
HOSTNAME_ID="${HOSTNAME_ID:-$(hostname)}"
mesh join server --token "$MESH_TOKEN" --insecure

# 启动隧道(前台)
exec mesh up
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Make the client entrypoint idempotent.

mesh join is unconditional here, but Join rejects an existing client config. Any docker compose stop/start will rerun the entrypoint and fail on "already registered", which breaks the restart/reconnect scenarios in this plan.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-07-08-e2e-impl.md` around lines 486 - 511, The
client entrypoint currently runs mesh join unconditionally in
entrypoint-client.sh, which causes restart/start flows to fail when the client
is already registered. Update the entrypoint logic around the mesh join call so
it first detects an existing client config or registration state and skips join
when already enrolled, while still preserving the normal join path for
first-time startup. Keep the change localized to the entrypoint script and the
join/setup flow used before exec mesh up.

Comment on lines +877 to +910
assert "client-a has mesh route" \
bash -c "dex mesh-client-a ip route | grep -q '10.100.0.0/24.*mesh0'"

# 01.4 ping server
assert "client-a ping server" \
dex mesh-client-a ping -c 3 -W 2 "$SERVER_IP"

# 01.5 ping peer
assert "client-a ping client-b" \
dex mesh-client-a ping -c 3 -W 2 "$B_IP"

# 01.6 不存在的 IP 必须 100% 丢包
NONEXIST=$(dex mesh-client-a ping -c 3 -W 1 10.100.0.99 2>&1 || true)
LOSS=$(echo "$NONEXIST" | grep -oP '\K\d+(?=% packet loss)' || echo 0)
assert "unreachable IP 100% loss" \
bash -c "[ '$LOSS' -eq 100 ]"

# 01.7 断开 client-b
docker compose -f "$COMPOSE" stop client-b >/dev/null
sleep 3
assert "client-b offline: a->b fails" \
bash -c "! dex mesh-client-a ping -c 3 -W 2 $B_IP 2>/dev/null"

# 01.8 重启 client-b,恢复
docker compose -f "$COMPOSE" start client-b >/dev/null
wait_for_client mesh-client-b
assert "client-b reconnect: a->b works" \
dex mesh-client-a ping -c 3 -W 3 "$B_IP"

# 01.9 fping 并发不丢
assert "fping burst no loss" \
bash -c "dex mesh-client-a fping -q -c 20 -p 100 $B_IP 2>&1 | grep -q '0% loss' || \
dex mesh-client-a fping -q -c 20 -p 100 $B_IP 2>&1 | grep -oP '\K\d+(?=% loss)' | head -1 | awk '{exit !(\$1<5)}'"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

git ls-files docs/superpowers/plans/2026-07-08-e2e-impl.md
rg -n '(^|[[:space:]])assert\s|(^|[[:space:]])dex\s|function assert|function dex|alias dex|dex\(' docs/superpowers/plans/2026-07-08-e2e-impl.md docs -g '!**/node_modules/**'
sed -n '840,930p' docs/superpowers/plans/2026-07-08-e2e-impl.md

Repository: kvmaker/mesh

Length of output: 9991


🏁 Script executed:

sed -n '580,690p' docs/superpowers/plans/2026-07-08-e2e-impl.md

Repository: kvmaker/mesh

Length of output: 2535


🏁 Script executed:

sed -n '620,690p' docs/superpowers/plans/2026-07-08-e2e-impl.md
python3 - <<'PY'
import subprocess, textwrap, os, tempfile, json, sys

script = r'''
#!/usr/bin/env bash
set -euo pipefail

dex() { echo "DEX:$*"; }
assert() {
  local label="$1"; shift
  if "$@"; then
    echo "PASS:$label"
  else
    echo "FAIL:$label"
  fi
}

echo "--- direct ---"
assert direct dex mesh-client-a ping -c 1 1.2.3.4

echo "--- bash-c with function ---"
if assert bashc bash -c "dex mesh-client-a ping -c 1 1.2.3.4" 2>&1; then :; fi

echo "--- negate syntax ---"
assert negate ! dex mesh-client-a ping -c 1 1.2.3.4

echo "--- pipeline syntax ---"
assert pipe dex mesh-client-a ip route | grep -q 'mesh0'
'''
p = subprocess.run(["bash", "-lc", script], capture_output=True, text=True)
print(p.stdout)
print(p.stderr, file=sys.stderr)
print("exit", p.returncode)
PY

Repository: kvmaker/mesh

Length of output: 2055


🏁 Script executed:

python3 - <<'PY'
import subprocess, textwrap

script = r'''
set -e
dex() { echo DEX; }

echo "parent:"
dex

echo "child:"
bash -c 'type dex >/dev/null 2>&1 && echo inherited || echo missing'
'''
p = subprocess.run(["bash", "-c", script], capture_output=True, text=True)
print(p.stdout)
print("exit", p.returncode)
PY

Repository: kvmaker/mesh

Length of output: 185


Drop the bash -c wrappers around dex. assert already runs the command directly; bash -c starts a fresh shell where dex is undefined, so these checks fail even when the mesh is healthy.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-07-08-e2e-impl.md` around lines 877 - 910, The
mesh validation steps wrap several `dex` invocations in `bash -c`, which breaks
execution because `assert` already runs the command directly and the new shell
loses access to `dex`. Update the affected checks in the mesh test block to call
`dex` directly, removing the `bash -c` wrappers around the route,
unreachable-IP, and offline ping assertions while keeping the existing `assert`
names and command logic intact.

Comment on lines +1182 to +1194
# 启动容器(需要 MESH_TOKEN)
export MESH_TOKEN="${MESH_TOKEN:-$(uuidgen)}"
docker compose -f "$COMPOSE" up -d --build
trap 'docker compose -f "$COMPOSE" down >/dev/null 2>&1 || true' EXIT

wait_for_server
# 把真实 token 注入两个 client 容器(entrypoint 已 join)
# 重新读 token 确保 client 用对
REAL_TOKEN=$(get_token)
docker compose -f "$COMPOSE" stop client-a client-b >/dev/null
MESH_TOKEN="$REAL_TOKEN" docker compose -f "$COMPOSE" start client-a client-b >/dev/null
wait_for_client mesh-client-a
wait_for_client mesh-client-b

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

file="docs/superpowers/plans/2026-07-08-e2e-impl.md"

echo "== context =="
wc -l "$file"
sed -n '1160,1210p' "$file" | cat -n

echo
echo "== relevant references in repo =="
rg -n "docker compose .*start client-a client-b|MESH_TOKEN|wait_for_client|get_token|entrypoint" -S docs/superpowers/plans/2026-07-08-e2e-impl.md .

Repository: kvmaker/mesh

Length of output: 14279


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Check Docker Compose semantics from installed docs/help if available.
docker compose start --help 2>/dev/null | sed -n '1,120p' || true
printf '\n---\n'
docker compose up --help 2>/dev/null | sed -n '1,120p' || true

Repository: kvmaker/mesh

Length of output: 155


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Programmatic probe of compose env interpolation behavior, using a minimal YAML only if docker compose is available.
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT

cat >"$tmpdir/compose.yaml" <<'YAML'
services:
  test:
    image: alpine
    environment:
      - MESH_TOKEN=${MESH_TOKEN:-placeholder}
YAML

echo "== config with default env =="
( cd "$tmpdir" && docker compose -f compose.yaml config 2>/dev/null | sed -n '1,120p' ) || true

echo "== config with override env =="
( cd "$tmpdir" && MESH_TOKEN=real docker compose -f compose.yaml config 2>/dev/null | sed -n '1,120p' ) || true

Repository: kvmaker/mesh

Length of output: 211


🌐 Web query:

Docker Compose start does not recreate containers environment variables passed on start are not applied existing container env

💡 Result:

Docker Compose containers are immutable regarding their environment variables; they cannot be updated while the container is running [1]. If you change environment variables in your compose.yaml or.env file, Docker Compose does not automatically apply these changes to existing, running containers [2][3]. To apply updated environment variables, you must recreate the container [2][1]. You can do this by running the following command: docker compose up -d --force-recreate The --force-recreate flag instructs Compose to stop and replace the existing containers, even if their configuration has not changed from the perspective of the Compose engine [4][5]. Key points to understand: 1. Immutability: Environment variables are fixed at the moment of container creation [1]. Recreating the container is the only way to inject new values [2][1]. 2. Detect vs. Force: Docker Compose typically detects changes in your configuration files and recreates containers automatically during a standard docker compose up [5]. If it fails to do so, it may be because the change is not explicitly detected as a configuration diff (such as changes inside an external env_file that was not re-parsed) or because the system believes the container is still current [4][6][5]. Using --force-recreate ensures the update occurs [4][5]. 3. Data Persistence: Recreating a container does not delete your volumes [5]. As long as your data is stored in volumes (defined in the volumes section of your compose file), your data will persist across the recreation process [5]. 4. Env Files vs. Interpolation: There is a distinction between variables used for interpolation (substituting values in the compose.yaml file before the container starts) and variables injected into the container at runtime via env_file [7][8]. Ensure your variables are correctly mapped in your service definition using the environment or env_file keys if you intend for them to be present inside the container environment [9][8].

Citations:


Create the clients after the real token is available. docker compose start reuses the existing container configuration, so the MESH_TOKEN="$REAL_TOKEN" prefix here won’t replace the placeholder value from the initial up -d. Fetch REAL_TOKEN first, or recreate client-a/client-b with the final token instead.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/superpowers/plans/2026-07-08-e2e-impl.md` around lines 1182 - 1194, The
client startup flow still reuses the placeholder MESH_TOKEN because `docker
compose start` does not apply the `MESH_TOKEN="$REAL_TOKEN"` prefix to existing
containers. Update the `wait_for_server` / `get_token` / `docker compose start`
sequence so the real token is obtained before the clients are created, or
recreate `client-a` and `client-b` with the final token instead of starting the
old containers. Ensure the final token is the one used by the client containers
after `REAL_TOKEN` is fetched.

Comment thread internal/config/config.go
Comment thread internal/tun/tun_linux.go
Comment thread internal/tunnel/client.go
Comment thread tests/e2e/fixtures/entrypoint-server.sh Outdated
Comment thread tests/e2e/scenarios/01-connectivity.sh Outdated
Comment thread tests/e2e/scenarios/02-performance.sh
- [Critical] applyTestMode 移除 "off"(语义危险:off 被误解为关闭实则启用自签)
- e2e.yml checkout persist-credentials: false
- tun_linux CreateTUNFromFile error 路径 close fd
- client.go tunReadLoop 记录 unexpected read error
- entrypoint-server mesh.db 存在则跳过 init(token 稳定)
- 01-connectivity 拆分 get_token 避免 set -e 掩盖
- 02-performance 加 cleanup trap 防 netem 残留
- 同步 spec/entrypoint 的 MESH_TEST_TLS=off→on

Co-Authored-By: Claude <noreply@anthropic.com>
@kvmaker
kvmaker merged commit e2ef688 into master Jul 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants