Skip to content

Fix/global proxy dispatcher#43919

Closed
RickyTong1 wants to merge 2 commits intoopenclaw:mainfrom
RickyTong1:fix/global-proxy-dispatcher
Closed

Fix/global proxy dispatcher#43919
RickyTong1 wants to merge 2 commits intoopenclaw:mainfrom
RickyTong1:fix/global-proxy-dispatcher

Conversation

@RickyTong1
Copy link
Copy Markdown

@RickyTong1 RickyTong1 commented Mar 12, 2026

Summary

  • Problem: LLM inference requests time out when the gateway runs behind a proxy (HTTP_PROXY / HTTPS_PROXY / ALL_PROXY), because Node.js 22+'s globalThis.fetch (undici) does not read proxy env vars by default, and the upstream streamSimple in @mariozechner/pi-ai does not accept a dispatcher parameter.
  • Why it matters: Users in corporate firewalls, and other restricted networks cannot use OpenClaw for LLM inference at all — the only working path (curl -x) proves the proxy itself is fine.
  • What changed: Added applyGlobalProxyDispatcher() in src/infra/net/global-proxy.ts that calls setGlobalDispatcher(new EnvHttpProxyAgent()) early in gateway startup (after dotenv loading), making all globalThis.fetch calls proxy-aware. Added ALL_PROXY / socks5:// / socks5h:// normalization in src/infra/net/proxy-env.ts. 17 unit tests.
  • What did NOT change (scope boundary): No modifications to existing Telegram, WhatsApp, media-understanding, or web-tools proxy log.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • When HTTP_PROXY, HTTPS_PROXY, or ALL_PROXY env vars are set, LLM inference requests now route through the proxy instead of timing out.
  • [net/global-proxy] global undici dispatcher set to EnvHttpProxyAgent (via ...) log line appears at startup when proxy is active.
  • ALL_PROXY with socks5:// or socks5h:// is rewritten to http:// for undici compatibility (most local proxy tools like Clash/V2Ray accept HTTP CONNECT on the same port).
  • NO_PROXY / no_proxy exclusions are respected — localhost services (Ollama, browser control, canvas host) are not affected.

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? Yes — existing globalThis.fetch calls are now routed through the user-configured proxy when env vars are set. No new outbound calls are introduced.
  • Command/tool execution surface changed? No
  • Data access scope changed? No
  • If any Yes, explain risk + mitigation: The proxy dispatcher only activates when the user explicitly sets proxy env vars. EnvHttpProxyAgent respects NO_PROXY exclusions, so internal/localhost traffic is unaffected. If EnvHttpProxyAgent construction fails (e.g. malformed URL), the error is logged and the gateway continues with direct connections.

Repro + Verification

Environment

  • OS: macOS (Darwin 25.2.0), also affects Linux
  • Runtime/container: Node.js 22+
  • Model/provider: Any (Google Gemini, Anthropic, OpenAI-compatible)
  • Integration/channel (if any): All channels — the fix is at the global fetch layer
  • Relevant config (redacted): HTTP_PROXY=http://127.0.0.1:7897 HTTPS_PROXY=http://127.0.0.1:7897 ALL_PROXY=socks5://127.0.0.1:7897

Steps

  1. Be in a network where direct connections to LLM APIs are blocked (common in China, corporate firewalls)
  2. Start gateway: env HTTP_PROXY="http://127.0.0.1:7897" HTTPS_PROXY="http://127.0.0.1:7897" openclaw gateway
  3. Send any message that triggers LLM inference

Expected

  • LLM inference requests succeed through the proxy
  • [net/global-proxy] global undici dispatcher set to EnvHttpProxyAgent (via HTTP_PROXY) in startup log

Actual (before fix)

  • Repeated LLM request timed out errors, even though curl -x http://127.0.0.1:7897 https://generativelanguage.googleapis.com/... succeeds

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Unit tests (17 cases, all passing):

✓ sets global dispatcher when HTTPS_PROXY is set
✓ sets global dispatcher when HTTP_PROXY is set
✓ sets global dispatcher when ALL_PROXY is set
✓ rewrites socks5:// ALL_PROXY to http:// for EnvHttpProxyAgent compatibility
✓ rewrites socks5h:// ALL_PROXY to http:// for EnvHttpProxyAgent compatibility
✓ passes http:// ALL_PROXY as-is
✓ passes all_proxy (lowercase) as explicit httpProxy/httpsProxy
✓ prefers lowercase all_proxy over uppercase ALL_PROXY
✓ does not pass explicit options when HTTP_PROXY is also set alongside ALL_PROXY
✓ does not pass explicit options when HTTPS_PROXY is set (no ALL_PROXY fallback needed)
✓ sets global dispatcher when lowercase proxy vars are set
✓ is a no-op when no proxy env var is set
✓ only applies once even if called multiple times
✓ skips if a proxy-like dispatcher is already installed
✓ does not skip if existing dispatcher is not proxy-like
✓ handles EnvHttpProxyAgent constructor failure gracefully

Regression tests pass with zero regressions:

  • src/infra/net/proxy-fetch.test.ts — 7/7 passed
  • src/infra/net/fetch-guard.ssrf.test.ts — 9/9 passed

Startup log confirming proxy activation:

2026-03-13T14:56:08.218+08:00 [net/global-proxy] global undici dispatcher set to EnvHttpProxyAgent (via HTTP_PROXY)

Human Verification

  • Verified scenarios:
    • Built and installed locally via npm link (OpenClaw 2026.3.13, commit 2b46c4c)
    • Started gateway with env OPENCLAW_DISABLE_BONJOUR=1 HTTP_PROXY="http://127.0.0.1:7897" HTTPS_PROXY="http://127.0.0.1:7897" ALL_PROXY="socks5://127.0.0.1:7897" openclaw gateway run --force
    • Confirmed [net/global-proxy] global undici dispatcher set to EnvHttpProxyAgent (via HTTP_PROXY) in startup log
    • Gateway started successfully, all channels connected, subagent dispatch unaffected
    • Localhost services (browser control on :18791, canvas host) remain accessible without proxy interference
  • Edge cases checked:
    • socks5h:// protocol variant (was broken by original regex, now fixed)
    • ALL_PROXY-only configuration (no HTTP_PROXY/HTTPS_PROXY set)
    • Idempotent re-invocation (safe to call multiple times)
    • Cooperative skip when another proxy dispatcher is already installed
    • Graceful failure on malformed proxy URL
  • What I did not verify:
    • Windows-specific behavior (lowercase vs uppercase env var precedence)
    • SOCKS-only proxy endpoints that do not accept HTTP CONNECT on the same port

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No — uses existing standard proxy env vars (HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, NO_PROXY)
  • Migration needed? No

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: Remove the applyGlobalProxyDispatcher() call in src/gateway/server.impl.ts (single line). Or simply unset proxy env vars — the function is a no-op without them.
  • Files/config to restore: src/gateway/server.impl.ts (remove import + call)
  • Known bad symptoms reviewers should watch for: If a SOCKS-only proxy does not accept HTTP CONNECT, the rewrite from socks5:// to http:// will cause connection failures. Users would see fetch errors instead of timeouts.

Risks and Mitigations

  • Risk: SOCKS5-to-HTTP rewrite assumes the proxy accepts HTTP CONNECT on the same port.
    • Mitigation: This is true for all major local proxy tools (Clash, V2Ray, Shadowsocks, Surge). For truly SOCKS-only endpoints, the connection will fail at connect time with a clear error, which is still better than silently timing out.
  • Risk: Global dispatcher affects all globalThis.fetch calls process-wide.
    • Mitigation: This is intentional — the whole point is to cover streamSimple which only uses globalThis.fetch. EnvHttpProxyAgent respects NO_PROXY so internal traffic is excluded. The function is cooperative (skips if a proxy dispatcher is already installed) and idempotent.

Comparison with existing open PRs

PR Approach Limitations
#33990 ProxyAgent in server-startup.ts Does not respect NO_PROXY; modifies Telegram code
#42320 EnvHttpProxyAgent in undici-global-dispatcher.ts Only covers embedded runner path, not startup/heartbeat/web-search
#41455 Similar to #42320 Excludes ALL_PROXY; only covers embedded runner

Files changed

  • src/infra/net/global-proxy.ts — New module: applyGlobalProxyDispatcher() + test reset helper
  • src/infra/net/global-proxy.test.ts — 17 unit tests
  • src/infra/net/proxy-env.ts — Added resolveAllProxyFallbackOptions(), SOCKS protocol normalization (builds on resolveEnvHttpProxyUrl / hasEnvHttpProxyConfigured from Fix env proxy bootstrap for model traffic #43248)
  • src/gateway/server.impl.ts — Import and call applyGlobalProxyDispatcher() after readConfigFileSnapshot()

Relationship with #43248

#43248 (merged 2026-03-11) added ensureGlobalUndiciEnvProxyDispatcher() in undici-global-dispatcher.ts, called inside the embedded runner's runEmbeddedAttempt(). This PR complements it with two things #43248 does not cover:

#43248 (merged) This PR
Injection point runEmbeddedAttempt() — per-run, embedded runner only startGatewayServer() — once at gateway startup, covers all fetch paths
Coverage Model traffic via embedded runner All globalThis.fetch: LLM inference, heartbeat, web search, usage tracking, etc.
ALL_PROXY support No — explicitly excluded Yes — resolveAllProxyFallbackOptions() bridges undici's EnvHttpProxyAgent gap
SOCKS protocol No Yes — socks5://, socks5h://, socks4:// rewritten to http://
dotenv timing N/A (runs after config is loaded) Explicitly placed after readConfigFileSnapshot() so .env-defined proxy vars are visible

This PR reuses the resolveEnvHttpProxyUrl and hasEnvHttpProxyConfigured helpers that #43248 added to proxy-env.ts, and adds resolveAllProxyFallbackOptions() on top for the ALL_PROXY/SOCKS gap.

@openclaw-barnacle openclaw-barnacle Bot added channel: whatsapp-web Channel integration: whatsapp-web gateway Gateway runtime size: S labels Mar 12, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 12, 2026

Greptile Summary

This PR fixes LLM inference timeouts for users behind proxies (corporate firewalls, restricted networks) by installing an undici EnvHttpProxyAgent as the global fetch dispatcher at gateway startup, making all globalThis.fetch calls — including those inside @mariozechner/pi-ai — proxy-aware. It also bridges the gap where EnvHttpProxyAgent ignores ALL_PROXY, and normalizes SOCKS5/5h URLs to http:// for undici compatibility.

  • src/infra/net/global-proxy.ts — New applyGlobalProxyDispatcher(): cooperative (skips if a proxy-like dispatcher already exists), idempotent (latches only on success — retries are allowed when no proxy vars were set or when the constructor fails), and safe (catch-and-warn on constructor error). Always merges loopback addresses into no_proxy to protect internal services like Ollama.
  • src/infra/net/proxy-env.ts — Adds resolveAllProxyFallbackOptions() to bridge ALL_PROXY/SOCKS protocol gaps. The SOCKS_PROTOCOL_RE regex covers socks4, socks4a, socks5, socks5h but misses the socks4h:// variant (used by some proxy clients for remote DNS).
  • src/gateway/server.impl.ts — Single call placed correctly after readConfigFileSnapshot() so .env-defined proxy vars are visible.
  • src/infra/net/global-proxy.test.ts — 24 test cases (the PR description cites 17 — appears to have been written before the loopback and retry tests were added). Tests are thorough and use proper module isolation via vi.resetModules().

Confidence Score: 4/5

  • Safe to merge; the proxy dispatcher only activates when the user explicitly sets proxy env vars, and the loopback bypass protects internal services from being accidentally proxied.
  • The implementation is well-designed with cooperative dispatch, idempotency, graceful failure handling, and thorough test coverage. The two minor issues found are: (1) the applied latch JSDoc describes incorrect "one-shot" semantics while the code intentionally allows retries, and (2) the SOCKS regex misses the rare socks4h:// variant. Neither affects correctness for the target use cases.
  • src/infra/net/proxy-env.ts for the socks4h:// regex gap; src/infra/net/global-proxy.ts for the misleading applied latch comment.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/infra/net/proxy-env.ts
Line: 69

Comment:
**`socks4h://` not covered by SOCKS protocol regex**

The regex handles `socks4a://` and `socks5h://` but misses `socks4h://`, which is a valid SOCKS4a variant supported by curl and some proxy clients (Shadowrocket, etc.) for remote DNS resolution. Given the PR's stated goal of covering all major SOCKS variants, this is an easy edge case to add.

```suggestion
const SOCKS_PROTOCOL_RE = /^socks(?:4[ah]?|5h?)?:\/\//i;
```

The updated non-capturing group reads: `4[ah]?` matches `socks4`, `socks4a`, `socks4h`; `5h?` matches `socks5`, `socks5h`.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/infra/net/global-proxy.ts
Line: 42-47

Comment:
**"One-shot latch" comment contradicts actual retry behavior**

The JSDoc on `applied` says *"One-shot latch"* and *"proxy env changes require a full gateway restart"*, and the exported function docstring says *"only the first invocation takes effect"*. These statements are contradicted by the implementation (and explicitly tested): `applied` is **not** set when the early-return guard fires (no proxy vars) or when the constructor throws, so a second call can succeed in both scenarios.

The comment should reflect the real semantics — latch locks only on a successful application (or when a pre-existing proxy dispatcher is detected):

```suggestion
/**
 * Success latch: set to `true` only after `EnvHttpProxyAgent` is
 * successfully installed (or a compatible dispatcher is already present).
 * A no-op early return (no proxy vars) or a constructor error leaves this
 * `false`, allowing a subsequent call to retry.
 *
 * Proxy env changes after a successful application require a full gateway
 * restart — the latch prevents redundant re-installation.
 */
let applied = false;
```

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: "fix(net): add global..."

Comment thread src/web/session.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dafff91035

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/web/session.ts Outdated
@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch from dafff91 to b4c5ff1 Compare March 12, 2026 10:15
@openclaw-barnacle openclaw-barnacle Bot removed the channel: whatsapp-web Channel integration: whatsapp-web label Mar 12, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7626662e2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/global-proxy.ts
@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch from f762666 to 22b5399 Compare March 12, 2026 11:03
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 22b5399fa3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/gateway/server.impl.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e2abda8782

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/global-proxy.ts Outdated
Comment thread src/infra/net/global-proxy.ts Outdated
@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch from e2abda8 to f5749de Compare March 13, 2026 01:42
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5749de77b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/gateway/server.impl.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ff1d32adb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/proxy-env.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 049facf2f0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/proxy-env.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5897b5a4d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/proxy-env.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 452c1f0bf7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/global-proxy.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 24e9a83019

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/gateway/server.impl.ts Outdated
@CaptainJackMyName
Copy link
Copy Markdown

This problem has been bothering me for days. Good job~

@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch 2 times, most recently from 5394ef8 to 4623c35 Compare March 16, 2026 02:12
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4623c35b46

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/global-proxy.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c677b2ba5d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/global-proxy.ts Outdated
@openclaw-barnacle openclaw-barnacle Bot added the channel: whatsapp-web Channel integration: whatsapp-web label Mar 18, 2026
@RickyTong1 RickyTong1 marked this pull request as draft March 18, 2026 04:39
@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch from bffb6d7 to 67775a7 Compare March 18, 2026 06:59
@openclaw-barnacle openclaw-barnacle Bot added size: M and removed channel: discord Channel integration: discord channel: imessage Channel integration: imessage channel: signal Channel integration: signal channel: slack Channel integration: slack channel: telegram Channel integration: telegram channel: whatsapp-web Channel integration: whatsapp-web size: L labels Mar 18, 2026
@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch from 67775a7 to 263cc2d Compare March 18, 2026 07:07
@RickyTong1 RickyTong1 marked this pull request as ready for review March 18, 2026 08:08
Comment thread src/infra/net/proxy-env.ts Outdated
Comment thread src/infra/net/global-proxy.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 263cc2d70a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/infra/net/global-proxy.ts
Comment thread src/infra/net/proxy-env.ts Outdated
…d loopback bypass

- Bootstrap undici EnvHttpProxyAgent at gateway startup when proxy env
  vars are detected (HTTP_PROXY, HTTPS_PROXY, ALL_PROXY)
- Normalize socks4/4a/4h/5/5h:// URLs to http:// for undici compat
- Route ALL_PROXY fallback through both httpProxy and httpsProxy
- Ensure loopback addresses (localhost, 127.0.0.1, [::1]) are always
  bypassed via NO_PROXY merging; preserve NO_PROXY=* wildcard
- Success-latch prevents redundant re-installation while allowing
  retry after no-op or constructor failure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@RickyTong1 RickyTong1 force-pushed the fix/global-proxy-dispatcher branch from 263cc2d to a3b4b85 Compare March 18, 2026 09:39
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a3b4b856e0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +72 to +74
if (applied) {
return;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Rebuild proxy dispatcher on in-process gateway restarts

In-process restarts re-enter startGatewayServer() (src/cli/gateway-cli/run-loop.ts:229-233), and each config read can add previously absent proxy vars from .env or config.env (src/config/io.ts:886, src/config/env-vars.ts:79-96). With this latch, those later values are never applied once the first proxy bootstrap succeeds. EnvHttpProxyAgent snapshots HTTP(S)_PROXY/NO_PROXY at construction time, so a gateway that first starts with only HTTP_PROXY will keep using that old proxy even after a restart adds HTTPS_PROXY or NO_PROXY, and users have to do a full process respawn for the change to take effect.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae7a965c96

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +100 to +103
const rawAllProxy = process.env.all_proxy?.trim() || process.env.ALL_PROXY?.trim() || "";
if (rawAllProxy && fallbackOptions.httpsProxy && fallbackOptions.httpsProxy !== rawAllProxy) {
log.warn(
`ALL_PROXY "${rawAllProxy}" rewritten to "${fallbackOptions.httpsProxy}" for undici compatibility — if your proxy only supports SOCKS, LLM requests will fail`,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Redact proxy credentials from the ALL_PROXY rewrite warning

If a user configures an authenticated SOCKS proxy via ALL_PROXY (for example socks5://user:pass@proxy:1080), this warning logs both the original and rewritten proxy URLs verbatim. Startup logs are often shared for debugging, so this change can leak proxy usernames/passwords into log files or issue reports. Please redact URL userinfo here before logging, similar to the existing URL-sanitizing helpers in the repo.

Useful? React with 👍 / 👎.

@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Apr 27, 2026
@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented Apr 27, 2026

Codex automated review: keeping this open.

Keep this PR open. Current main has shipped the narrower HTTP_PROXY/HTTPS_PROXY EnvHttpProxyAgent bootstrap path, but it still intentionally excludes ALL_PROXY/all_proxy and SOCKS fallback behavior. That remaining scope is the central behavior this PR attempts to add, so it is not obsolete; it needs maintainer review, rebase, and security cleanup rather than conservative closure.

Best possible solution:

Keep this PR open for maintainer review, but require a rebase onto current main and a narrower implementation through the existing proxy-env and undici-global-dispatcher helpers. Add explicit tests for ALL_PROXY/all_proxy, socks4/socks4a/socks4h/socks5/socks5h normalization, lowercase-empty precedence, malformed and authenticated proxy URL redaction, NO_PROXY wildcard and loopback behavior, dispatcher replacement, and SSRF-preserving strict paths. If maintainers decide OpenClaw should intentionally keep Undici's HTTP_PROXY/HTTPS_PROXY-only semantics, close this only with that explicit product/security rationale and leave #43821 or a narrower follow-up as the canonical record.

What I checked:

  • Gateway startup has only the centralized env-proxy bootstrap: startGatewayServer() calls bootstrapGatewayNetworkRuntime(), and that wrapper only invokes ensureGlobalUndiciEnvProxyDispatcher(). (src/gateway/server.impl.ts:296, fee16865b229)
  • Current bootstrap is gated on HTTP(S) proxy env only: ensureGlobalUndiciEnvProxyDispatcher() returns unless hasEnvHttpProxyConfigured("https") is true, then constructs a plain EnvHttpProxyAgent without explicit ALL_PROXY/SOCKS fallback options. (src/infra/net/undici-global-dispatcher.ts:91, fee16865b229)
  • Current proxy-env contract explicitly excludes ALL_PROXY: proxy-env.ts documents that ALL_PROXY is ignored by EnvHttpProxyAgent, and proxy-env tests assert ALL_PROXY-only environments are not considered configured for EnvHttpProxyAgent-style resolution. (src/infra/net/proxy-env.ts:28, fee16865b229)
  • Gateway regression coverage proves HTTPS_PROXY startup bootstrap, not ALL_PROXY/SOCKS: The current gateway network runtime e2e test sets HTTPS_PROXY and verifies the global dispatcher becomes EnvHttpProxyAgent; it does not cover ALL_PROXY-only, socks5://, or socks5h:// behavior. (src/gateway/server-network-runtime.e2e.test.ts:60, fee16865b229)
  • ALL_PROXY remains a security-sensitive contract elsewhere: Provider/media SSRF code explains why ALL_PROXY-only auto-upgrade is avoided, and fetch-guard SSRF tests assert ALL_PROXY-only trusted proxy mode keeps DNS pinning instead of using EnvHttpProxyAgent. (src/media-understanding/shared.ts:241, fee16865b229)
  • PR diff and discussion show useful but risky remaining work: Provided PR context shows the head adds src/infra/net/global-proxy.ts, adds resolveAllProxyFallbackOptions()/SOCKS normalization in src/infra/net/proxy-env.ts, and calls the dispatcher from gateway startup. Review comments also identify process-wide routing, NO_PROXY/loopback behavior, mixed-case precedence, and proxy credential redaction concerns. The changed files are limited to gateway/net helpers/tests, with no package, lockfile, workflow, or dependency-source changes, so the security concern is runtime network routing and log redaction rather than supply-chain execution. (ae7a965c968f)

Remaining risk / open question:

  • Closing as implemented would drop the still-missing ALL_PROXY/all_proxy and SOCKS fallback behavior without an explicit maintainer decision to reject that scope.
  • Merging as-is changes process-wide globalThis.fetch routing; it must preserve current SSRF, NO_PROXY, lowercase precedence, and loopback-direct contracts.
  • Review comments flagged possible proxy credential leakage when logging rewritten authenticated proxy URLs; URL userinfo needs redaction before merge.
  • The SOCKS-to-HTTP rewrite assumes the configured SOCKS listener also accepts HTTP CONNECT, which may fail for true SOCKS-only endpoints.
  • The PR needs rebase and integration through current src/infra/net/proxy-env.ts and src/infra/net/undici-global-dispatcher.ts helpers rather than adding an overlapping global dispatcher path.

Codex Review notes: model gpt-5.5, reasoning high; reviewed against fee16865b229.

@steipete
Copy link
Copy Markdown
Contributor

Thanks @RickyTong1. I carried this forward on main with a narrower core fix:

What changed: ALL_PROXY / all_proxy now feed the global Undici EnvHttpProxyAgent path and provider proxy-fetch helper. HTTP_PROXY / HTTPS_PROXY remain the only envs used by SSRF trusted-proxy auto-upgrade checks, so guarded fetch behavior does not get widened by all-proxy settings.

Blacksmith Testbox verification:

  • pnpm check:changed
  • pnpm build
  • pnpm test src/infra/net/proxy-env.test.ts src/infra/net/undici-global-dispatcher.test.ts src/infra/net/proxy-fetch.test.ts src/infra/net/fetch-guard.ssrf.test.ts src/media-understanding/shared.test.ts src/cli/run-main.exit.test.ts
  • Live built-artifact smoke with only ALL_PROXY set: passed via dist/plugin-sdk/runtime-env.js; proxy saw CONNECT all-proxy-live.invalid:80 and the tunneled request.

I also attempted the broad pnpm test suite on Testbox; it is currently red on unrelated infra/Windows shards (src/infra/diagnostic-trace-context.test.ts timeout and Windows ACL assertion failures), outside this proxy change.

Closing this PR because the behavior is now landed on main.

@steipete steipete closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime size: L stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add global HTTP proxy support via environment variables HTTP_PROXY/HTTPS_PROXY environment variables ignored by undici fetch

3 participants