GCP support for application load balancer in front of API connector deployments - #1347
Conversation
…nal_lb_host. Wire ingress and public endpoint URLs from a customer-owned ALB host signal, document the pattern, and teach psoxy-test to call ALB URLs with self-signed TLS. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Claude Code Review
Claude Code Review is paused for this repository. To reconnect it, an admin of this repository's GitHub organization (or the account owner, for personal repositories) who can also manage your Claude organization's Code Review settings needs to re-link GitHub in Code Review settings. This is a one-time step.
Tip: disable this comment in your organization's Code Review settings.
There was a problem hiding this comment.
Pull request overview
Adds beta support for running GCP API connector deployments behind a customer-composed external Application Load Balancer (ALB), and updates the psoxy-test tooling/docs to work with non-*.run.app / non-*.cloudfunctions.net hosts (including IP-based PoC endpoints with self-signed TLS).
Changes:
- Introduces
api_connector_external_lb_hostingcp-hostto publish ALB-based connector endpoints and set Cloud Functions ingress toALLOW_INTERNAL_AND_GCLBwhen enabled. - Extends
psoxy-testto support PoC/self-signed TLS via--allow-insecure-tlsand--cacert, including request option building and new unit tests. - Adds and links documentation + an example (commented) Terraform composition for external ALB + Cloud Armor.
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/psoxy-test/test/utils.test.js | Adds unit tests for buildHttpsRequestOptions (IP host + insecure TLS; custom domain default TLS). |
| tools/psoxy-test/test/gcp.test.js | Updates GCP tests for new logs URL behavior on non-Cloud-Function hosts and updated request signature. |
| tools/psoxy-test/test/aws.test.js | Updates AWS tests for updated request signature (extra TLS options argument). |
| tools/psoxy-test/lib/utils.js | Adds buildHttpsRequestOptions and threads TLS options into the request wrapper. |
| tools/psoxy-test/lib/gcp.js | Passes TLS options through to request layer; clarifies getLogsURL behavior for external ALB/custom domains. |
| tools/psoxy-test/lib/aws.js | Passes TLS options through to request layer. |
| tools/psoxy-test/cli-call.js | Adds CLI flags --allow-insecure-tls and --cacert. |
| infra/modules/gcp-proxy-api/variables.tf | Adds ingress_settings and external_lb_base_url module inputs for ALB composition support. |
| infra/modules/gcp-proxy-api/main.tf | Switches Pub/Sub push to Cloud Function URI; parameterizes ingress; computes public proxy endpoint URL and test CLI flags. |
| infra/modules/gcp-host/variables.tf | Adds api_connector_external_lb_host input with validation and documentation. |
| infra/modules/gcp-host/main.tf | Enforces allowlist presence when ALB host set; wires ALB base URL + ingress settings into connectors; publishes external endpoint URLs. |
| infra/examples-dev/gcp/variables.tf | Adds example variable for api_connector_external_lb_host; updates allowlist description for Cloud Armor parity. |
| infra/examples-dev/gcp/networking.tf | New commented reference composition for external ALB + Cloud Armor (beta). |
| infra/examples-dev/gcp/main.tf | Wires api_connector_external_lb_host into the example root module; extends connector outputs. |
| docs/SUMMARY.md | Adds navigation entry for the new GCP external ALB doc. |
| docs/guides/psoxy-test-tool.md | Documents testing external ALB endpoints, including -f gcp, --allow-insecure-tls, and --cacert. |
| docs/gcp/vpc.md | Clarifies that the VPC doc covers egress only; links to external ALB ingress doc. |
| docs/gcp/README.md | Adds link to VPC (egress) and related external ALB doc. |
| docs/gcp/authentication-authorization.md | Links to external ALB + Cloud Armor doc alongside PSC/connectivity notes. |
| docs/development/README.md | Adds entry for external ALB + Cloud Armor documentation. |
| docs/development/gcp-private-service-connect.md | Adds external ALB + Cloud Armor option to the connectivity matrix and clarifies ILB vs external ALB. |
| docs/development/gcp-external-alb.md | New beta design doc describing the external ALB + Cloud Armor composition pattern and module wiring. |
| docs/configuration/ip-allowlisting.md | Updates GCP guidance to reference the external ALB + Cloud Armor composition path and its relationship to allowed_data_access_ip_blocks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| proxy_endpoint_is_cloud_function = can(regex("\\.run\\.app", local.proxy_endpoint_url)) || can(regex("\\.cloudfunctions\\.net", local.proxy_endpoint_url)) | ||
| proxy_endpoint_host = try(regex("^https?://([^/]+)", local.proxy_endpoint_url), "") | ||
| proxy_endpoint_is_ip = can(regex("^[0-9.]+$", local.proxy_endpoint_host)) || can(regex("^\\[[0-9a-fA-F:]+\\]$", local.proxy_endpoint_host)) | ||
| command_cli_call_flags = trimspace(join(" ", compact([ |
| function requestWrapper(url, method = 'GET', headers, body = {}, tlsOptions = {}) { | ||
| url = typeof url === 'string' ? new URL(url) : url; | ||
| const responseBody = []; | ||
| const requestOptions = buildHttpsRequestOptions(url, method, headers, tlsOptions); | ||
|
|
||
| return new Promise((resolve, reject) => { |
Extract LB host without regex capture groups, reject invalid --cacert paths asynchronously, and remove Terraform interpolation from the external_lb_base_url description so validate succeeds. Co-authored-by: Cursor <cursoragent@cursor.com>
| const requestOptions = { | ||
| hostname: url.host, | ||
| port: 443, | ||
| hostname: url.hostname, |
| let parsedUrl; | ||
| let requestOptions; | ||
| try { | ||
| parsedUrl = typeof url === 'string' ? new URL(url) : url; |
There was a problem hiding this comment.
we may drop this URL parsing here, since buildHttpsRequestOptions parses again
| port: 443, | ||
| hostname: url.hostname, | ||
| port: url.port || 443, | ||
| path: url.pathname + (params !== '' ? `?${params}` : ''), |
There was a problem hiding this comment.
2nd look at this, I believe it can be simplified to path: url.pathname + url.seach
jlorper
left a comment
There was a problem hiding this comment.
a couple of minor suggestions
| proxy_endpoint_is_cloud_function = can(regex("\\.run\\.app", local.proxy_endpoint_url)) || can(regex("\\.cloudfunctions\\.net", local.proxy_endpoint_url)) | ||
| # Prefer split/trimprefix over regex capture groups (capture groups return a list when >1 group) | ||
| proxy_endpoint_host = split("/", trimprefix(trimprefix(local.proxy_endpoint_url, "https://"), "http://"))[0] | ||
| proxy_endpoint_is_ip = can(regex("^[0-9.]+$", local.proxy_endpoint_host)) || can(regex("^\\[[0-9a-fA-F:]+\\]$", local.proxy_endpoint_host)) |
There was a problem hiding this comment.
| proxy_endpoint_is_ip = can(regex("^[0-9.]+$", local.proxy_endpoint_host)) || can(regex("^\\[[0-9a-fA-F:]+\\]$", local.proxy_endpoint_host)) | |
| proxy_endpoint_is_ip = can(cidrhost("${local.proxy_endpoint_host}/32", 0)) || can(cidrhost("${local.proxy_endpoint_host}/128", 0) |
better than regexp
| # --allow-insecure-tls: IP hosts (self-signed PoC ALB) proceed despite untrusted cert | ||
| proxy_endpoint_is_cloud_function = can(regex("\\.run\\.app", local.proxy_endpoint_url)) || can(regex("\\.cloudfunctions\\.net", local.proxy_endpoint_url)) | ||
| # Prefer split/trimprefix over regex capture groups (capture groups return a list when >1 group) | ||
| proxy_endpoint_host = split("/", trimprefix(trimprefix(local.proxy_endpoint_url, "https://"), "http://"))[0] |
There was a problem hiding this comment.
shouldn't we just throw errors if no SSL is in use? or at least log, if we allow it for testing / dev
- Detect ALB IP hosts with cidrhost instead of regex - Require https:// for proxy endpoints (Terraform check + psoxy-test) - Simplify request path to pathname + search; warn on --allow-insecure-tls Co-authored-by: Cursor <cursoragent@cursor.com>
Checking proxy_endpoint_url fails terraform test because the Cloud Function URI is unknown until apply. Co-authored-by: Cursor <cursoragent@cursor.com>
Older Terraform evaluates both sides of ||, so startswith(null) failed module tests. Co-authored-by: Cursor <cursoragent@cursor.com>
* update release refs to rc-v0.6.8 * Review logging message (#1343) * Anthropic rules update (#1344) * Claude update * Fix file format * Recovering missing fields * LocalDate as date * document GCP APIs and DWD scope strings for Google Workspace connectors (#1345) * Document GCP APIs and DWD scope strings for Google Workspace connectors. Customers need explicit lists of APIs to enable and comma-separated OAuth scope URLs for Domain-wide Delegation grants, including a superset for shared service accounts. Co-authored-by: Cursor <cursoragent@cursor.com> * Merge DWD scope CSV into Required OAuth Scopes on connector pages. Avoid a separate Domain-wide Delegation section; keep the paste-ready scope string under the existing OAuth scopes heading. Co-authored-by: Cursor <cursoragent@cursor.com> * Revert docs/README.md Google Workspace table to short scope list. Keep detailed API and DWD scope documentation on the Google Workspace connector pages only. Co-authored-by: Cursor <cursoragent@cursor.com> * Classify Google setup errors and slim troubleshooting docs. Return specific X-Psoxy-Error codes and sanitized bodies for API-not-enabled and OAuth setup failures; document error-to-cause mapping as a skimmable list with dummy log examples. Co-authored-by: Cursor <cursoragent@cursor.com> * Rename SOURCE_DWD_NOT_GRANTED and inject GoogleApiSetupErrorInterpreter. Use SOURCE_AUTHORIZATION_NOT_GRANTED for cross-source admin-consent failures; wire ObjectMapper via Dagger constructor injection. Co-authored-by: Cursor <cursoragent@cursor.com> * Document raw Google error signals for older proxy versions. Show parsed log/response fragments that match GoogleApiSetupErrorInterpreter, with legacy X-Psoxy-Error fallbacks. Co-authored-by: Cursor <cursoragent@cursor.com> * Note that OAuth scope mismatch can surface as 401 Unauthorized. Document indistinguishable 401 for missing DWD vs wrong scopes; broaden SOURCE_AUTHORIZATION_NOT_GRANTED message accordingly. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix OAuth scope hint to reference OAUTH_SCOPES env var. Deployed proxies read scopes from the environment via EnvVarsConfigService, not config.yaml. Co-authored-by: Cursor <cursoragent@cursor.com> * Log OAuth token exchange failures at ERROR severity. 401 and other oauth2.googleapis.com/token failures now use SEVERE so they appear as errors in Cloud Logging. Co-authored-by: Cursor <cursoragent@cursor.com> * Always log connection-setup IOExceptions at SEVERE. Drop Google-specific oauth2.googleapis.com/token check from ApiDataRequestHandler; scope hint only when setup error interpreter matches. Co-authored-by: Cursor <cursoragent@cursor.com> * Remove GoogleApiSetupErrorInterpreter; keep docs-only troubleshooting. Roll back Google-specific error classification code and rewrite troubleshooting around log signals mapped to setup conditions. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> * Zoom rules update (#1346) * Updated test * Updated rules * misc fixes for v0.6.8 (#1342) * Fix String.format placeholders in config cache retry log. MessageFormat-style {0} placeholders were passed to String.format, so retry attempt details were never interpolated. Co-authored-by: Cursor <cursoragent@cursor.com> * Bump minor Java and Node dependency versions. Routine maintenance: update Maven property versions and AWS/GCP-adjacent libraries, and confirm npm audit fix reports no vulnerabilities in tool packages. Co-authored-by: Cursor <cursoragent@cursor.com> * Upgrade Jackson to 2.22.0 via jackson-bom. Jackson 2.22 uses patch-less versioning for jackson-annotations; importing the BOM aligns module versions consistently with the main Worklytics codebase. Co-authored-by: Cursor <cursoragent@cursor.com> * Update Jackson BOM to latest --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Jose Lorenzo <jose@worklytics.co> * update release refs to v0.6.8 (#1348) * GCP support for application load balancer in front of API connector deployments (#1347) * Add beta GCP external ALB composition support via api_connector_external_lb_host. Wire ingress and public endpoint URLs from a customer-owned ALB host signal, document the pattern, and teach psoxy-test to call ALB URLs with self-signed TLS. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review: fix ALB IP CLI flags and cacert Promise errors. Extract LB host without regex capture groups, reject invalid --cacert paths asynchronously, and remove Terraform interpolation from the external_lb_base_url description so validate succeeds. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review: HTTPS enforcement and simpler IP/path handling. - Detect ALB IP hosts with cidrhost instead of regex - Require https:// for proxy endpoints (Terraform check + psoxy-test) - Simplify request path to pathname + search; warn on --allow-insecure-tls Co-authored-by: Cursor <cursoragent@cursor.com> * Fix CI: assert HTTPS on known external_lb_base_url only. Checking proxy_endpoint_url fails terraform test because the Cloud Function URI is unknown until apply. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix HTTPS check for Terraform <1.12 null short-circuit. Older Terraform evaluates both sides of ||, so startswith(null) failed module tests. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> * v0.6.8 fixes (#1349) * bump package-lock * improve docs * improve those docs * refactor ALB stuff * comment out external api alb in example * fix IPv6-CIDR parsing in IP allowlists Apache Commons Net SubnetUtils is IPv4-only, so valid IPv6-CIDR entries like /128 were rejected at startup. Co-authored-by: Cursor <cursoragent@cursor.com> * TODO to hide function url in prod; not possible via tf atm * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Address PR review comments on ALB example wiring Remove the misleading top-level api_connector_external_lb_host variable; document the commented main.tf binding instead. Clarify troubleshooting placeholders to use cloud-function-name and environment_id_prefix. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Keep example-repo scripts LF on publish for WSL shebang compatibility. (#1351) Co-authored-by: Cursor <cursoragent@cursor.com> * fix missing String::format --------- Co-authored-by: aperez-worklytics <75276364+aperez-worklytics@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Jose Lorenzo <jose@worklytics.co> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Features
api_connector_external_lb_hostto wire the connector to the ALB hostChange implications
CHANGELOG.md: new beta option for GCP deployments that route connector traffic through an external ALB; customers using this should setapi_connector_external_lb_hostand expect Terraform changes when enabling it.