Skip to content

GCP support for application load balancer in front of API connector deployments - #1347

Merged
eschultink merged 5 commits into
rc-v0.6.8from
s230-gcp-behind-alb
Jul 21, 2026
Merged

GCP support for application load balancer in front of API connector deployments#1347
eschultink merged 5 commits into
rc-v0.6.8from
s230-gcp-behind-alb

Conversation

@eschultink

Copy link
Copy Markdown
Member

Features

  • adds beta support for composing GCP API connectors behind an external Application Load Balancer, using api_connector_external_lb_host to wire the connector to the ALB host
  • test tool support for arbitrary IP/domain other than those obv aws/gcp-hosted gateway/lamba/cf

Change implications

  • dependencies added/changed? no
  • something important to note in future release notes?
    • NOTE in CHANGELOG.md: new beta option for GCP deployments that route connector traffic through an external ALB; customers using this should set api_connector_external_lb_host and expect Terraform changes when enabling it.
    • breaking changes? no (beta feature; not a breaking change to stable/GA modules)

…nal_lb_host.

Wire ingress and public endpoint URLs from a customer-owned ALB host signal, document the pattern, and teach psoxy-test to call ALB URLs with self-signed TLS.

Co-authored-by: Cursor <cursoragent@cursor.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

Claude Code Review is paused for this repository. To reconnect it, an admin of this repository's GitHub organization (or the account owner, for personal repositories) who can also manage your Claude organization's Code Review settings needs to re-link GitHub in Code Review settings. This is a one-time step.

Tip: disable this comment in your organization's Code Review settings.

@eschultink eschultink changed the title S230 : gcp behind alb GCP support for application load balancer in front of API connector deployments Jul 20, 2026
Comment thread tools/psoxy-test/lib/utils.js Dismissed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds beta support for running GCP API connector deployments behind a customer-composed external Application Load Balancer (ALB), and updates the psoxy-test tooling/docs to work with non-*.run.app / non-*.cloudfunctions.net hosts (including IP-based PoC endpoints with self-signed TLS).

Changes:

  • Introduces api_connector_external_lb_host in gcp-host to publish ALB-based connector endpoints and set Cloud Functions ingress to ALLOW_INTERNAL_AND_GCLB when enabled.
  • Extends psoxy-test to support PoC/self-signed TLS via --allow-insecure-tls and --cacert, including request option building and new unit tests.
  • Adds and links documentation + an example (commented) Terraform composition for external ALB + Cloud Armor.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tools/psoxy-test/test/utils.test.js Adds unit tests for buildHttpsRequestOptions (IP host + insecure TLS; custom domain default TLS).
tools/psoxy-test/test/gcp.test.js Updates GCP tests for new logs URL behavior on non-Cloud-Function hosts and updated request signature.
tools/psoxy-test/test/aws.test.js Updates AWS tests for updated request signature (extra TLS options argument).
tools/psoxy-test/lib/utils.js Adds buildHttpsRequestOptions and threads TLS options into the request wrapper.
tools/psoxy-test/lib/gcp.js Passes TLS options through to request layer; clarifies getLogsURL behavior for external ALB/custom domains.
tools/psoxy-test/lib/aws.js Passes TLS options through to request layer.
tools/psoxy-test/cli-call.js Adds CLI flags --allow-insecure-tls and --cacert.
infra/modules/gcp-proxy-api/variables.tf Adds ingress_settings and external_lb_base_url module inputs for ALB composition support.
infra/modules/gcp-proxy-api/main.tf Switches Pub/Sub push to Cloud Function URI; parameterizes ingress; computes public proxy endpoint URL and test CLI flags.
infra/modules/gcp-host/variables.tf Adds api_connector_external_lb_host input with validation and documentation.
infra/modules/gcp-host/main.tf Enforces allowlist presence when ALB host set; wires ALB base URL + ingress settings into connectors; publishes external endpoint URLs.
infra/examples-dev/gcp/variables.tf Adds example variable for api_connector_external_lb_host; updates allowlist description for Cloud Armor parity.
infra/examples-dev/gcp/networking.tf New commented reference composition for external ALB + Cloud Armor (beta).
infra/examples-dev/gcp/main.tf Wires api_connector_external_lb_host into the example root module; extends connector outputs.
docs/SUMMARY.md Adds navigation entry for the new GCP external ALB doc.
docs/guides/psoxy-test-tool.md Documents testing external ALB endpoints, including -f gcp, --allow-insecure-tls, and --cacert.
docs/gcp/vpc.md Clarifies that the VPC doc covers egress only; links to external ALB ingress doc.
docs/gcp/README.md Adds link to VPC (egress) and related external ALB doc.
docs/gcp/authentication-authorization.md Links to external ALB + Cloud Armor doc alongside PSC/connectivity notes.
docs/development/README.md Adds entry for external ALB + Cloud Armor documentation.
docs/development/gcp-private-service-connect.md Adds external ALB + Cloud Armor option to the connectivity matrix and clarifies ILB vs external ALB.
docs/development/gcp-external-alb.md New beta design doc describing the external ALB + Cloud Armor composition pattern and module wiring.
docs/configuration/ip-allowlisting.md Updates GCP guidance to reference the external ALB + Cloud Armor composition path and its relationship to allowed_data_access_ip_blocks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +393 to +396
proxy_endpoint_is_cloud_function = can(regex("\\.run\\.app", local.proxy_endpoint_url)) || can(regex("\\.cloudfunctions\\.net", local.proxy_endpoint_url))
proxy_endpoint_host = try(regex("^https?://([^/]+)", local.proxy_endpoint_url), "")
proxy_endpoint_is_ip = can(regex("^[0-9.]+$", local.proxy_endpoint_host)) || can(regex("^\\[[0-9a-fA-F:]+\\]$", local.proxy_endpoint_host))
command_cli_call_flags = trimspace(join(" ", compact([

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds plausible.

Comment on lines +154 to 159
function requestWrapper(url, method = 'GET', headers, body = {}, tlsOptions = {}) {
url = typeof url === 'string' ? new URL(url) : url;
const responseBody = [];
const requestOptions = buildHttpsRequestOptions(url, method, headers, tlsOptions);

return new Promise((resolve, reject) => {
Extract LB host without regex capture groups, reject invalid --cacert paths asynchronously, and remove Terraform interpolation from the external_lb_base_url description so validate succeeds.

Co-authored-by: Cursor <cursoragent@cursor.com>
const requestOptions = {
hostname: url.host,
port: 443,
hostname: url.hostname,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

let parsedUrl;
let requestOptions;
try {
parsedUrl = typeof url === 'string' ? new URL(url) : url;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may drop this URL parsing here, since buildHttpsRequestOptions parses again

Comment thread tools/psoxy-test/lib/utils.js Outdated
port: 443,
hostname: url.hostname,
port: url.port || 443,
path: url.pathname + (params !== '' ? `?${params}` : ''),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2nd look at this, I believe it can be simplified to path: url.pathname + url.seach

@jlorper jlorper left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a couple of minor suggestions

Comment thread infra/modules/gcp-proxy-api/main.tf Outdated
proxy_endpoint_is_cloud_function = can(regex("\\.run\\.app", local.proxy_endpoint_url)) || can(regex("\\.cloudfunctions\\.net", local.proxy_endpoint_url))
# Prefer split/trimprefix over regex capture groups (capture groups return a list when >1 group)
proxy_endpoint_host = split("/", trimprefix(trimprefix(local.proxy_endpoint_url, "https://"), "http://"))[0]
proxy_endpoint_is_ip = can(regex("^[0-9.]+$", local.proxy_endpoint_host)) || can(regex("^\\[[0-9a-fA-F:]+\\]$", local.proxy_endpoint_host))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
proxy_endpoint_is_ip = can(regex("^[0-9.]+$", local.proxy_endpoint_host)) || can(regex("^\\[[0-9a-fA-F:]+\\]$", local.proxy_endpoint_host))
proxy_endpoint_is_ip = can(cidrhost("${local.proxy_endpoint_host}/32", 0)) || can(cidrhost("${local.proxy_endpoint_host}/128", 0)

better than regexp

# --allow-insecure-tls: IP hosts (self-signed PoC ALB) proceed despite untrusted cert
proxy_endpoint_is_cloud_function = can(regex("\\.run\\.app", local.proxy_endpoint_url)) || can(regex("\\.cloudfunctions\\.net", local.proxy_endpoint_url))
# Prefer split/trimprefix over regex capture groups (capture groups return a list when >1 group)
proxy_endpoint_host = split("/", trimprefix(trimprefix(local.proxy_endpoint_url, "https://"), "http://"))[0]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we just throw errors if no SSL is in use? or at least log, if we allow it for testing / dev

eschultink and others added 3 commits July 21, 2026 08:59
- Detect ALB IP hosts with cidrhost instead of regex
- Require https:// for proxy endpoints (Terraform check + psoxy-test)
- Simplify request path to pathname + search; warn on --allow-insecure-tls

Co-authored-by: Cursor <cursoragent@cursor.com>
Checking proxy_endpoint_url fails terraform test because the Cloud Function URI is unknown until apply.

Co-authored-by: Cursor <cursoragent@cursor.com>
Older Terraform evaluates both sides of ||, so startswith(null) failed module tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@eschultink
eschultink merged commit f68f5ba into rc-v0.6.8 Jul 21, 2026
120 checks passed
@eschultink
eschultink deleted the s230-gcp-behind-alb branch July 21, 2026 18:32
eschultink added a commit that referenced this pull request Jul 24, 2026
* update release refs to rc-v0.6.8

* Review logging message (#1343)

* Anthropic rules update (#1344)

* Claude update

* Fix file format

* Recovering missing fields

* LocalDate as date

* document GCP APIs and DWD scope strings for Google Workspace connectors (#1345)

* Document GCP APIs and DWD scope strings for Google Workspace connectors.

Customers need explicit lists of APIs to enable and comma-separated OAuth scope URLs for Domain-wide Delegation grants, including a superset for shared service accounts.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Merge DWD scope CSV into Required OAuth Scopes on connector pages.

Avoid a separate Domain-wide Delegation section; keep the paste-ready scope string under the existing OAuth scopes heading.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Revert docs/README.md Google Workspace table to short scope list.

Keep detailed API and DWD scope documentation on the Google Workspace connector pages only.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Classify Google setup errors and slim troubleshooting docs.

Return specific X-Psoxy-Error codes and sanitized bodies for API-not-enabled and OAuth setup failures; document error-to-cause mapping as a skimmable list with dummy log examples.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Rename SOURCE_DWD_NOT_GRANTED and inject GoogleApiSetupErrorInterpreter.

Use SOURCE_AUTHORIZATION_NOT_GRANTED for cross-source admin-consent failures; wire ObjectMapper via Dagger constructor injection.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Document raw Google error signals for older proxy versions.

Show parsed log/response fragments that match GoogleApiSetupErrorInterpreter, with legacy X-Psoxy-Error fallbacks.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Note that OAuth scope mismatch can surface as 401 Unauthorized.

Document indistinguishable 401 for missing DWD vs wrong scopes; broaden SOURCE_AUTHORIZATION_NOT_GRANTED message accordingly.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix OAuth scope hint to reference OAUTH_SCOPES env var.

Deployed proxies read scopes from the environment via EnvVarsConfigService, not config.yaml.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Log OAuth token exchange failures at ERROR severity.

401 and other oauth2.googleapis.com/token failures now use SEVERE so they appear as errors in Cloud Logging.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Always log connection-setup IOExceptions at SEVERE.

Drop Google-specific oauth2.googleapis.com/token check from ApiDataRequestHandler; scope hint only when setup error interpreter matches.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Remove GoogleApiSetupErrorInterpreter; keep docs-only troubleshooting.

Roll back Google-specific error classification code and rewrite troubleshooting around log signals mapped to setup conditions.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

* Zoom rules update (#1346)

* Updated test

* Updated rules

* misc fixes for v0.6.8 (#1342)

* Fix String.format placeholders in config cache retry log.

MessageFormat-style {0} placeholders were passed to String.format, so retry attempt details were never interpolated.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Bump minor Java and Node dependency versions.

Routine maintenance: update Maven property versions and AWS/GCP-adjacent libraries, and confirm npm audit fix reports no vulnerabilities in tool packages.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Upgrade Jackson to 2.22.0 via jackson-bom.

Jackson 2.22 uses patch-less versioning for jackson-annotations; importing the BOM aligns module versions consistently with the main Worklytics codebase.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Update Jackson BOM to latest

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jose Lorenzo <jose@worklytics.co>

* update release refs to v0.6.8 (#1348)

* GCP support for application load balancer in front of API connector deployments (#1347)

* Add beta GCP external ALB composition support via api_connector_external_lb_host.

Wire ingress and public endpoint URLs from a customer-owned ALB host signal, document the pattern, and teach psoxy-test to call ALB URLs with self-signed TLS.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review: fix ALB IP CLI flags and cacert Promise errors.

Extract LB host without regex capture groups, reject invalid --cacert paths asynchronously, and remove Terraform interpolation from the external_lb_base_url description so validate succeeds.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review: HTTPS enforcement and simpler IP/path handling.

- Detect ALB IP hosts with cidrhost instead of regex
- Require https:// for proxy endpoints (Terraform check + psoxy-test)
- Simplify request path to pathname + search; warn on --allow-insecure-tls

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix CI: assert HTTPS on known external_lb_base_url only.

Checking proxy_endpoint_url fails terraform test because the Cloud Function URI is unknown until apply.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix HTTPS check for Terraform <1.12 null short-circuit.

Older Terraform evaluates both sides of ||, so startswith(null) failed module tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

* v0.6.8 fixes (#1349)

* bump package-lock

* improve docs

* improve those docs

* refactor ALB stuff

* comment out external api alb in example

* fix IPv6-CIDR parsing in IP allowlists

Apache Commons Net SubnetUtils is IPv4-only, so valid IPv6-CIDR entries like /128 were rejected at startup.

Co-authored-by: Cursor <cursoragent@cursor.com>

* TODO to hide function url in prod; not possible via tf atm

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Address PR review comments on ALB example wiring

Remove the misleading top-level api_connector_external_lb_host variable; document the commented main.tf binding instead. Clarify troubleshooting placeholders to use cloud-function-name and environment_id_prefix.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Keep example-repo scripts LF on publish for WSL shebang compatibility. (#1351)

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix missing String::format

---------

Co-authored-by: aperez-worklytics <75276364+aperez-worklytics@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jose Lorenzo <jose@worklytics.co>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants