fix(cloud-tests): show folder-nested GCP projects in connection picker by tofikwest · Pull Request #2899 · trycompai/comp

tofikwest · 2026-05-21T21:08:55Z

Summary

Fixes a customer-reported (Greg @ Propper) bug where GCP projects nested inside a folder (org → folder → project) never appeared in the connection picker, even though the user had full IAM access.

Root cause — two bugs combined:

detectProjectsForOrg filters parent.id:<orgId> which GCP's v1 list endpoint interprets as immediate children only — a project whose immediate parent is a folder under the org never matches.
Both project-detection paths called v1/projects with pageSize=50 and never followed nextPageToken. Accounts with many sandboxes / Gemini default projects / etc. hit the page cap before reaching production projects on later pages, silently dropping them.

Greg's exact reproduction: org 43356919874 (propper.ai), folder 9724350536 (propper), projects propperai-prod + propperai-demo under that folder — invisible in picker, despite gcloud projects list --filter="parent.id=9724350536" returning both.

What changed

New listProjectsPaginated(token, filter) helper:
- Follows nextPageToken until exhaustion (pageSize=200, well under GCP's 500 max).
- Stops at a 1000-project safety cap.
- On a mid-pagination error returns the results collected so far — matches the prior "best-effort" picker posture so a single transient 5xx doesn't blank the dropdown.
detectProjectsForOrg now issues two parallel paginated calls and merges + dedupes:
1. Direct org children (parent.id:<orgId>) — preserves existing behavior.
2. Folder-nested projects (parent.type:folder) — the new arm that surfaces Greg's projects. GCP's v1 list API has no "descendants-of-org" query, so the user's IAM scope is the effective filter here.
detectProjects (no-org-known path) refactored to use the same paginator. Direct list paginated; org-scoped fallback paginated. Otherwise unchanged.

Backward compatibility

Customers whose projects all live directly under an org → no behavior change (the new folder-nested arm returns 0 for them).
Existing connections are not touched — project detection only runs at connect/re-auth flows.
No new GCP permissions required — same resourcemanager.projects.list the picker has always used.
No schema or DB changes.

Tests

9 new unit tests covering:

Single-page no-pagination case (current happy path preserved).
Multi-page pagination follows nextPageToken end-to-end.
pageToken is correctly propagated to subsequent requests.
Mid-pagination 5xx returns collected results (best-effort posture).
Safety cap (1000) stops paginating even if more pages remain.
detectProjectsForOrg returns the union of direct + folder-nested.
detectProjectsForOrg dedupes overlapping results.
The two arms of detectProjectsForOrg run in parallel (Promise.all).
Empty result on both arms returns [].

cd apps/api && npx jest src/cloud-security → 242/242 pass (one pre-existing suite failure on remediation.controller.spec.ts is the known Postgres-TLS env issue, identical on main).

Manual test plan

Reconnect GCP on Greg's tenant; confirm propperai-prod and propperai-demo appear in picker
Reconnect on a tenant with all projects directly under org; confirm picker shows the same set as before
Reconnect on an account with >50 accessible projects; confirm all appear (not just first 50)
No regression on existing connected projects — they should sync as before

Independent of in-flight PR

This is on its own branch off main (tofik/fix-gcp-project-picker-pagination) — unrelated to PR #2885 (auto-remediate fixes).

🤖 Generated with Claude Code

Summary by cubic

Show folder‑nested GCP projects in the connection picker, paginate project listing for complete results, scope folder queries to the selected org, and cap per-folder queries to avoid throttling and silent drops.

Bug Fixes
- Added a paginated listProjectsPaginated(token, filter) that follows nextPageToken, uses pageSize=200, caps at 1000, and returns partial results on errors.
- Updated detectProjectsForOrg to run two paginated queries in parallel and merge/dedupe: direct org children, plus folder‑nested projects scoped to that org by recursively enumerating folders via v2/folders?parent=organizations/<orgId> and querying each with parent.type:folder AND parent.id:<folderId>. Uses Promise.allSettled so either arm failing doesn’t blank the picker.
- Capped concurrent folder→project queries to 5 to avoid GCP throttling (429) that could look like empty folders.
- Isolated per‑folder failures and added a 500‑folder safety cap for traversal.
- Refactored detectProjects to use the same paginator for direct and org‑scoped fallback paths. Existing permissions and behavior for orgs without folder nesting remain unchanged.

^{Written for commit d333f59. Summary will update on new commits. Review in cubic}

Customers with an org→folder→project hierarchy (a common SOC2-friendly layout) reported that their production projects never appeared in our GCP connection picker. Two bugs combined to cause it: 1. detectProjectsForOrg filtered with `parent.id:<orgId>`, which GCP's v1/projects API interprets as "immediate org children only". A project whose immediate parent is a folder under the org never matches, even if the user has full IAM access to it. 2. Both detectProjects and detectProjectsForOrg called the v1/projects list endpoint with pageSize=50 and never followed nextPageToken. For accounts with many sandboxes / Gemini default projects / etc., the first 50 results consumed the whole page and the production projects on later pages were silently dropped. Fix: - New `listProjectsPaginated(token, filter)` helper that follows nextPageToken until exhaustion, uses pageSize=200, and stops at a 1000-project safety cap. On a mid-pagination error it returns what it has so far — matches the prior "best-effort" picker posture and prevents a single transient 5xx from blanking the dropdown. - detectProjectsForOrg now issues two parallel paginated calls: one for direct org children (existing behavior) and one for any folder-nested project the caller has access to (`parent.type:folder`). Results merged + deduped. GCP's v1 list API has no "descendants-of-org" query, so the user's IAM scope is the effective filter for the folder-nested arm. - detectProjects refactored to use the same paginator. Direct list is paginated; org-scoped fallback is paginated. Backward compatibility: customers whose projects all live directly under an org see no behavior change (the folder-nested call returns 0 for them). Existing connections aren't touched — detection only runs at connect/re-auth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

vercel · 2026-05-21T21:09:01Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
comp-framework-editor	Ready	Preview, Comment	May 21, 2026 9:37pm

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
app	Skipped		May 21, 2026 9:37pm
portal	Skipped		May 21, 2026 9:37pm

cubic-dev-ai

1 issue found across 2 files

Confidence score: 3/5

There is a concrete regression risk in apps/api/src/cloud-security/providers/gcp-security.service.ts: detectProjectsForOrg appears to broaden scope and return folder-nested projects from all accessible organizations instead of only the requested organizationId.
Because this is a medium-severity, high-confidence behavior change (6/10, 8/10), it could lead to user-visible cross-organization results and incorrect project targeting, so some validation is needed before merging.
Pay close attention to apps/api/src/cloud-security/providers/gcp-security.service.ts - ensure organization filtering is strictly enforced to the selected organizationId when traversing folder-nested projects.

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…picker Switches `detectProjectsForOrg` from `Promise.all` → `Promise.allSettled` so a transient failure on the new folder-nested arm (e.g., GCP rejects the `parent.type:folder`-alone filter, DNS blip, transient 5xx during pagination) cannot blank the entire picker. The direct arm matches the prior production code path, so as long as it succeeds we are guaranteed to be no worse than today's prod even if the folder arm fails outright. Two new tests lock in the isolation: - direct arm succeeds, folder arm throws → returns direct results - folder arm succeeds, direct arm throws → returns folder results Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

cubic-dev-ai

1 issue found across 2 files (changes from recent commits).

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…rget org Addresses cubic P2 on PR #2899: the previous folder arm filter (`parent.type:folder` alone) would have returned projects from ANY org the caller had access to, violating the `detectProjectsForOrg` contract for multi-org users. The folder arm now recursively enumerates folders under the target org via `v2/folders?parent=organizations/<orgId>` (and recursively under each child folder), then queries each discovered folder with `parent.type:folder AND parent.id:<folderId>` — the paired filter shape GCP explicitly documents for by-parent project queries and which triggers their alternate consistent index. Behaviors preserved: - Promise.allSettled isolation so a failure on either arm cannot blank the picker. - Per-folder query failures are isolated (one bad folder doesn't kill the rest). - Safety cap of 500 folders to bound API usage. New tests: - Greg's exact scenario: only this org's folder gets queried. - Recursive sub-folder traversal (org → folder → sub-folder → projects). - Dedupe across arms. - Folder arm failure → direct arm still works. - Direct arm failure → folder arm still works. - Per-folder failure isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

cubic-dev-ai

1 issue found across 2 files (changes from recent commits).

_{Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic}

Addresses cubic P2 on PR #2899: the previous `Promise.all` over every folder ID fires N simultaneous requests to cloudresourcemanager. For tenants with many folders, that can trip GCP's per-user read-quota, and because `listProjectsPaginated` returns the projects-collected-so- far on a non-OK response (including 429 Too Many Requests), a throttled folder query LOOKS like an empty folder — silently truncating the picker with no visible error. The fix bounds concurrent in-flight folder queries to 5 via a small inlined `mapWithConcurrency` helper. GCP's read quota (~600 req/min/user on cloudresourcemanager) is well above this, so the cap stays comfortably below throttling thresholds even for very deep folder hierarchies. Test added that builds a 20-folder tenant, holds the per-folder project queries open, and verifies: - peak in-flight count never exceeds 5 - all 20 folder queries eventually run (cap doesn't truncate work) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

tofikwest · 2026-05-21T21:38:27Z

@cubic-dev-ai review it

cubic-dev-ai · 2026-05-21T21:38:34Z

@cubic-dev-ai review it

@tofikwest I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 2 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

claudfuen · 2026-05-21T21:47:25Z

🎉 This PR is included in version 3.61.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

vercel Bot deployed to Preview – comp-framework-editor May 21, 2026 21:09 View deployment

cubic-dev-ai Bot reviewed May 21, 2026

View reviewed changes

Comment thread apps/api/src/cloud-security/providers/gcp-security.service.ts Outdated

vercel Bot temporarily deployed to Preview – portal May 21, 2026 21:22 Inactive

vercel Bot temporarily deployed to Preview – app May 21, 2026 21:22 Inactive

vercel Bot deployed to Preview – comp-framework-editor May 21, 2026 21:23 View deployment

cubic-dev-ai Bot reviewed May 21, 2026

View reviewed changes

Comment thread apps/api/src/cloud-security/providers/gcp-security.service.ts

vercel Bot temporarily deployed to Preview – app May 21, 2026 21:28 Inactive

vercel Bot temporarily deployed to Preview – portal May 21, 2026 21:28 Inactive

vercel Bot deployed to Preview – comp-framework-editor May 21, 2026 21:29 View deployment

cubic-dev-ai Bot reviewed May 21, 2026

View reviewed changes

Comment thread apps/api/src/cloud-security/providers/gcp-security.service.ts Outdated

vercel Bot temporarily deployed to Preview – portal May 21, 2026 21:36 Inactive

vercel Bot temporarily deployed to Preview – app May 21, 2026 21:36 Inactive

vercel Bot deployed to Preview – comp-framework-editor May 21, 2026 21:37 View deployment

cubic-dev-ai Bot reviewed May 21, 2026

View reviewed changes

tofikwest merged commit dd253d0 into main May 21, 2026
11 checks passed

tofikwest deleted the tofik/fix-gcp-project-picker-pagination branch May 21, 2026 21:41

claudfuen added the released label May 21, 2026

tofikwest mentioned this pull request May 22, 2026

fix(cloud-tests): unblock GCP picker when folder enumeration returns 403 #2916

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(cloud-tests): show folder-nested GCP projects in connection picker#2899

fix(cloud-tests): show folder-nested GCP projects in connection picker#2899
tofikwest merged 4 commits into
mainfrom
tofik/fix-gcp-project-picker-pagination

tofikwest commented May 21, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

vercel Bot commented May 21, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

tofikwest commented May 21, 2026

Uh oh!

cubic-dev-ai Bot commented May 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Uh oh!

claudfuen commented May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tofikwest commented May 21, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Backward compatibility

Tests

Manual test plan

Independent of in-flight PR

Summary by cubic

Uh oh!

vercel Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tofikwest commented May 21, 2026

Uh oh!

cubic-dev-ai Bot commented May 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

claudfuen commented May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tofikwest commented May 21, 2026 •

edited by cubic-dev-ai Bot

Loading

vercel Bot commented May 21, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading