Skip to content

Conversation

@ehsandeep
Copy link
Member

@ehsandeep ehsandeep commented Jul 10, 2025

Summary by CodeRabbit

  • New Features

    • Added support for organization-wide GCP asset discovery using the Cloud Asset Inventory API, enabling unified resource discovery across all projects within a GCP organization.
    • Users can now choose between organization-level and project-level GCP discovery modes.
  • Documentation

    • Expanded and clarified GCP integration documentation, including detailed setup guides, configuration examples, IAM role requirements, and troubleshooting tips for both discovery approaches.
    • Added a dedicated guide for GCP Asset API integration.
    • Updated README with new GCP Asset API support information.
  • Chores

    • Updated dependencies to newer versions for improved stability and compatibility.
  • Bug Fixes

    • Improved GKE provider code clarity and fixed comments to accurately reflect GCP GKE functionality.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 10, 2025

Walkthrough

The changes introduce organization-level GCP asset discovery using the Cloud Asset Inventory API, alongside the existing project-level discovery using individual GCP service APIs. Documentation was expanded with detailed setup, configuration, and troubleshooting guides for both approaches. The codebase adds a new OrganizationProvider type for org-wide discovery, updates dependency versions, and refactors GKE provider logic.

Changes

File(s) Change Summary
pkg/providers/gcp/gcp.go Added OrganizationProvider for org-level discovery; refactored provider creation to support org or project mode; unified resource parsing.
pkg/providers/gcp/gke.go Renamed GKE client field; refactored resource fetching into helper method; corrected comments.
PROVIDERS.md Expanded GCP integration docs; described org-level and project-level discovery; listed required IAM roles; updated references.
README.md Added documentation section on GCP Asset API; described both discovery approaches; linked to new detailed guide.
docs/GCP_ASSET_API.md New comprehensive documentation for GCP Asset API integration, setup, configuration, troubleshooting, and security best practices.
go.mod Upgraded multiple dependencies (Google Cloud, OpenTelemetry, Go libraries); added and removed indirect dependencies.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Cloudlist
    participant GCP_AssetAPI
    participant GCP_ServiceAPIs

    User->>Cloudlist: Configure GCP provider
    alt organization_id present
        Cloudlist->>GCP_AssetAPI: Query assets (org-wide)
        GCP_AssetAPI-->>Cloudlist: Return assets
    else
        Cloudlist->>GCP_ServiceAPIs: Query resources (per service)
        GCP_ServiceAPIs-->>Cloudlist: Return resources
    end
    Cloudlist-->>User: Display discovered assets/resources
Loading

Poem

A rabbit hopped through GCP’s gate,
Now finds assets, small and great!
Org-wide or project, it sniffs each track,
With docs and code, there’s nothing it lacks.
Dependencies fresh, providers anew—
Cloudlist’s discovery just grew!
🐇✨

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🔭 Outside diff range comments (1)
pkg/providers/gcp/gcp.go (1)

253-265: Extract duplicated project enumeration code.

The project enumeration logic is duplicated between newIndividualProvider and newOrganizationProvider.

Extract to a shared function:

func enumerateProjects(ctx context.Context, creds option.ClientOption) ([]string, error) {
    projects := []string{}
    manager, err := cloudresourcemanager.NewService(ctx, creds)
    if err != nil {
        return nil, errorutil.NewWithErr(err).Msgf("could not create resource manager")
    }
    
    list := manager.Projects.List()
    err = list.Pages(ctx, func(resp *cloudresourcemanager.ListProjectsResponse) error {
        for _, project := range resp.Projects {
            projects = append(projects, project.ProjectId)
        }
        return nil
    })
    if err != nil {
        return nil, errorutil.NewWithErr(err).Msgf("could not list projects")
    }
    
    return projects, nil
}

Then use it in both functions:

provider.projects, err = enumerateProjects(context.Background(), creds)
if err != nil {
    return nil, err
}

Also applies to: 596-612

🧹 Nitpick comments (1)
docs/GCP_ASSET_API.md (1)

85-98: Consider sanitizing private key examples.

While these are template examples and not real private keys, consider using placeholder text like "private_key": "-----BEGIN PRIVATE KEY-----\n[PRIVATE_KEY_CONTENT]\n-----END PRIVATE KEY-----\n" to avoid triggering security scanners.

-      "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
+      "private_key": "-----BEGIN PRIVATE KEY-----\n[PRIVATE_KEY_CONTENT]\n-----END PRIVATE KEY-----\n",

Also applies to: 214-227

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9ee6544 and 303c411.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (6)
  • PROVIDERS.md (1 hunks)
  • README.md (1 hunks)
  • docs/GCP_ASSET_API.md (1 hunks)
  • go.mod (9 hunks)
  • pkg/providers/gcp/gcp.go (4 hunks)
  • pkg/providers/gcp/gke.go (2 hunks)
🧰 Additional context used
🧠 Learnings (5)
pkg/providers/gcp/gke.go (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
PROVIDERS.md (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
docs/GCP_ASSET_API.md (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
go.mod (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
pkg/providers/gcp/gcp.go (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
🧬 Code Graph Analysis (1)
pkg/providers/gcp/gke.go (1)
pkg/schema/schema.go (2)
  • Resources (39-42)
  • NewResources (45-50)
🪛 Gitleaks (8.26.0)
docs/GCP_ASSET_API.md

90-219: Identified a Private Key, which may compromise cryptographic security and sensitive data encryption.

(private-key)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Analyze (go)
  • GitHub Check: Test Builds (1.22.x, ubuntu-latest)
  • GitHub Check: Test Builds (1.22.x, macOS-latest)
  • GitHub Check: Lint Test
  • GitHub Check: Test Builds (1.22.x, windows-latest)
  • GitHub Check: release-test
🔇 Additional comments (13)
README.md (1)

85-96: LGTM! Clear and well-structured documentation addition.

The new Documentation section effectively introduces the dual GCP discovery approaches and properly directs users to the detailed setup guide. The format is consistent and informative.

PROVIDERS.md (1)

50-125: Excellent restructuring of GCP documentation.

The dual-approach structure is well-organized and clearly differentiates between individual service APIs and organization-level Asset API. The IAM roles, configuration examples, and key differences summary provide comprehensive guidance for users to choose the appropriate discovery method.

pkg/providers/gcp/gke.go (4)

8-8: Good addition of Asset API import.

The import supports the dual discovery mode architecture introduced in this PR.


19-25: Well-structured provider updates.

The addition of assetClient field and renaming of gke field clearly separates the two discovery approaches while maintaining backward compatibility.


32-49: Solid delegation pattern implementation.

The conditional logic properly routes to the appropriate discovery method. The empty getResourcesWithAssetClient implementation is correct since organization-level GKE discovery is handled by the org provider rather than individual service providers.


100-100: Correct field reference update.

The update to use the renamed gke field maintains consistency with the struct changes.

docs/GCP_ASSET_API.md (1)

1-245: Comprehensive and well-structured documentation.

This documentation file excellently covers both GCP discovery approaches with detailed setup instructions, configuration examples, troubleshooting guidance, and security considerations. The organization and flow make it easy for users to understand and implement the Asset API integration.

go.mod (3)

36-37: Verify compatibility of major Google API library updates.

The updates from golang.org/x/oauth2 v0.15.0 to v0.30.0 and google.golang.org/api v0.126.0 to v0.232.0 represent significant version jumps. Please ensure these updates don't introduce breaking changes or compatibility issues with existing GCP provider functionality.

#!/bin/bash
# Description: Search for Google API usage patterns that might be affected by the version updates
# Expected: Find usage of deprecated or changed APIs

# Search for oauth2 usage patterns
echo "=== OAuth2 Usage ==="
rg -A 3 -B 1 "oauth2\." --type go

# Search for google.golang.org/api usage
echo "=== Google API Usage ==="
rg -A 3 -B 1 "google\.golang\.org/api" --type go

# Search for potential breaking changes in GCP providers
echo "=== GCP Provider API Calls ==="
fd -e go -p "pkg/providers/gcp" --exec rg -l "\.Do\(\)" {} \;

180-180: Essential dependency for Asset API functionality.

The addition of cloud.google.com/go/asset v1.21.1 is correctly placed as a direct dependency and is necessary for the organization-level GCP discovery feature.


188-195: Appropriate supporting dependencies.

The new Google Cloud and OpenTelemetry indirect dependencies align with the Asset API integration and provide necessary instrumentation and authentication capabilities.

Also applies to: 234-243

pkg/providers/gcp/gcp.go (3)

7-8: Imports look correct based on learnings.

The asset API imports are appropriate, and based on the retrieved learnings, using ContentType_RESOURCE is the correct approach to access both resource data and IAM policies when available.


71-160: Excellent refactoring of the Resources method.

The modular approach with individual service providers improves maintainability and error isolation. Each service failure is logged as a warning without affecting other services.


162-179: Clean implementation of provider selection logic.

The conditional creation based on organization_id presence is well-structured with helpful debug logging.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
pkg/providers/gcp/gcp.go (2)

471-539: Refactor parseAssetToResource for better maintainability.

This method is still 68 lines long and handles multiple asset types in a single switch statement. The previous review comment about refactoring this method for better maintainability remains valid.

Consider breaking this into smaller, focused methods:

func (p *OrganizationProvider) parseAssetToResource(asset *assetpb.Asset) *schema.Resource {
    if asset == nil || asset.Resource == nil || asset.Resource.Data == nil {
        return nil
    }

    parsers := map[string]func(*assetpb.Asset) *schema.Resource{
        "compute.googleapis.com/Instance":             p.parseComputeInstance,
        "compute.googleapis.com/ForwardingRule":       p.parseForwardingRule,
        "dns.googleapis.com/ResourceRecordSet":        p.parseDNSRecordSet,
        "storage.googleapis.com/Bucket":               p.parseStorageBucket,
        "run.googleapis.com/Service":                  p.parseCloudRunService,
        "cloudfunctions.googleapis.com/CloudFunction": p.parseCloudFunction,
        "container.googleapis.com/Cluster":            p.parseGKECluster,
        "tpu.googleapis.com/Node":                     p.parseTPUNode,
        "file.googleapis.com/Instance":                p.parseFilestoreInstance,
    }

    if parser, ok := parsers[asset.AssetType]; ok {
        return parser(asset)
    }
    return nil
}

543-550: Improve getStringField function safety.

The function doesn't handle nil data parameter, which could cause a panic.

Apply this diff to add nil safety:

 func getStringField(data *structpb.Struct, fieldNames ...string) string {
+    if data == nil {
+        return ""
+    }
     for _, fieldName := range fieldNames {
         if field, ok := data.Fields[fieldName]; ok {
             return field.GetStringValue()
         }
     }
     return ""
 }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 303c411 and 5bc1125.

📒 Files selected for processing (2)
  • pkg/providers/gcp/gcp.go (5 hunks)
  • pkg/providers/gcp/gke.go (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/providers/gcp/gke.go
🧰 Additional context used
🧠 Learnings (1)
pkg/providers/gcp/gcp.go (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
🧬 Code Graph Analysis (1)
pkg/providers/gcp/gcp.go (2)
pkg/schema/schema.go (5)
  • ServiceMap (251-251)
  • Provider (18-28)
  • Resources (39-42)
  • OptionBlock (192-192)
  • Resource (141-160)
pkg/inventory/inventory.go (1)
  • New (36-51)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Test Builds (1.22.x, macOS-latest)
  • GitHub Check: Test Builds (1.22.x, ubuntu-latest)
  • GitHub Check: Test Builds (1.22.x, windows-latest)
  • GitHub Check: Lint Test
  • GitHub Check: release-test
  • GitHub Check: Analyze (go)
🔇 Additional comments (12)
pkg/providers/gcp/gcp.go (12)

8-9: LGTM! New imports properly added for Cloud Asset API integration.

The new imports for the Cloud Asset API (asset, assetpb), iterator package, and protobuf types are correctly added to support the organization-level asset discovery functionality.

Also applies to: 18-18, 21-21


37-44: LGTM! Well-structured OrganizationProvider.

The OrganizationProvider struct is well-designed with clear fields for organization-level asset discovery. The separation of concerns between individual service providers and organization-level providers is appropriate.


47-54: LGTM! Helpful service documentation added.

The comments clearly explain which services provide IP addresses or DNS names, making the purpose and scope of each service transparent.


74-163: LGTM! Proper error handling and resource aggregation.

The refactored Resources method correctly aggregates resources from individual service providers with proper error handling. The use of gologger.Warning() for non-fatal errors allows the method to continue processing other services even if one fails.


165-182: LGTM! Clean provider factory pattern.

The refactored New function properly implements a factory pattern that returns the appropriate provider type based on configuration. The logging helps with debugging and understanding which provider type is being created.


184-270: LGTM! Proper separation of individual provider logic.

The newIndividualProvider function correctly encapsulates the original provider creation logic with proper error handling and service initialization.


272-318: LGTM! Proper interface implementation.

The OrganizationProvider correctly implements the schema.Provider interface with appropriate logging and error handling in the Resources method.


364-366: LGTM! Proper error handling implemented.

The iterator error handling now correctly uses errors.Is(err, iterator.Done) instead of string comparison, addressing the previous review comment. This is more robust and won't break with SDK updates.


561-592: LGTM! Proper nil checking in extractComputeInstanceIPs.

The function correctly handles nil checks at each level of nested access, preventing potential panics when accessing the protobuf structure.


594-616: LGTM! Safe DNS record data extraction.

The function properly handles nil checks and correctly extracts DNS record data based on record type. The logic for handling A and AAAA records is appropriate.


618-632: LGTM! Safe TPU endpoint extraction.

The function correctly handles nil checks when extracting TPU network endpoint information from the nested protobuf structure.


634-659: LGTM! Safe Filestore IP extraction.

The function properly handles nil checks at each level when extracting IP addresses from Filestore network configuration.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
pkg/providers/gcp/gcp.go (2)

471-539: Consider refactoring the large parseAssetToResource method.

This 68-line method with a large switch statement remains complex and could benefit from the refactoring suggested in previous reviews. The method handles multiple responsibilities and lacks comprehensive nil checks.

Consider breaking this into smaller, focused parsing functions as previously suggested:

func (p *OrganizationProvider) parseAssetToResource(asset *assetpb.Asset) *schema.Resource {
    if asset == nil || asset.Resource == nil || asset.Resource.Data == nil {
        return nil
    }

    parsers := map[string]func(*assetpb.Asset) *schema.Resource{
        "compute.googleapis.com/Instance":             p.parseComputeInstance,
        "compute.googleapis.com/ForwardingRule":       p.parseForwardingRule,
        "dns.googleapis.com/ResourceRecordSet":        p.parseDNSRecordSet,
        // ... other parsers
    }

    if parser, ok := parsers[asset.AssetType]; ok {
        return parser(asset)
    }
    return nil
}

543-550: Add nil safety check to getStringField function.

The function doesn't handle a nil data parameter, which could cause a panic.

Apply this diff to add nil safety:

 func getStringField(data *structpb.Struct, fieldNames ...string) string {
+    if data == nil {
+        return ""
+    }
     for _, fieldName := range fieldNames {
         if field, ok := data.Fields[fieldName]; ok {
             return field.GetStringValue()
         }
     }
     return ""
 }
🧹 Nitpick comments (2)
docs/GCP_ASSET_API.md (2)

250-250: Fix markdown formatting for error heading.

The markdown linter correctly identifies that emphasis is being used instead of a proper heading.

Apply this fix:

-**Error: "The caller does not have permission"**
+### Error: "The caller does not have permission"

260-260: Fix markdown formatting for error heading.

Similar formatting issue with emphasis instead of heading.

Apply this fix:

-**Error: "No RESOURCE found that matches asset type"**
+### Error: "No RESOURCE found that matches asset type"
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5bc1125 and 5ed6147.

📒 Files selected for processing (2)
  • docs/GCP_ASSET_API.md (1 hunks)
  • pkg/providers/gcp/gcp.go (5 hunks)
🧰 Additional context used
🧠 Learnings (2)
docs/GCP_ASSET_API.md (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
pkg/providers/gcp/gcp.go (2)
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: In Google Cloud Go SDK's assetpb package, ContentType_RESOURCE includes IAM policy data when available. There is no ContentType_RESOURCE_AND_IAM_POLICY constant in the assetpb package. The correct approach is to use ContentType_RESOURCE and access asset.IamPolicy directly, as shown in the GCP provider implementations.
Learnt from: dogancanbakir
PR: projectdiscovery/cloudlist#687
File: pkg/providers/gcp/bucket.go:30-30
Timestamp: 2025-07-09T17:50:43.982Z
Learning: The Google Cloud Go SDK assetpb package does not have a ContentType_RESOURCE_AND_IAM_POLICY constant. The available constants are ContentType_RESOURCE and ContentType_IAM_POLICY as separate values. Using ContentType_RESOURCE appears to include IAM policy data in the response, allowing access to asset.IamPolicy.
🪛 Gitleaks (8.26.0)
docs/GCP_ASSET_API.md

90-219: Identified a Private Key, which may compromise cryptographic security and sensitive data encryption.

(private-key)


323-341: Identified a Private Key, which may compromise cryptographic security and sensitive data encryption.

(private-key)

🪛 markdownlint-cli2 (0.17.2)
docs/GCP_ASSET_API.md

250-250: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)


260-260: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Test Builds (1.22.x, windows-latest)
  • GitHub Check: Test Builds (1.22.x, macOS-latest)
  • GitHub Check: Test Builds (1.22.x, ubuntu-latest)
  • GitHub Check: Analyze (go)
  • GitHub Check: Lint Test
🔇 Additional comments (7)
pkg/providers/gcp/gcp.go (6)

364-366: Good fix for iterator handling.

The use of errors.Is(err, iterator.Done) properly addresses the previous review comment about fragile string comparison error handling.


552-562: Nil safety check properly implemented.

Good addition of the nil check for the data parameter that addresses the previous review feedback.


75-163: Well-structured service provider orchestration.

The refactored Resources method properly separates concerns by delegating to individual service providers. The error handling with warnings allows the discovery to continue even if individual services fail, which is the correct approach for this use case.


165-182: Smart conditional provider creation maintains backwards compatibility.

The approach of checking for organization_id to determine which provider type to create maintains backwards compatibility while enabling the new organization-level discovery feature.


354-354: Verify ContentType usage with retrieved learnings.

Based on the retrieved learnings, ContentType_RESOURCE is correct and includes IAM policy data when available. However, ensure this aligns with the asset parsing logic expectations.

The usage of assetpb.ContentType_RESOURCE is correct based on the retrieved learnings about the GCP SDK.


288-318: Comprehensive organization-level resource discovery implementation.

The Resources method properly implements organization-wide asset discovery with appropriate service filtering and error handling. The logging provides good visibility into the discovery process.

docs/GCP_ASSET_API.md (1)

1-394: Excellent comprehensive documentation.

This documentation provides thorough coverage of both GCP discovery approaches with clear setup instructions, configuration examples, and troubleshooting guidance. The comparison table effectively helps users choose the right approach, and the multi-organization support section adds significant value.

@ehsandeep ehsandeep linked an issue Jul 10, 2025 that may be closed by this pull request
@ehsandeep ehsandeep merged commit 280296a into dev Jul 10, 2025
9 checks passed
@ehsandeep ehsandeep deleted the gcp_org_level_clean_v2 branch July 10, 2025 23:28
visnetodev pushed a commit to visnetotest/cloudlist that referenced this pull request Dec 7, 2025
* Added GCP org asset API

* added docs

* feat: gcp provider misc fixes

* Added TPU / Filestore service

* feat: simplify code + misc

* multi cloud docs update + panic check

---------

Co-authored-by: Ice3man <nizamulrana@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multiple account (org level) integration support for GCP

3 participants