Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tables for better use of API filters, context cancellation in list calls and page limiting for limit clause in query. Closes #336 #337

Merged
merged 8 commits into from
Jan 18, 2022

Conversation

c0d3r-arnab
Copy link
Contributor

@c0d3r-arnab c0d3r-arnab commented Dec 9, 2021

Tables

  • gcp_bigquery_dataset
  • gcp_bigquery_job
  • gcp_bigquery_table
  • gcp_cloudfunctions_function
  • gcp_compute_address
  • gcp_compute_backend_bucket
  • gcp_compute_backend_service
  • gcp_compute_disk
  • gcp_compute_firewall
  • gcp_compute_forwarding_rule
  • gcp_compute_global_address
  • gcp_compute_global_forwarding_rule
  • gcp_compute_instance_template
  • gcp_compute_machine_type
  • gcp_compute_network
  • gcp_compute_node_group
  • gcp_compute_node_template
  • gcp_compute_region
  • gcp_compute_resource_policy
  • gcp_compute_router
  • gcp_compute_snapshot
  • gcp_compute_ssl_policy
  • gcp_compute_subnetwork
  • gcp_compute_target_https_proxy
  • gcp_compute_target_pool
  • gcp_compute_target_ssl_proxy
  • gcp_compute_target_vpn_gateway
  • gcp_compute_url_map
  • gcp_compute_vpn_tunnel
  • gcp_compute_zone
  • gcp_dns_managed_zone
  • gcp_dns_policy
  • gcp_dns_record_set
  • gcp_kms_key_ring
  • gcp_kms_key
  • gcp_logging_bucket
  • gcp_logging_exclusion
  • gcp_logging_metric
  • gcp_logging_sink
  • gcp_monitoring_alert_policy
  • gcp_monitoring_group
  • gcp_monitoring_notification_channel
  • gcp_organization
  • gcp_project_organization_policy
  • gcp_project_service
  • gcp_pubsub_snapshot
  • gcp_pubsub_subscription
  • gcp_pubsub_topic
  • gcp_service_account
  • gcp_sql_backup
  • gcp_sql_database_instance

Integration test logs

integration_test.txt

Example query results

output.txt

@c0d3r-arnab c0d3r-arnab marked this pull request as ready for review January 11, 2022 16:04
Copy link
Contributor

@ParthaI ParthaI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -312,7 +312,11 @@ func listComputeInstances(ctx context.Context, d *plugin.QueryData, h *plugin.Hy
limit := d.QueryContext.Limit
if d.QueryContext.Limit != nil {
if *limit < *pageSize {
pageSize = limit

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the line no 309 plugin.Logger(ctx).Trace("listComputeInstances", "IAM I HERE")

limit := d.QueryContext.Limit
if d.QueryContext.Limit != nil {
if *limit < *pageSize {
if *limit < 1 {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revisit -- do we need this in GCP?

if *limit < *pageSize {
if *limit < 1 {
pageSize = types.Int64(1)
} else {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

				pageSize = types.Int64(1)
			} else ```

Revisit as GCP does not have any limit issue, we have not experienced this.

Copy link

@rajlearner17 rajlearner17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as per latest testing of all compliance & example queries in this branch

if len(filters) > 0 {
filterString = strings.Join(filters, " ")
}
plugin.Logger(ctx).Trace("listComputeBackendBuckets", "filter string", filterString)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
plugin.Logger(ctx).Trace("listComputeBackendBuckets", "filter string", filterString)

if len(filters) > 0 {
filterString = strings.Join(filters, " ")
}
plugin.Logger(ctx).Trace("listComputeBackendServices", "filter string", filterString)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
plugin.Logger(ctx).Trace("listComputeBackendServices", "filter string", filterString)

if len(filters) > 0 {
filterString = strings.Join(filters, " ")
}
plugin.Logger(ctx).Trace("listComputeFirewalls", "filter string", filterString)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
plugin.Logger(ctx).Trace("listComputeFirewalls", "filter string", filterString)

Copy link
Contributor

@bigdatasourav bigdatasourav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bigdatasourav bigdatasourav merged commit 99825fa into main Jan 18, 2022
@bigdatasourav bigdatasourav deleted the issue-336 branch January 18, 2022 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants