Timeout while listing BigQuery datasets.

[ahoying]

From support email:

I also just ran another debug run and again BQ hung here's a snippet of that run:

2017-10-11 11:29:21,138 main INFO _run_pipelines Running pipeline LoadBigqueryDatasetsPipeline
2017-10-11 11:29:22,164 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #1
2017-10-11 11:29:24,126 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #2
2017-10-11 11:29:25,961 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #3
2017-10-11 11:29:27,931 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #4
2017-10-11 11:29:30,017 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #5
2017-10-11 11:29:32,357 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #6
2017-10-11 11:29:34,907 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #7
2017-10-11 11:29:37,425 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #8
2017-10-11 11:29:39,512 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #9
2017-10-11 11:29:41,035 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #10
[ 686 lines removed ]
2017-10-11 11:31:35,611 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #1
2017-10-11 11:31:35,663 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #1
2017-10-11 11:31:35,710 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #1
2017-10-11 11:31:35,762 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #1
2017-10-11 11:31:35,812 google.cloud.security.common.gcp_api._base_repository DEBUG execute_paged_query Executing paged request #1
[ Manually terminated ]

Looks like it processed ~700 calls in ~134s. Then just stopped. I let it sit for probably 3m before terminating it.

[ahoying]

This should be fixed with the 2.0 crawling architecture, closing issue.