Dry run queries should not fail with 404:Not found job

[diego-serrano-glean]

While using the library to execute a query in dry run mode, I got an error like the following: google.api_core.exceptions.NotFound: 404 GET https://bigquery.googleapis.com/bigquery/v2/projects/swast-scratch/queries/None?maxResults=0&location=US: Not found: Job swast-scratch:US.None

The code just submits a query in dry-run mode. This is the code I am trying to run:

from google.cloud import bigquery
from google.cloud.bigquery import dbapi

query_config = bigquery.QueryJobConfig(dry_run=True)
client = bigquery.Client(default_query_job_config=query_config)
conn = dbapi.connect(client=client)
cur = conn.cursor()

# This line fails with "404 job not found" when in dry_run mode.
cur.execute(
    """
    SELECT country_name
    FROM `bigquery-public-data.utility_us.country_code_iso`
    WHERE country_name LIKE 'U%'
    """
)

results = cur.fetchall()
print(results)

What I expect as a result is the stats, or an exception if the query is invalid.

The following is the full stack trace:

$ python try_dbapi.py
Traceback (most recent call last):
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/cloud/bigquery/dbapi/cursor.py", line 175, in execute
    self._query_job.result()
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/cloud/bigquery/job.py", line 3210, in result
    timeout=timeout,
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/cloud/bigquery/client.py", line 1287, in _get_query_results
    retry, method="GET", path=path, query_params=extra_params, timeout=timeout
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/cloud/bigquery/client.py", line 556, in _call_api
    return call()
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
    on_error=on_error,
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target
    return target()
  File "/Users/swast/miniconda3/envs/scratch/lib/python3.7/site-packages/google/cloud/_http.py", line 423, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.NotFound: 404 GET https://bigquery.googleapis.com/bigquery/v2/projects/swast-scratch/queries/None?maxResults=0&location=US: Not found: Job swast-scratch:US.None

[plamut]

I can confirm the issue. There are two things here:

• The DB API Cursor does not take the client's default query job config into account if the latter exists (should be easy to fix).
• Even if dry run is manually set to True, the request still results in 404 when result() is called on such query job.

Update: Interestingly, the minimal sample using the BigQuery client works just fine, even with the query from the issue description:

This query will process 3473 bytes.

The DB API apparently does something wrong behind the scenes.

Update 2: Actually, it's pretty clear - dry runs do not create query jobs on the backend, and the linked sample does not call .result() on the QueryJob object, it simply reads the total_bytes_processed attribute.

DB API cursor should do the same rather than always calling .result() on it.