tests.system.test_gbq: test_create_table failed

[flaky-bot]

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

---

commit: 928e47b
buildURL: Build Status, Sponge
status: failed

Test output

target = functools.partial(functools.partial(
sleep_generator = 
deadline = 600.0, on_error = None
def retry_target(target, predicate, sleep_generator, deadline, on_error=None):
    """Call a function and retry if it fails.

    This is the lowest-level retry helper. Generally, you'll use the
    higher-level retry helper :class:`Retry`.

    Args:
        target(Callable): The function to call and retry. This must be a
            nullary function - apply arguments with `functools.partial`.
        predicate (Callable[Exception]): A callable used to determine if an
            exception raised by the target should be considered retryable.
            It should return True to retry or False otherwise.
        sleep_generator (Iterable[float]): An infinite iterator that determines
            how long to sleep between retries.
        deadline (float): How long to keep retrying the target. The last sleep
            period is shortened as necessary, so that the last retry runs at
            ``deadline`` (and not considerably beyond it).
        on_error (Callable[Exception]): A function to call while processing a
            retryable exception.  Any error raised by this function will *not*
            be caught.

    Returns:
        Any: the return value of the target function.

    Raises:
        google.api_core.RetryError: If the deadline is exceeded while retrying.
        ValueError: If the sleep generator stops yielding values.
        Exception: If the target raises a method that isn't retryable.
    """
    if deadline is not None:
        deadline_datetime = datetime_helpers.utcnow() + datetime.timedelta(
            seconds=deadline
        )
    else:
        deadline_datetime = None

    last_exc = None

    for sleep in sleep_generator:
        try:


          return target()


.nox/prerelease/lib/python3.8/site-packages/google/api_core/retry.py:190:

self = <google.cloud.bigquery._http.Connection object at 0x7f3e623b6dc0>

method = 'GET'

path = '/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table'

query_params = None, data = None, content_type = None, headers = None

api_base_url = None, api_version = None, expect_json = True

_target_object = None, timeout = None
def api_request(
    self,
    method,
    path,
    query_params=None,
    data=None,
    content_type=None,
    headers=None,
    api_base_url=None,
    api_version=None,
    expect_json=True,
    _target_object=None,
    timeout=_DEFAULT_TIMEOUT,
):
    """Make a request over the HTTP transport to the API.

    You shouldn't need to use this method, but if you plan to
    interact with the API using these primitives, this is the
    correct one to use.

    :type method: str
    :param method: The HTTP method name (ie, ``GET``, ``POST``, etc).
                   Required.

    :type path: str
    :param path: The path to the resource (ie, ``'/b/bucket-name'``).
                 Required.

    :type query_params: dict or list
    :param query_params: A dictionary of keys and values (or list of
                         key-value pairs) to insert into the query
                         string of the URL.

    :type data: str
    :param data: The data to send as the body of the request. Default is
                 the empty string.

    :type content_type: str
    :param content_type: The proper MIME type of the data provided. Default
                         is None.

    :type headers: dict
    :param headers: extra HTTP headers to be sent with the request.

    :type api_base_url: str
    :param api_base_url: The base URL for the API endpoint.
                         Typically you won't have to provide this.
                         Default is the standard API base URL.

    :type api_version: str
    :param api_version: The version of the API to call.  Typically
                        you shouldn't provide this and instead use
                        the default for the library.  Default is the
                        latest API version supported by
                        google-cloud-python.

    :type expect_json: bool
    :param expect_json: If True, this method will try to parse the
                        response as JSON and raise an exception if
                        that cannot be done.  Default is True.

    :type _target_object: :class:`object`
    :param _target_object:
        (Optional) Protected argument to be used by library callers. This
        can allow custom behavior, for example, to defer an HTTP request
        and complete initialization of the object at a later time.

    :type timeout: float or tuple
    :param timeout: (optional) The amount of time, in seconds, to wait
        for the server response.

        Can also be passed as a tuple (connect_timeout, read_timeout).
        See :meth:`requests.Session.request` documentation for details.

    :raises ~google.cloud.exceptions.GoogleCloudError: if the response code
        is not 200 OK.
    :raises ValueError: if the response content type is not JSON.
    :rtype: dict or str
    :returns: The API response payload, either as a raw string or
              a dictionary if the response is valid JSON.
    """
    url = self.build_api_url(
        path=path,
        query_params=query_params,
        api_base_url=api_base_url,
        api_version=api_version,
    )

    # Making the executive decision that any dictionary
    # data will be sent properly as JSON.
    if data and isinstance(data, dict):
        data = json.dumps(data)
        content_type = "application/json"

    response = self._make_request(
        method=method,
        url=url,
        data=data,
        content_type=content_type,
        headers=headers,
        target_object=_target_object,
        timeout=timeout,
    )

    if not 200 <= response.status_code < 300:


      raise exceptions.from_http_response(response)


E           google.api_core.exceptions.BadGateway: 502 GET https://bigquery.googleapis.com/bigquery/v2/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table?prettyPrint=false: 

E           

E             

E             

E             <title>Error 502 (Server Error)!!1</title>

E             <style>

E               {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

E             </style>

E             

E             
502. That’s an error.

E             
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.  That’s all we know.
.nox/prerelease/lib/python3.8/site-packages/google/cloud/_http/init.py:480: BadGateway
The above exception was the direct cause of the following exception:
self = <pandas_gbq.gbq._Table object at 0x7f3e623b6580>

table_id = 'test_create_table'
def exists(self, table_id):
    """Check if a table exists in Google BigQuery

    Parameters
    ----------
    table : str
        Name of table to be verified

    Returns
    -------
    boolean
        true if table exists, otherwise false
    """
    from google.api_core.exceptions import NotFound

    table_ref = self._table_ref(table_id)
    try:


      self.client.get_table(table_ref)


pandas_gbq/gbq.py:1184:

self = <google.cloud.bigquery.client.Client object at 0x7f3e623b6c10>

table = TableReference(DatasetReference('precise-truck-742', 'python_bigquery_pandas_tests_system_20211130190108_4fc5fb'), 'test_create_table')

retry = <google.api_core.retry.Retry object at 0x7f3e68a3c850>, timeout = None
def get_table(
    self,
    table: Union[Table, TableReference, TableListItem, str],
    retry: retries.Retry = DEFAULT_RETRY,
    timeout: TimeoutType = DEFAULT_TIMEOUT,
) -> Table:
    """Fetch the table referenced by ``table``.

    Args:
        table (Union[ \
            google.cloud.bigquery.table.Table, \
            google.cloud.bigquery.table.TableReference, \
            google.cloud.bigquery.table.TableListItem, \
            str, \
        ]):
            A reference to the table to fetch from the BigQuery API.
            If a string is passed in, this method attempts to create a
            table reference from a string using
            :func:`google.cloud.bigquery.table.TableReference.from_string`.
        retry (Optional[google.api_core.retry.Retry]):
            How to retry the RPC.
        timeout (Optional[float]):
            The number of seconds to wait for the underlying HTTP transport
            before using ``retry``.

    Returns:
        google.cloud.bigquery.table.Table:
            A ``Table`` instance.
    """
    table_ref = _table_arg_to_table_ref(table, default_project=self.project)
    path = table_ref.path
    span_attributes = {"path": path}


  api_response = self._call_api(


        retry,
        span_name="BigQuery.getTable",
        span_attributes=span_attributes,
        method="GET",
        path=path,
        timeout=timeout,
    )

.nox/prerelease/lib/python3.8/site-packages/google/cloud/bigquery/client.py:999:

self = <google.cloud.bigquery.client.Client object at 0x7f3e623b6c10>

retry = <google.api_core.retry.Retry object at 0x7f3e68a3c850>

span_name = 'BigQuery.getTable'

span_attributes = {'path': '/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table'}

job_ref = None, headers = None

kwargs = {'method': 'GET', 'path': '/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table', 'timeout': None}

call = <function Retry.call..retry_wrapped_func at 0x7f3e3b7b1280>
def _call_api(
    self,
    retry,
    span_name=None,
    span_attributes=None,
    job_ref=None,
    headers: Optional[Dict[str, str]] = None,
    **kwargs,
):
    kwargs = _add_server_timeout_header(headers, kwargs)
    call = functools.partial(self._connection.api_request, **kwargs)

    if retry:
        call = retry(call)

    if span_name is not None:
        with create_span(
            name=span_name, attributes=span_attributes, client=self, job_ref=job_ref
        ):


          return call()


.nox/prerelease/lib/python3.8/site-packages/google/cloud/bigquery/client.py:747:

args = (), kwargs = {}

target = functools.partial(functools.partial(<bound method JSONConnection.api_request of <google.cloud.bigquery._http.Connectio...-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table', timeout=None))

sleep_generator = <generator object exponential_sleep_generator at 0x7f3e3b7cbeb0>
@functools.wraps(func)
def retry_wrapped_func(*args, **kwargs):
    """A wrapper that calls target function with retry."""
    target = functools.partial(func, *args, **kwargs)
    sleep_generator = exponential_sleep_generator(
        self._initial, self._maximum, multiplier=self._multiplier
    )


  return retry_target(


        target,
        self._predicate,
        sleep_generator,
        self._deadline,
        on_error=on_error,
    )

.nox/prerelease/lib/python3.8/site-packages/google/api_core/retry.py:283:

target = functools.partial(functools.partial(<bound method JSONConnection.api_request of <google.cloud.bigquery._http.Connectio...-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table', timeout=None))

predicate = <function _should_retry at 0x7f3e68a34ca0>

sleep_generator = <generator object exponential_sleep_generator at 0x7f3e3b7cbeb0>

deadline = 600.0, on_error = None
def retry_target(target, predicate, sleep_generator, deadline, on_error=None):
    """Call a function and retry if it fails.

    This is the lowest-level retry helper. Generally, you'll use the
    higher-level retry helper :class:`Retry`.

    Args:
        target(Callable): The function to call and retry. This must be a
            nullary function - apply arguments with `functools.partial`.
        predicate (Callable[Exception]): A callable used to determine if an
            exception raised by the target should be considered retryable.
            It should return True to retry or False otherwise.
        sleep_generator (Iterable[float]): An infinite iterator that determines
            how long to sleep between retries.
        deadline (float): How long to keep retrying the target. The last sleep
            period is shortened as necessary, so that the last retry runs at
            ``deadline`` (and not considerably beyond it).
        on_error (Callable[Exception]): A function to call while processing a
            retryable exception.  Any error raised by this function will *not*
            be caught.

    Returns:
        Any: the return value of the target function.

    Raises:
        google.api_core.RetryError: If the deadline is exceeded while retrying.
        ValueError: If the sleep generator stops yielding values.
        Exception: If the target raises a method that isn't retryable.
    """
    if deadline is not None:
        deadline_datetime = datetime_helpers.utcnow() + datetime.timedelta(
            seconds=deadline
        )
    else:
        deadline_datetime = None

    last_exc = None

    for sleep in sleep_generator:
        try:
            return target()

        # pylint: disable=broad-except
        # This function explicitly must deal with broad exceptions.
        except Exception as exc:
            if not predicate(exc):
                raise
            last_exc = exc
            if on_error is not None:
                on_error(exc)

        now = datetime_helpers.utcnow()

        if deadline_datetime is not None:
            if deadline_datetime <= now:


              raise exceptions.RetryError(


                    "Deadline of {:.1f}s exceeded while calling {}".format(
                        deadline, target
                    ),
                    last_exc,
                ) from last_exc

E                   google.api_core.exceptions.RetryError: Deadline of 600.0s exceeded while calling functools.partial(functools.partial(<bound method JSONConnection.api_request of <google.cloud.bigquery._http.Connection object at 0x7f3e623b6dc0>>, method='GET', path='/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table', timeout=None)), last exception: 502 GET https://bigquery.googleapis.com/bigquery/v2/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table?prettyPrint=false: 

E                   

E                     

E                     

E                     <title>Error 502 (Server Error)!!1</title>

E                     <style>

E                       {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

E                     </style>

E                     

E                     
502. That’s an error.

E                     
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.  That’s all we know.
.nox/prerelease/lib/python3.8/site-packages/google/api_core/retry.py:205: RetryError
During handling of the above exception, another exception occurred:
gbq_table = <pandas_gbq.gbq._Table object at 0x7f3e623b6580>
def test_create_table(gbq_table):
    schema = gbq._generate_bq_schema(tm.makeMixedDataFrame())


  gbq_table.create("test_create_table", schema)


tests/system/test_gbq.py:1464:

pandas_gbq/gbq.py:1206: in create

if self.exists(table_id):

pandas_gbq/gbq.py:1189: in exists

self.process_http_error(ex)

ex = RetryError("Deadline of 600.0s exceeded while calling functools.partial(functools.partial(<bound method JSONConnection...ruck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table', timeout=None))")
@staticmethod
def process_http_error(ex):
    # See `BigQuery Troubleshooting Errors
    # <https://cloud.google.com/bigquery/troubleshooting-errors>`__


  raise GenericGBQException("Reason: {0}".format(ex))


E       pandas_gbq.exceptions.GenericGBQException: Reason: Deadline of 600.0s exceeded while calling functools.partial(functools.partial(<bound method JSONConnection.api_request of <google.cloud.bigquery._http.Connection object at 0x7f3e623b6dc0>>, method='GET', path='/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table', timeout=None)), last exception: 502 GET https://bigquery.googleapis.com/bigquery/v2/projects/precise-truck-742/datasets/python_bigquery_pandas_tests_system_20211130190108_4fc5fb/tables/test_create_table?prettyPrint=false: 

E       

E         

E         

E         <title>Error 502 (Server Error)!!1</title>

E         <style>

E           {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

E         </style>

E         

E         
502. That’s an error.

E         
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.  That’s all we know.
pandas_gbq/gbq.py:375: GenericGBQException

[tswast]

In the prerelease job, but does seem to be a genuine flake.

[tswast]

Related to #418 ?

[flaky-bot]

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

---

When run at the same commit (928e47b), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

[tswast]

Looks like BadGateway is already a retryable exception and this timed out after 10 minutes.

https://github.com/googleapis/python-bigquery/blob/44221f437f93fae2fac10d074b39a28008b9ad01/google/cloud/bigquery/retry.py#L29

I don't think there's much else we can do. Was probably a transient issue in the backend.