Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve error group tracebacks on < py11 #825

Merged
merged 4 commits into from Aug 16, 2023

Conversation

daniel-sanche
Copy link
Contributor

We make use of exception groups to expose a set of exceptions in a single raise, so we can expose multiple retry attempt failures at once, or a number of failed mutations in a bulk request.

Exception Groups is a Python 3.11 feature, meaning it exposes all exceptions in a useful, readable way in newer versions of Python. For 3.10 and lower, the group is treated as a base exception, with most of the useful debug information hidden. This PR patches in sub-exception information into our exception group-compatible objects, making exceptions behave similarly across Python versions

Related Issue: #794


Examples:

RetryExceptionGroup

Previous <=3.10 output:

google.cloud.bigtable.data.exceptions.RetryExceptionGroup: 2 failed attempts. Latest: ValueError

3.11 output:

  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 1, in <module>
  | google.cloud.bigtable.data.exceptions.RetryExceptionGroup: 2 failed attempts
  +-+---------------- 1 ----------------
    | ValueError: 0
    +---------------- 2 ----------------
    | ValueError: 1
    +------------------------------------

New <=3.10 output:

google.cloud.bigtable.data.exceptions.RetryExceptionGroup: 2 failed attempts
--+----------------  1 ----------------
  | ValueError: 0
  +----------------  2 ----------------
  | ValueError: 1
  +------------------------------------

MutationsExceptionGroup

Previous <=3.10 output:

google.cloud.bigtable.data.exceptions.MutationsExceptionGroup: 2 failed entries from 20 attempted.

3.11 output:

  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 1, in <module>
  | google.cloud.bigtable.data.exceptions.MutationsExceptionGroup: 2 failed entries from 20 attempted.
  +-+---------------- 1 ----------------
    | google.cloud.bigtable.data.exceptions.RetryExceptionGroup: 2 failed attempts
    +-+---------------- 1 ----------------
      | ValueError: 0
      +---------------- 2 ----------------
      | ValueError: 1
      +------------------------------------
    |
    | The above exception was the direct cause of the following exception:
    |
    | google.cloud.bigtable.data.exceptions.FailedMutationEntryError: Failed idempotent mutation entry at index 0
    +---------------- 2 ----------------
    | google.cloud.bigtable.data.exceptions.RetryExceptionGroup: 2 failed attempts
    +-+---------------- 1 ----------------
      | ValueError: 0
      +---------------- 2 ----------------
      | ValueError: 1
      +------------------------------------
    |
    | The above exception was the direct cause of the following exception:
    |
    | google.cloud.bigtable.data.exceptions.FailedMutationEntryError: Failed idempotent mutation entry at index 1
    +------------------------------------

New <=3.10 output:

google.cloud.bigtable.data.exceptions.MutationsExceptionGroup: 2 failed entries from 20 attempted.
--+----------------  1 ----------------
  | RetryExceptionGroup: 2 failed attempts
  --+----------------  1 ----------------
    | ValueError: 0
    +----------------  2 ----------------
    | ValueError: 1
    +------------------------------------
  |
  | The above exception was the direct cause of the following exception:
  |
  | FailedMutationEntryError: Failed idempotent mutation entry at index 0
  +----------------  2 ----------------
  | RetryExceptionGroup: 2 failed attempts
  --+----------------  1 ----------------
    | ValueError: 0
    +----------------  2 ----------------
    | ValueError: 1
    +------------------------------------
  |
  | The above exception was the direct cause of the following exception:
  |
  | FailedMutationEntryError: Failed idempotent mutation entry at index 1
  +------------------------------------

ShardedReadRowsExceptionGroup

Previous <=3.10 output

google.cloud.bigtable.data.exceptions.ShardedReadRowsExceptionGroup: 2 sub-exceptions (from 100 queries attempted)

3.11 output:

  + Exception Group Traceback (most recent call last):
  |   File "<stdin>", line 1, in <module>
  | google.cloud.bigtable.data.exceptions.ShardedReadRowsExceptionGroup: 2 sub-exceptions (from 100 queries attempted)
  +-+---------------- 1 ----------------
    | Exception Group Traceback (most recent call last):
    |   File "<stdin>", line 1, in <module>
    | google.cloud.bigtable.data.exceptions.RetryExceptionGroup: 2 failed attempts
    +-+---------------- 1 ----------------
      | ValueError: 0
      +---------------- 2 ----------------
      | ValueError: 1
      +------------------------------------
    |
    | The above exception was the direct cause of the following exception:
    |
    | google.cloud.bigtable.data.exceptions.FailedQueryShardError: Failed query at index 0
    +---------------- 2 ----------------
    | google.cloud.bigtable.data.exceptions.FailedQueryShardError: Failed query at index 0
    +------------------------------------

New <=3.10 output:

google.cloud.bigtable.data.exceptions.ShardedReadRowsExceptionGroup: 2 sub-exceptions (from 100 queries attempted)
--+----------------  1 ----------------
  | RetryExceptionGroup: 2 failed attempts
  --+----------------  1 ----------------
    | ValueError: 0
    +----------------  2 ----------------
    | ValueError: 1
    +------------------------------------
  |
  | The above exception was the direct cause of the following exception:
  |
  | FailedQueryShardError: Failed query at index 0
  +----------------  2 ----------------
  | RetryExceptionGroup: 2 failed attempts
  --+----------------  1 ----------------
    | ValueError: 0
    +----------------  2 ----------------
    | ValueError: 1
    +------------------------------------
  |
  | The above exception was the direct cause of the following exception:
  |
  | FailedQueryShardError: Failed query at index 0
  +------------------------------------

@daniel-sanche daniel-sanche requested review from a team as code owners June 29, 2023 22:30
@conventional-commit-lint-gcf
Copy link

conventional-commit-lint-gcf bot commented Jun 29, 2023

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot
https://conventionalcommits.org/

@product-auto-label product-auto-label bot added size: l Pull request size is large. api: bigtable Issues related to the googleapis/python-bigtable API. labels Jun 29, 2023
@daniel-sanche daniel-sanche merged commit aa760b2 into googleapis:v3 Aug 16, 2023
6 of 8 checks passed
@daniel-sanche daniel-sanche deleted the improve-errors2 branch August 16, 2023 17:42
daniel-sanche added a commit that referenced this pull request Feb 5, 2024
* feat: add new v3.0.0 API skeleton (#745)

* feat: improve rows filters (#751)

* feat: read rows query model class (#752)

* feat: implement row and cell model classes (#753)

* feat: add pooled grpc transport (#748)

* feat: implement read_rows (#762)

* feat: implement mutate rows (#769)

* feat: literal value filter (#767)

* feat: row_exists and read_row (#778)

* feat: read_modify_write and check_and_mutate_row (#780)

* feat: sharded read rows (#766)

* feat: ping and warm with metadata (#810)

* feat: mutate rows batching (#770)

* chore: restructure module paths (#816)

* feat: improve timeout structure (#819)

* fix: api errors apply to all bulk mutations

* chore: reduce public api surface (#820)

* feat: improve error group tracebacks on < py11 (#825)

* feat: optimize read_rows (#852)

* chore: add user agent suffix (#842)

* feat: optimize retries (#854)

* feat: add test proxy (#836)

* chore(tests): add conformance tests to CI for v3 (#870)

* chore(tests): turn off fast fail for conformance tets (#882)

* feat: add TABLE_DEFAULTS enum for table method arguments (#880)

* fix: pass None for retry in gapic calls (#881)

* feat: replace internal dictionaries with protos in gapic calls (#875)

* chore: optimize gapic calls (#863)

* feat: expose retryable error codes to users (#879)

* chore: update api_core submodule (#897)

* chore: merge main into experimental_v3 (#900)

* chore: pin conformance tests to v0.0.2 (#903)

* fix: bulk mutation eventual success (#909)

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/python-bigtable API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants