v3.39.0

v3.39.0 is a minor release of the 3.x maintenance line. It backports bug fixes from master (#1119, #1121, #1128). The 3.x line still supports Python 3.10 and sqlalchemy>=1.0.0. The minimum tenacity version is raised to 8.0.1 (see Dependencies).

Dependencies

• The minimum tenacity version is now 8.0.1 (was 4.1.0). Older releases break retry_api_call(): releases before 6.1.0 fail on Python 3.11+ (asyncio.coroutine was removed), and 8.0.0 raises TypeError from after_log on the first retry (#1126, #1105).

Cursors

• ArrowCursor (ArrowCursor, AsyncArrowCursor, AioArrowCursor) reads result files with the connection session's credentials (#1116, #1115):
  • Profiles without static keys (credential_process, IAM Identity Center, role_arn with source_profile) read result files with the profile's credentials instead of PyArrow's default credential chain.
  • With the role_arn connection argument, PyArrow no longer assumes the role a second time; it reads with the role credentials the connection already obtained (including MFA via serial_number). These credentials expire after duration_seconds, as they do for the connection's boto3 clients.
  • With session=, the session's credentials are used, not aws_access_key_id/aws_secret_access_key connection arguments that the session was not built from.
  • A failure to resolve the session's credentials is raised instead of falling back to PyArrow's default credential chain.
• Native-format arrays keep empty and whitespace-only items and items with commas: ARRAY['x', '', 'y'] used to come back as ['x', 'y'], and ARRAY['a,b', 'c'] as ['a', 'b', 'c']. Leading and trailing spaces of array items are kept (#1054, #1041).
• Typed JSON elements: with array(json), map(..., json), or row(... json) type hints, JSON string elements stay strings ("123" used to become 123), row(json) fields no longer raise JSONDecodeError, and array(json) values whose first element is not a string are parsed (they raised JSONDecodeError) (#1033, #1054).
• PandasCursor:
  • PandasDataFrameIterator.get_chunk() (with chunksize) converts time columns as iteration does, instead of returning Timestamps on the current date (#1054).
  • A NULL TIME read from a result file is None instead of NaT, in the fetch methods and in as_pandas() (isna() is still true for it) (#1054).
• With managed query result storage, PandasCursor and PolarsCursor results with TIME columns no longer raise (AttributeError / TypeError) (#1005).
• PolarsCursor: closing the result set stops a chunk iterator returned by iter_chunks() (#1005).

SQLAlchemy

• DDL treats a table as Iceberg only when its table_type property is ICEBERG (case-insensitive). A table whose properties mention table_type and iceberg without 'table_type'='ICEBERG' now compiles to CREATE EXTERNAL TABLE with the Hive layout (previously CREATE TABLE with the Iceberg layout). On an S3 Tables catalog, such a table raises CompileError ("S3 Tables support only Iceberg tables") (#1043, #1039).

What's Changed

• Backport #1116 to 3.x: Read Arrow results with the connection session's credentials by @laughingman7743 in #1119, which backports:
  • Read Arrow results with the connection session's credentials by @laughingman7743 in #1116
• Backport #1043 and #1054 (with parts of #1005 and #1033) to 3.x by @laughingman7743 in #1121, which backports these master pull requests:
  • Keep the whole pandas and Polars results reusable (GetQueryResults fallback commit only on 3.x) by @laughingman7743 in #1005
  • Convert time zone values to aware times and empty TIME/JSON text to NULL (JSON elements only on 3.x) by @laughingman7743 in #1033
  • Detect Iceberg tables from the table_type property only by @laughingman7743 in #1043
  • Keep complex values that the parsers lost or mixed up by @laughingman7743 in #1054
• Backport #1126 (tenacity floor) and #1127 to 3.x by @laughingman7743 in #1128, which backports:
  • Review the package metadata for 4.0.0 (tenacity floor only on 3.x) by @laughingman7743 in #1126
  • Split typed arrays of comma-free element types at every comma by @laughingman7743 in #1127

Full Changelog: v3.38.0...v3.39.0