v3.37.0

v3.37.0 is a minor release of the 3.x maintenance line. It backports the SQLAlchemy ARRAY support and fixes from master (#893). The 3.x line keeps supporting Python 3.10 and sqlalchemy>=1.0.0.

SQLAlchemy

• Reflected array<...> columns are AthenaArray with their element types, instead of String, and types.ARRAY maps to AthenaArray (#774).
• SQLAlchemy ARRAY values, expressions (indexing, slices, concatenation, ANY/ALL), and partial updates are supported (#774, #775, #776).
• Behavior changes for SQLAlchemy queries that select ARRAY values (#774):
  • Typed SELECT results are decoded through a JSON projection, so rewritten queries need explicit SELECT columns, and literal SQL expressions need labels.
  • Textual ORDER BY supports selected names, lists of them, and ordinals. Use SQLAlchemy expressions for other orderings.
  • Raw SQL and arrays with unknown element types keep the documented cursor fallback.
  • ARRAY binds of decimal values require an explicit Numeric precision and scale, to prevent silent rounding.
• CREATE TABLE renders STRUCT column types, including STRUCT values inside MAP and ARRAY, as Hive STRUCT<name:type>. Athena rejected the previous ROW(...) output (#870, #855). Thanks to @mvanhorn for the fix and @aminghadersohi for the report. Integers inside ARRAY, MAP, and STRUCT column types render as INT. Code that compares compiled DDL strings must expect STRUCT<...> and INT.
• CAST to Double or DOUBLE_PRECISION renders DOUBLE instead of the 32-bit REAL (#826).
• The ARRAY support was checked at compile level on SQLAlchemy 1.4.54. The test suite runs on SQLAlchemy 2.0.

Cursors

• CSV result sets keep binary NULLs as None (#770).

Development

• The Test workflow runs the AWS suites only for ready pull requests, on the newest Python version (#789, #837, #863). Tests are rerun once on Athena service-side failures (#811), and each test session uses its own S3 Tables namespace (#815).
• The Release workflow runs every suite on every supported Python version before publishing (#862).

What's Changed

• Backport CI and SQLAlchemy changes to 3.x for v3.37.0 by @laughingman7743 in #893, which backports these master pull requests:
  • Enable Iceberg UPDATE target subquery compliance by @laughingman7743 in #769
  • Restore SQLAlchemy CTE compliance coverage by @laughingman7743 in #772
  • Restore SQLAlchemy binary compliance and preserve CSV binary NULLs by @laughingman7743 in #770
  • Skip AWS integration jobs for external fork pull requests by @laughingman7743 in #789
  • Support native SQLAlchemy ARRAY types and typed round trips by @laughingman7743 in #774
  • Support SQLAlchemy ARRAY expressions in SELECT and WHERE by @laughingman7743 in #775
  • Support SQLAlchemy ARRAY partial UPDATE and slice resizing by @laughingman7743 in #776
  • Rerun tests once on Athena service-side query failures by @laughingman7743 in #811
  • Give each test session its own S3 Tables namespace by @laughingman7743 in #815
  • Run the three test suites in parallel by @laughingman7743 in #814
  • Re-enable SQLAlchemy parameter and SQL rendering compliance tests by @laughingman7743 in #825
  • SQLAlchemy compliance: audit numeric range and float precision by @laughingman7743 in #826
  • Run AWS test suites only on ready pull requests and related changes by @laughingman7743 in #837
  • Run pull-request AWS tests on the newest Python version only by @laughingman7743 in #863
  • Run the full test matrix in the Release workflow before publishing by @laughingman7743 in #862
  • Set up the PyAthena test fixtures only in processes that run tests by @laughingman7743 in #866
  • Insert SQLAlchemy compliance fixture rows once for read-only classes by @laughingman7743 in #867
  • Give the executemany and partition tests their own tables by @laughingman7743 in #873
  • fix: render Hive STRUCT syntax in table column DDL by @mvanhorn in #870

New Contributors

• @mvanhorn made their first contribution in #870

Full Changelog: v3.36.0...v3.37.0