Add chunk processing support for PolarsCursor

[laughingman7743]

Summary

• Add chunksize parameter to PolarsCursor and AsyncPolarsCursor
• Implement lazy chunk loading for memory-efficient data processing
• When chunksize is set, all data access methods (fetchone(), fetchmany(), fetchall(), iter_chunks()) load data lazily in chunks
• Use Polars' native lazy evaluation APIs (scan_csv(), scan_parquet() with collect_batches())
• Support both CSV and Parquet (UNLOAD) result formats

Motivation

This follows the same pattern as PandasCursor's chunksize option, allowing users to process large datasets without loading the entire result set into memory.

Usage

Standard fetch methods with chunked loading

from pyathena import connect
from pyathena.polars.cursor import PolarsCursor

# With chunksize, data is loaded lazily in chunks
cursor = connect(s3_staging_dir="s3://bucket/path/",
                 region_name="us-west-2",
                 cursor_class=PolarsCursor).cursor(chunksize=50_000)

cursor.execute("SELECT * FROM large_table")

# fetchone/fetchmany load data chunk by chunk as needed
for row in cursor:
    process_row(row)

Explicit chunk iteration

cursor = connect(...).cursor(PolarsCursor, chunksize=50_000)
cursor.execute("SELECT * FROM large_table")

for chunk in cursor.iter_chunks():
    # Each chunk is a polars.DataFrame
    process_chunk(chunk)

Implementation Details

• DataFrameIterator: Wrapper class that provides unified iteration over chunked or non-chunked DataFrames
• When chunksize is None (default): Eager loading (existing behavior)
• When chunksize is set: Lazy evaluation with bounded memory usage

Changes

• pyathena/polars/result_set.py: Add DataFrameIterator class, chunksize parameter, lazy chunk loading
• pyathena/polars/cursor.py: Add chunksize parameter and iter_chunks() method
• pyathena/polars/async_cursor.py: Add chunksize parameter
• docs/polars.rst: Add documentation for chunksize options with examples
• tests/pyathena/polars/test_cursor.py: Add comprehensive tests for chunk processing

Test Plan

• test_iter_chunks - Basic chunk iteration
• test_iter_chunks_without_chunksize - Yields entire DataFrame as single chunk
• test_iter_chunks_many_rows - Large dataset chunk iteration
• test_iter_chunks_unload - Chunk iteration with Parquet/UNLOAD
• test_iter_chunks_data_consistency - Chunked vs regular reading produce same data
• test_iter_chunks_chunk_sizes - Chunk size validation
• test_fetchone_with_chunksize - fetchone with lazy loading
• test_fetchmany_with_chunksize - fetchmany with lazy loading
• test_fetchall_with_chunksize - fetchall with lazy loading
• test_iterator_with_chunksize - Iterator protocol with lazy loading

🤖 Generated with Claude Code