Skip to content

[orm] QuerySet .iterator(chunk_size) — chunked server-side streaming #23

@ujeenet

Description

@ujeenet

Django ref: https://docs.djangoproject.com/en/6.0/ref/models/querysets/#iterator

What's missing

.iterator(chunk_size=2000) returns an iterator over results, fetching one chunk at a time from a server-side cursor — never buffering the whole result set in memory.

Current rustango state

fetch_pool returns Vec<T> fully buffered. Apps processing million-row exports OOM.

Acceptance

  • QuerySet::iterator_pool(chunk_size, &pool) → impl Stream<Item=Result<T>>.
  • Tri-dialect: PG cursor support, MySQL streaming, SQLite (always streams).
  • Integration test with a large fixture.

Part of the QuerySet API Extensions epic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    django-parityDjango 6.0 parity gap (from docs.djangoproject.com/en/6.0 audit)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions