Skip to content

v1.0.13

Choose a tag to compare

View all tags
@ivbeg ivbeg released this 25 Jun 07:49
v1.0.13
88c0e05

iterabledata 1.0.13

Highlight: block-based AI documentation with structured, machine-readable output.

Added

  • Block-based AI documentation: New ai.doc.generate_blocks() produces structured documentation as independent blocks (general, schema, quality, examples, statistics, codebook) returning {source, blocks{name:{markdown,data}}, full_document_markdown}. Backed by a block registry (iterable/ai/blocks.py); lineage and geo_coverage are registered but deferred. ai.doc.generate() stays fully backward compatible and delegates to the block engine.
  • Structured LLM output: generate_structured() on the provider abstraction (JSON Schema where supported, json_object + Pydantic validation fallback) with per-block Pydantic models. Schema-block LLM calls are batched when columns > 100.
  • AI context, progress, and sampling: context parameter threaded into prompts; in-process progress hooks with a Stage enum and progress callback plus per-stage structured JSON logging; size-based sampling tiers driven by MAX_ROWS_SAMPLING.
  • File metadata and multi-table docs: File metadata helpers (size, sha256, encoding, record_count, table_count); XLS/XLSX multi-table documentation with tables= selection and per-table schema blocks.
  • Generic OpenAI-compatible provider: Honors LLM_PROVIDER / LLM_BASE_URL / LLM_API_KEY / LLM_DEFAULT_MODEL environment variables.
  • Extended column statistics: ops.stats.compute() gains optional null_fraction, top_values, and is_dictionary.
  • Format metadata registry, ingest/codec deduplication, uncovered format tests, conformance fixture auto-discovery, format doc stubs, and per-package coverage tracking.

Changed

  • Fixture consolidation, detect/guidance/dataframe/codec module splits, partial strict mypy core (blocking in CI), documented placeholder formats, ZIP wrappers on BaseFileIterable, fixed Docusaurus docs build, pre-commit Ruff bumped to v0.14.9, per-format install hints.

Fixed

  • XML tagname no longer overwritten with None after super().__init__().
  • Pickle read_bulk returns [] at EOF instead of raising StopIteration.

See CHANGELOG.md for full details.

Assets 4
Loading