Skip to content

v1.4.0

Choose a tag to compare

View all tags
@ivbeg ivbeg released this 26 Jun 15:36
· 20 commits to master since this release
v1.4.0
95587be

What's new in 1.4.0

This release builds on iterabledata as the I/O foundation to add several net-new capabilities.

Added

  • ai commands — AI-assisted workflows backed by iterable.ai: ai doc (block-based dataset documentation with metadata enrichment and PII-safe sampling), ai filter (natural-language/DSL → filter translation, with --apply to execute and stream matching rows), ai plan (declarative conversion planning), and ai suggest (transform suggestions). Supports OpenAI, Anthropic, Gemini, Azure, OpenRouter, Ollama, LM Studio, and Perplexity.
  • formats commands — formats list surfaces iterabledata's full format catalog; --capabilities shows the runtime capability matrix (read/write/streaming/etc.) per format, with machine-readable JSON output.
  • mcp commands — mcp serve starts a Model Context Protocol stdio server exposing undatum's agent tools; mcp tools lists them. New undatum-mcp console script.
  • Agent tools (undatum.tools) — 17 JSON-schema agent tools for LLM function calling: the 12 iterabledata foundation tools plus 5 undatum-specific tools (query_sql, frequency, deduplicate, mask_fields, sample_data). Includes OpenAI/Anthropic tool definitions, a unified call_tool, and a LangChain get_tools() adapter.
  • SDK DataFrame & typed-row interop — Dataset.to_pandas(), to_polars(), to_dask(), as_dataclasses(), and as_pydantic().
  • Bulk conversion — convert --recursive (with --to-ext) converts a directory or glob pattern.
  • Extended database engines — db query and file-reading commands now reach MS SQL Server (mssql://, sqlserver://), ClickHouse (clickhouse://), MongoDB (mongodb://), and Elasticsearch/OpenSearch (elasticsearch://, opensearch://) via iterabledata's read-only drivers; driver options can be passed through the URI query string.
  • Cloud storage URIs beyond S3 — GCS (gs:///gcs://), Azure (az:///abfs:///abfss://), and s3a:// are opened natively via iterabledata's fsspec support; s3:// writes are now supported.
  • New optional extras: mcp, langchain, polars, dask, cloud (fsspec/s3fs/gcsfs/adlfs), mssql (pyodbc), clickhouse (clickhouse-driver).

Changed

  • convert now routes through iterabledata's engine, supporting any format it can read/write (100+ formats, including cloud URIs). Read-only and schema-required output formats (protobuf, Cap'n Proto, Thrift) fail fast with actionable, capability-aware errors and writable-format suggestions.
  • SUPPORTED_FILE_TYPES, COMPRESSED_FILE_TYPES, TEXT_DATA_TYPES, and BINARY_FILE_TYPES are now derived at import time from iterabledata's registries (with static fallbacks).
  • doc command metadata extraction and semantic-type/PII detection now delegate to iterable.ai.metadata and iterable.ai.semantic while keeping CLI output backward compatible.

Full Changelog: v1.3.0...v1.4.0

Assets 2
Loading