reqstorm 2.1.0

Structured data: JSON responses to typed columns

Give a schema and each JSON field goes to its own typed column, checked before it is written:

from reqstorm import Field

schema = {
    "id": Field("id", int, required=True, key=True),
    "price": Field("pricing.amount", float),
    "tags": Field("tags", "json"),
    "page": Field("$.meta.page", int),
}
reqstorm.fetch_to_db_sync(urls, connection, table="products", schema=schema,
                          explode="items", rejects_table="products_rejects")

• Typed columns: int, float, str, bool, "datetime" and "json" become BIGINT, DOUBLE PRECISION, TEXT, BOOLEAN, TIMESTAMPTZ and JSONB in PostgreSQL, with equivalents in MySQL and SQLite.
• Strict checks: NaN and Infinity are always rejected. coerce=True accepts compatible values such as "12.5".
• Nested JSON: dotted paths, "json" columns, and explode for one row per array element.
• Rejected records: never written; listed with their reasons in Summary.errors and the optional rejects table.
• Upserts and resume: upserts on key=True fields; resume through the source columns.
• Same schema for files: works for SQLite files, JSON Lines and CSV.
• New helpers: reqstorm.extract(results, schema) and reqstorm.infer_schema(samples).

Guide: https://reqstorm.github.io/guide/structured-data/