Skip to content

RFC 022: Semi-structured and format functions #39

@dannymeijer

Description

@dannymeijer

Use this issue to track InQL RFC 022, which already exists at docs/rfcs/022_semi_structured_format_functions.md.

Area

  • Specification (RFCs)
  • Package & tests
  • Documentation

Summary

RFC 022 defines InQL's semi-structured and format-oriented function families: JSON value functions, CSV value functions, schema inference helpers, type predicates for dynamic values, URL helpers, and hashing functions.

Motivation

These functions are practical data-engineering tools, but they should live in explicit format families rather than being mixed into the core scalar catalog.

Proposal sketch

The RFC defines format-oriented scalar function families with explicit schema/option typing, error behavior, and lowering/diagnostic expectations. Source and sink contracts remain separate from scalar payload parsing.

Open design questions to resolve before Planned:

  • Should from_json accept model types directly as schema arguments, or only explicit schema values?
  • Should invalid JSON path expressions be compile-time errors when literal and runtime errors otherwise?
  • What option-record shape should CSV and JSON scalar parsers use?
  • Should hash functions return binary values or lowercase hexadecimal strings by default?
  • Which variant-style type predicates are portable enough for InQL core, and which should stay in a Snowflake-compatibility extension?

Alternatives considered

The RFC rejects placing all format helpers in the common scalar catalog, making JSON and CSV functions source-only, and adding full XML/variant support in the same RFC.

Impact / compatibility

This RFC affects format helper APIs, typechecking for structured return values and schema/model arguments, Prism/Substrait lowering or diagnostics for parser options and return values, and docs separating scalar format functions from session read/write APIs.

Implementation notes (optional)

Handle after the registry and scalar foundations, and coordinate with RFC 024 for dialect or extension-only format behavior.

Checklist

  • I checked for an existing RFC or issue covering this.
  • I can describe how this impacts existing code and how to migrate (if needed).

Metadata

Metadata

Assignees

No one assigned

    Labels

    RFCRFC design and planningdocumentationImprovements or additions to documentationpackageLibrary source, tests, incan.tomlspecificationdocs/rfcs/ normative RFCs

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions