Skip to content

✨ feat(mq-run): auto-parse structured files by extension when no input format is specified#1749

Merged
harehare merged 1 commit into
mainfrom
feat/auto-parse-by-file-extension
May 16, 2026
Merged

✨ feat(mq-run): auto-parse structured files by extension when no input format is specified#1749
harehare merged 1 commit into
mainfrom
feat/auto-parse-by-file-extension

Conversation

@harehare

Copy link
Copy Markdown
Owner

When -I is not given, automatically prepend the appropriate module import and parse call based on the file extension:

  • .json → import "json" | json::json_parse()
  • .yaml/.yml → import "yaml" | yaml::yaml_parse()
  • .toml → import "toml" | toml::toml_parse()
  • .xml → import "xml" | xml::xml_parse()
  • .toon → import "toon" | toon::toon_parse()
  • .csv → import "csv" | csv::csv_parse(true)
  • .tsv → import "csv" | csv::tsv_parse(true)
  • .psv → import "csv" | csv::psv_parse(true)

…t format is specified

When `-I` is not given, automatically prepend the appropriate module
import and parse call based on the file extension:

- .json  → import "json" | json::json_parse()
- .yaml/.yml → import "yaml" | yaml::yaml_parse()
- .toml  → import "toml" | toml::toml_parse()
- .xml   → import "xml"  | xml::xml_parse()
- .toon  → import "toon" | toon::toon_parse()
- .csv   → import "csv"  | csv::csv_parse(true)
- .tsv   → import "csv"  | csv::tsv_parse(true)
- .psv   → import "csv"  | csv::psv_parse(true)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds mq-run behavior to automatically parse structured input files based on file extension when -I/--input-format is not specified, plus related CLI/help and reference documentation.

Changes:

  • Adds extension-to-query-prefix auto parsing for JSON, YAML, TOML, XML, Toon, CSV, TSV, and PSV inputs.
  • Prepends the generated import/parse query before executing user queries.
  • Documents built-in parsing modules and adds unit/CLI tests for selected auto-parse behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
crates/mq-run/src/cli.rs Adds CLI help text, auto-query-prefix logic, execution integration, and tests.
docs/books/src/reference/modules_and_imports.md Adds reference documentation for built-in parsing modules and links to CLI auto-parsing docs.
Comments suppressed due to low confidence (1)

crates/mq-run/src/cli.rs:585

  • Auto-parsing is applied even in --stream mode, where execute receives one line at a time. That makes multi-line .json/.yaml/.toml files fail because each individual line is sent to the structured parser, and it also makes .csv headers unavailable to later rows. Disable the prefix for streaming input or only apply it when the parser sees the full file content.
        let query = match self.auto_query_prefix(file) {
            Some(prefix) => {
                effective_query = format!("{} | {}", prefix, query);

Comment thread crates/mq-run/src/cli.rs
| `csv` | `csv::tsv_parse(has_header)` | Parses TSV (`\t` delimiter) |
| `csv` | `csv::psv_parse(has_header)` | Parses PSV (`\|` delimiter) |

These modules are also used automatically when you process a file whose extension matches (see [CLI auto-parsing](./cli.md#auto-parsing-by-file-extension)).
@harehare harehare merged commit c4e6b3a into main May 16, 2026
12 checks passed
@harehare harehare deleted the feat/auto-parse-by-file-extension branch May 16, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants