Skip to content

feat: add Converter node for CSV <-> JSON conversion - #429

Merged
ckakgun merged 4 commits into
heymrun:mainfrom
asimodabas:impl/converter-node
Aug 1, 2026
Merged

feat: add Converter node for CSV <-> JSON conversion#429
ckakgun merged 4 commits into
heymrun:mainfrom
asimodabas:impl/converter-node

Conversation

@asimodabas

Copy link
Copy Markdown
Contributor

Adds a converter node for format conversion inside workflows, as discussed in #423.

What

A technology-neutral Converter node (Data category, 1 input / 1 output, no credentials). First conversions:

  • csvToJson — CSV text → array of row objects (or arrays when there is no header)
  • jsonToCsv — array of objects/rows → CSV text

The conversion field leaves room for more formats later without changing the node's contract. Parsing/formatting uses Python's csv module, so quoting, embedded delimiters, and newlines follow RFC 4180. Output is exposed as $label.result.

Wiring (per AGENTS.md)

  • Backend: modular handler + registry entry, DSL prompt block, and output-expression hint.
  • Frontend: node type, definition, properties panel, icon/color, and node registration across the required maps.
  • Docs: new node page plus node reference/manifest updates.

Tests

  • Backend unit tests for the handler (both directions, headers, custom delimiter, quoting/escaping, empty input, build→parse round-trip).
  • Playwright spec for the properties panel (fields render/switch per conversion direction).

The change is purely additive and doesn't touch existing node behavior. ./check.sh and the E2E suite pass locally.

@ckakgun

ckakgun commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Hi @asimodabas 👑

Thanks for this, it is a clean contribution. The Converter naming, the modular
handler and registry wiring, the docs page and the unit tests all match what we
asked for, and RFC 4180 handling looks solid. Lint, typecheck, ruff and the
backend tests are green on my side too.

Three things before we merge:

  1. Expression dialog wiring is missing. AGENTS.md asks that expression-capable
    fields be reachable from the evaluate dialog with 1/n navigation, but
    "converter" is not in the openPrimaryExpandDialogForSelectedNode chain or the
    selectedNodeHasPrimaryEvaluateExpandTarget switch in
    usePropertiesPanelController.ts. The consoleLog branch is a good template.

  2. features.md needs the node in both places. The per-node section is there, but
    the node types summary paragraph still reads "data nodes like Set, Variable,
    and Execute".

  3. Parsing edge cases: a UTF-8 BOM is not stripped, so Excel exports give a
    first key of "\ufeffname"; duplicate headers collapse ("a,a\n1,2" returns
    [{"a": "2"}]); and the delimiter is truncated with [:1], so a tab cannot be
    entered at all.

One more: the original proposal had trimValues, defaulting to true. It is not in
the PR, so "name, age" keeps the space and the key comes back as " age". Was
leaving it out deliberate, or should we add it?

And once again, thank you for your time and effort. We're so close to merging this MR. 🚀🔥

…mmary, BOM/duplicate-header/tab handling, trimValues
@asimodabas

Copy link
Copy Markdown
Contributor Author

Thanks, good catches 🪄 . Pushed fixes for all four:

  1. Wired converter into openPrimaryExpandDialogForSelectedNode and the selectedNodeHasPrimaryEvaluateExpandTarget switch, with the source field on a converterSourceInputRef (used the consoleLog branch as the template). Evaluate dialog opens with 1/n now.

  2. Added Converter to the node-types summary paragraph in features.md.

  3. Parsing:

    • strip a leading UTF-8 BOM so Excel exports don't give a name key with the BOM glued on
    • dedupe headers, so a,a\n1,2 gives [{"a": "1", "a_2": "2"}]
    • \t now resolves to a real tab instead of being cut by the [:1]
  4. Added trimValues back (default true, csvToJson only) name, age gives age instead of age. I'd trimmed the field set to keep the first version lean and ended up skipping this one good catch 🌟 It's now a toggle in the panel and the docs.

Added tests for the BOM, duplicate-header, tab and trim cases, plus a Playwright check for the trim toggle. check.sh and e2e are green locally.

@ckakgun

ckakgun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Hi Asım 🙋‍♀️ ,
I've just verified everything locally and it all checks out. The evaluate dialog opens
with 1/n, the summary paragraph is updated, and BOM, duplicate headers, tab
delimiters and trimValues all behave correctly. Tests, ruff, typecheck and lint
are green. Here is the node on the canvas:

Screenshot 2026-08-01 at 7 41 19 AM

One thing left. The new dedupe can pick a name that already exists in the file,
and then a column is dropped:

"a,a,a_2\n1,2,3" with hasHeader true

header ['a', 'a', 'a_2']
dedupe ['a', 'a_2', 'a_2']
result [{"a": "1", "a_2": "3"}]

The middle value "2" is lost, because the generated a_2 collides with the real
a_2 column. Bumping the counter until the candidate name is free should fix it.
A test for that case would be good too. 🙏

Also worth one line in the docs: trimValues strips whitespace inside quoted
fields, so " padded " comes back as "padded". RFC 4180 treats that as data and
the docs currently claim RFC 4180 handling. The default is fine, let's just say
it explicitly.

Thanks for the quick fixes and follow ups 💯

@asimodabas

Copy link
Copy Markdown
Contributor Author

Nice catch on the dedupe collision. Fixed it so the suffix keeps bumping until the candidate is actually free it now skips both already-used names and real columns elsewhere in the header, so nothing gets dropped:

a,a,a_2\n1,2,3[{"a": "1", "a_3": "2", "a_2": "3"}]

The real a_2 keeps its value and the duplicate a becomes a_3. Added a test for that case.

Also added a docs line: trimValues strips whitespace inside quoted fields too (" padded "padded), which RFC 4180 treats as data so the note says to set trimValues: false if you want it preserved exactly.

Thanks for the careful reviews 🙏

@ckakgun ckakgun left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnarounds across all three rounds @asimodabas. Nice first
contribution, hope to see more 🥇

@ckakgun
ckakgun enabled auto-merge August 1, 2026 11:00
@ckakgun
ckakgun merged commit 26f9aca into heymrun:main Aug 1, 2026
2 checks passed
eryue0220 pushed a commit to eryue0220/heym that referenced this pull request Aug 4, 2026
The converter node landed in heymrun#429 with a minimal DSL entry that listed the
fields but not their behavior, so the assistant had no signal about when to
reach for the node or what the non-default settings do.

- Document the "\t" delimiter for tab-separated values
- Note that hasHeader:false yields arrays of cell values, not objects
- Explain what trimValues:false preserves inside quoted fields
- Note that converterColumns falls back to inferring columns from row keys
- Document the $label.conversion echo key alongside $label.result
- Add a worked csvToJson example with its resolved output
- Steer CSV work to the converter node instead of an execute or llm node,
  since RFC 4180 quoting, embedded newlines, and BOM stripping are already
  handled there

Prompt text only, no behavior change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants