Skip to content

Update Arrow IPC section with PoC findings#41

Merged
jghoman merged 1 commit intomainfrom
update-arrow-ipc-findings
Apr 17, 2026
Merged

Update Arrow IPC section with PoC findings#41
jghoman merged 1 commit intomainfrom
update-arrow-ipc-findings

Conversation

@jghoman
Copy link
Copy Markdown
Collaborator

@jghoman jghoman commented Apr 17, 2026

Summary

  • Updates the Arrow IPC sections in WICKED_COOL_NEXT_STEPS.md with validated results from the experiment/arrow-message-payload branch
  • Corrects the original claim: there is one hop (zero-copy wrap), not two (the original text implied a memcpy between librdkafka and Arrow)
  • Documents the duckdb_arrow_stream ABI gotcha (guaranteed segfault if you pass a wrapper struct) and buffer lifetime invariant
  • Adds smoke-test results table

Test plan

  • Review rendered markdown for correctness
  • Links to experiment branch resolve

The experiment/arrow-message-payload branch validates the zero-copy claim.
Add smoke-test results, the corrected data flow (one hop, not two), and
document the duckdb_arrow_stream ABI gotcha and buffer lifetime invariant.
@jghoman jghoman merged commit 13dfe88 into main Apr 17, 2026
15 checks passed
@jghoman jghoman deleted the update-arrow-ipc-findings branch April 17, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant