Skip to content

Enhance record batch API with UTF8 and date/time support - #100

Merged
yuvimittal merged 9 commits into
arxlang:mainfrom
yogendra-17:feat/arrow-temporal-types
Aug 3, 2026
Merged

Enhance record batch API with UTF8 and date/time support#100
yuvimittal merged 9 commits into
arxlang:mainfrom
yogendra-17:feat/arrow-temporal-types

Conversation

@yogendra-17

Copy link
Copy Markdown
Contributor

Pull Request description

Draft. Extends IRX Arrow C++ record batch bridge with:

  • String columns: utf8, large_utf8.
  • Temporal columns: date32, date64, time32(s|ms), time64(us|ns), timestamp(s|ms|us|ns).

Files:

  • packages/irx/src/irx/builder/runtime/arrow/native/irx_record_batch.{h,cpp} — native builders/getters.
  • packages/irx/src/irx/builder/runtime/record_batch.py — registry.
  • packages/irx/src/irx/record_batch.py — high-level API + _date_to_int / _datetime_to_int / _time_to_int
    helpers.
  • packages/irx/tests/test_record_batch.py — tests.

Note: Overlaps with open PR #93 until it merges.

How to test these changes

  • pytest packages/irx/tests/test_record_batch.py -v
  • ...

Pull Request checklists

This PR is a:

  • bug-fix
  • new feature
  • maintenance

About this PR:

  • it includes tests.
  • the tests are executed on CI.
  • the tests generate log file(s) (path).
  • pre-commit hooks were executed locally.
  • this PR requires a project documentation update.

Author's checklist:

  • I have reviewed the changes and it contains no misspelling.
  • The code is well commented, especially in the parts that contain more
    complexity.
  • New and old tests passed locally.

Additional information

Reviewer's checklist

Copy and paste this template for your review's note:

## Reviewer's Checklist

- [ ] I managed to reproduce the problem locally from the `main` branch
- [ ] I managed to test the new changes locally
- [ ] I confirm that the issues mentioned were fixed/resolved .

@yuvimittal
yuvimittal marked this pull request as ready for review July 28, 2026 12:56
@yogendra-17
yogendra-17 force-pushed the feat/arrow-temporal-types branch from 59be155 to 3123332 Compare July 28, 2026 13:43
case IRX_COL_DATE32:
st = static_cast<arrow::Date32Builder *>(
b->builders[col].get())
->Append(static_cast<int32_t>(v));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If v overflows int32_t (e.g. a millisecond-epoch value passed by mistake) data is silently corrupted. Add a bounds check: if (v < INT32_MIN || v > INT32_MAX) return set_err("value out of int32 range", IRX_ERR_OOB); before the cast.

case IRX_COL_TIME32_MS:
st = static_cast<arrow::Time32Builder *>(
b->builders[col].get())
->Append(static_cast<int32_t>(v));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same silent narrowing issue as in append_date. Needs the same overflow guard.

@yogendra-17
yogendra-17 force-pushed the feat/arrow-temporal-types branch from 7c2773f to ecf2758 Compare August 1, 2026 13:00
@yuvimittal
yuvimittal merged commit d3390a6 into arxlang:main Aug 3, 2026
39 checks passed
@yuvimittal

Copy link
Copy Markdown
Member

@yogendra-17 , thanks for working on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants