Skip to content

test(amber): add unit tests for ArrowUtils#4758

Merged
Yicong-Huang merged 2 commits into
apache:mainfrom
Yicong-Huang:test-arrow-utils
May 3, 2026
Merged

test(amber): add unit tests for ArrowUtils#4758
Yicong-Huang merged 2 commits into
apache:mainfrom
Yicong-Huang:test-arrow-utils

Conversation

@Yicong-Huang
Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Adds scalatest coverage for common/workflow-core/src/main/scala/org/apache/texera/amber/util/ArrowUtils.scala. The Arrow ↔ Texera type / schema converters had no dedicated spec.

The getTexeraTuple / setTexeraTuple / appendTexeraTuple row-level helpers are intentionally out of scope for this PR — they need a real VectorSchemaRoot and substantial Arrow scaffolding, and warrant a dedicated PR.

Any related issues, documentation, discussions?

Closes #4751.

Two contract divergences are pinned in the spec with explanatory comments and filed separately as Bug issues: (1) toAttributeType's catch-all case 64 | _ => LONG collapses every non-{16,32}-bit width onto LONG (including widths like 8 that are clearly not LONG), and (2) fromAttributeType collapses STRING, LARGE_BINARY, and ANY onto the same Utf8 Arrow type — LARGE_BINARY is recovered via field metadata, but ANY loses its distinction entirely on round-trip and surfaces as STRING.

How was this PR tested?

sbt scalafmtCheckAll
sbt "WorkflowCore/testOnly org.apache.texera.amber.util.ArrowUtilsSpec"

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-4-7)

Closes apache#4751

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.20%. Comparing base (cb2f375) to head (b3206f8).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #4758      +/-   ##
============================================
+ Coverage     43.09%   43.20%   +0.10%     
  Complexity     2037     2037              
============================================
  Files           957      957              
  Lines         34077    34077              
  Branches       3753     3753              
============================================
+ Hits          14686    14723      +37     
+ Misses        18620    18581      -39     
- Partials        771      773       +2     
Flag Coverage Δ
access-control-service 28.12% <ø> (ø)
amber 41.35% <ø> (+0.24%) ⬆️
computing-unit-managing-service 0.00% <ø> (ø)
config-service 0.00% <ø> (ø)
file-service 33.24% <ø> (ø)
workflow-compiling-service 47.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a dedicated ScalaTest spec for ArrowUtils in workflow-core, covering Arrow ↔ Texera type/schema conversions (including the LARGE_BINARY metadata round-trip) and pinning two known conversion-contract divergences as documented “current behavior”.

Changes:

  • Add unit tests for toAttributeType / fromAttributeType mappings, including unsupported-type failure behavior.
  • Add unit tests for toTexeraSchema / fromTexeraSchema, including LARGE_BINARY preservation via texera_type field metadata.
  • Add a small test for the bool2int implicit conversion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Yicong-Huang Yicong-Huang enabled auto-merge (squash) May 3, 2026 03:22
@Yicong-Huang Yicong-Huang merged commit 9bcbbc0 into apache:main May 3, 2026
16 checks passed
@Yicong-Huang Yicong-Huang deleted the test-arrow-utils branch May 3, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add unit tests for ArrowUtils

4 participants