Skip to content

Add multimodal SQL helper UDFs#467

Merged
JingsongLi merged 1 commit into
apache:mainfrom
JingsongLi:codex/add-multimodal-sql-udfs
Jul 7, 2026
Merged

Add multimodal SQL helper UDFs#467
JingsongLi merged 1 commit into
apache:mainfrom
JingsongLi:codex/add-multimodal-sql-udfs

Conversation

@JingsongLi

Copy link
Copy Markdown
Contributor

Summary

Adds Python DataFusion SQL helper UDFs for BLOB-backed multimodal workflows and vector JSON bridging. This builds on the existing video frame extraction support with media metadata, thumbnail generation, and JSON-to-vector conversion helpers.

Changes

  • Register Python binding built-ins for media_info, media_thumbnail, vector_from_json, and vector_to_json.
  • Add media inspection and PNG thumbnail generation for inline BLOB bytes and resolvable BlobDescriptor inputs.
  • Add JSON vector bridge helpers that convert JSON float arrays to Arrow List<Float32> values and back to JSON strings.
  • Extend Python DataFusion tests for registration, image/video metadata, thumbnails, invalid inputs, and vector conversion.
  • Document the Python-only multimodal helper functions in SQL docs, README, and package description.

Testing

  • cargo check -p pypaimon_rust
  • cargo fmt --check
  • git diff --check
  • uv run --no-sync pytest tests/test_datafusion.py

Notes

These helper functions are registered by pypaimon_rust.datafusion.SQLContext; they are not registered by the Rust paimon_datafusion::SQLContext.

Relates to #227.

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Reviewed the multimodal SQL helper UDF implementation and documentation, and verified locally with cargo check plus targeted Python UDF tests.

@JingsongLi JingsongLi merged commit 3aab881 into apache:main Jul 7, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants