Skip to content

sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280 - #28016

Merged
Titaniumtown merged 1 commit into
ggml-org:masterfrom
philip-jingxin:sycl-fwht-kronecker
Sep 1, 2026
Merged

sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280#28016
Titaniumtown merged 1 commit into
ggml-org:masterfrom
philip-jingxin:sycl-fwht-kronecker

Conversation

@philip-jingxin

@philip-jingxin philip-jingxin commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Overview

Extends FWHT with Kronecker product to support non-power-of-two common embedding dimensions 384, 640, 768, 1280

Additional information

Changes

  • Implemented kronecker_kernel to handle base blocks of $m=12$ and $m=20$ using the Paley I matrix construction.
  • Extended operator tests in tests/test-backend-ops.cpp to cover non-power-of-two sizes for Hadamard transformation.

Testing & Verification

Environment

  • Device: Intel(R) UHD Graphics 770
  • Driver Version: 26.05.037020
  • Command: ./build_hadamard/bin/test-backend-ops test -o MUL_MAT_HADAMARD -b SYCL0

Correctness Test

Passed 16/16 verification tests, including non-power-of-two sizes (m= 384, 640, 768, 1280) across various configurations:

Backend 1/2: SYCL0
  Device description: Intel(R) UHD Graphics 770
  MUL_MAT_HADAMARD(m=384,  n=1,  k=384):  OK
  MUL_MAT_HADAMARD(m=384,  n=32, k=384):  OK
  MUL_MAT_HADAMARD(m=384,  n=4,  k=384):  OK
  MUL_MAT_HADAMARD(m=640,  n=1,  k=640):  OK
  MUL_MAT_HADAMARD(m=640,  n=32, k=640):  OK
  MUL_MAT_HADAMARD(m=768,  n=1,  k=768):  OK
  MUL_MAT_HADAMARD(m=1280, n=1,  k=1280): OK
  16/16 tests passed

Performance Benchmarks

Measured on Intel UHD Graphics 770:

Size Batch Latency (us/run) Throughput
384 1 4.20 us 70.19 GFLOPS
384 32 5.33 us 1.77 TFLOPS
640 1 16.74 us 48.95 GFLOPS
640 32 19.57 us 1.34 TFLOPS
768 1 17.23 us 68.47 GFLOPS
1280 1 72.43 us 45.24 GFLOPS

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES - Gemini was used to debug and ensure the correctness of the implementation.

@philip-jingxin
philip-jingxin requested review from a team and ggerganov as code owners August 30, 2026 10:33
@github-actions github-actions Bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Aug 30, 2026

@philip-jingxin philip-jingxin left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pr extends the FWHT to support a wider range of common embeddings. If you guys find this useful, I'll extends this to more generalized dimensions. Thanks @ggerganov

@arthw arthw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good job!
The UT cases are passed on B60 and iGPU in Ultra 7 265K.

Thank you!

@Titaniumtown
Titaniumtown merged commit 1f3d318 into ggml-org:master Sep 1, 2026
1 check passed
@Titaniumtown

Copy link
Copy Markdown
Contributor

@philip-jingxin Thank you! Very reasonable changes.

@Titaniumtown

Titaniumtown commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

I too hastilly merged this PR, I believe as it breaks github actions. @philip-jingxin can you create a follow-up PR fixing the build issues?

@philip-jingxin

Copy link
Copy Markdown
Contributor Author

I too hastilly merged this PR, I believe as it breaks github actions. @philip-jingxin can you create a follow-up PR fixing the build issues?

@Titaniumtown My bad! I've created a follow-up PR fixing the -Werror,-Wunused-variable warning and reintroducing the feature. Here you go: #28254

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants