Repository navigation
b12550
Pre-releaseggml : add backend scheduler tests (#27258)
-
Adds initial scheduler stress tests
-
Adds new sleep op to simplify scheduler testing
-
Adds testcase addressing #23321
-
Adds experimental vulkan GGML_OP_SLEEP
-
Improves structure and naming of previous tests. Adds new tests,
including one which detects the nkvo-bug correctly, and validates the
proposed hotfix. -
Cleanup: Removes CUDA specific comments and nvtx-markers
-
Fix: forgot to add sleep specific test case
-
Fix oversights identified by CI
-
Drops default prints so that it behaves like other ctests. Adds -v flag
to toggle this behavior -
Fix: Early exit in CPU-only settings (cause for graviton CI fails)
-
Improves stress_test_linked_list by removing overlap with
test_chain_all_backends and test_pair_user_inputs. The latter ones test
either activations or user inputs in isolation and just once, whereas
the stress test tests all combinations multiple rounds -
Exclude ACCEL devices from new tests due to incompatibility
-
Fixes vulkan test failure in llvmpipe
-
Adds asserts, improves comments and some checks
-
Fixes nitpicks in white-space and variable naming
-
Adds comments that certain vk drivers could time OP_SLEEP incorrectly
-
Drops limitiation to CPU/GPU in test-sleep
-
Aligns OP_SLEEP guards in cuda with vk guards (input shape and
contiguity) -
Implements feedback for OP_SLEEP to work in place (no copy)
-
Apply non-code suggestions from JohannesGaesslers code review
Co-authored-by: Johannes Gäßler johannesg@5d6.de
-
Removes struct backend_supports; enables tests to return skip, instead
of only pass/fail -
Make clang-tidy happy
-
Removes sched_backend_caps, improves create_test_scheduler according to
feedback (L149) -
Removes "name" field from struct sched_check in favor of
ggml_set_name/ggml_get_name -
Drops unnecessary shader_clock.comp test and guards
-
Replaces sched_ prefixes with helper_ prefix
-
Moves test suite to new RAII types for better memory management
-
Adds sleep option to test_chain_all_backends
-
Adds sleep option to first-run arm of the y-shaped test
-
Fix switchcase order for legibility in stress_test_dag
-
Simplfies stress_test_linked_list via double for-loop
-
Adds pseudo-random input to stress_test_linked_list, and runs all
transitions both with and without CPU input (next to activation copy as
input) -
Fix RPC Proto patch due to GGML_OPS bump
-
Adds ggml-cpp.h treatment to test-sleep.cpp
-
Adds TODO pointers for subsequent additional tests.
-
Apply suggestion to ggml.h
Co-authored-by: Johannes Gäßler johannesg@5d6.de
- Apply suggestion to tests/test-backend-sched.cpp
Co-authored-by: Johannes Gäßler johannesg@5d6.de
- Apply suggestion to tests/test-backend-sched.cpp
Co-authored-by: Johannes Gäßler johannesg@5d6.de
-
Fixes node compatibility check to not take buffer locations into account.
This led to skipped tests due to incorrectly rejected ops due to wrong buffer locations in multi-GPU (which is one of the scheduler behaviors under test) -
Apply batched suggestions from gg code review
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
-
Completes callsite change of gg-suggestion
-
Switch test-sleep from std::chrono::steady_clock::now() to ggml_time_us
to avoid mac clock drift and spurious test failures.
This makes the test a bit nonsensical on CPU (same clock checks itself),
but the test still has merit on the other backends where
independent device clocks are used.
Co-authored-by: Johannes Gäßler johannesg@5d6.de
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
Website:
Attestations:
macOS/iOS:
- macOS Apple Silicon (arm64)
- macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
- macOS Intel (x64)
- iOS XCFramework
Linux:
- Ubuntu x64 (CPU)
- Ubuntu arm64 (CPU)
- Ubuntu 24.04 s390x (CPU)
- Ubuntu 26.04 s390x (CPU)
- Ubuntu 26.04 s390x (zDNN)
- Ubuntu x64 (Vulkan)
- Ubuntu arm64 (Vulkan)
- Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries
- Ubuntu x64 (CUDA 13) - CUDA 13.5 libraries
- Ubuntu arm64 (CUDA 13) - CUDA 13.5 libraries
- Ubuntu x64 (ROCm 10.0)
- Ubuntu x64 (OpenVINO)
- Ubuntu x64 (SYCL FP32)
- Ubuntu x64 (SYCL FP16)
- Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Android:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows arm64 (OpenCL Adreno)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.5 DLLs
- Windows arm64 (CUDA 13) - CUDA 13.5 DLLs
- Windows x64 (Vulkan)
- Windows arm64 (Vulkan)
- Windows x64 (OpenVINO)
- Windows x64 (SYCL)
- Windows x64 (ROCm 10.0)
openEuler:
- DISABLED
- openEuler x86 (310p)
- openEuler x86 (910b, ACL Graph)
- openEuler aarch64 (310p)
- openEuler aarch64 (910b, ACL Graph)
UI: