Skip to content

b11856

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 24 Oct 15:06
3642238

tests : add speculative rejection draft sampling test (#30213)

  • tests : add speculative rejection draft sampling test

This commit adds a test for speculative rejection sampling test that
excercises the speculative rejection sampling mechanism in
common/speculative.cpp.

The motivation for this is have a test that verifies that the
speculative draft models emitts tokens that match the target models
distribution. So when rejection sampling occurs it should still
produce the same probability distribution as if we directly sampled
from the target model.

Running the test:

$ ctest --test-dir build -R "^test-speculative-rejection-sampling$" \
    --n_trials 5000 -V

This will generate two models, one target and one draft model.

For subsequent runs use the "fixture exclude any" option to avoid
recreating the models:

$ ctest --test-dir build-cpu-debug -R \
"^test-speculative-rejection-sampling$" \
    -FA "^speculative-models$" --n_trails 5000 -V
  • remove draft model

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: