Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Min P sampler implementation [alternative to Top P/Top K] #3841

Merged
merged 25 commits into from
Oct 31, 2023
Merged

Min P sampler implementation [alternative to Top P/Top K] #3841

merged 25 commits into from
Oct 31, 2023

Commits on Oct 25, 2023

  1. cuda : prints wip

    @ggerganov
    ggerganov committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    59d1232 View commit details
    Browse the repository at this point in the history

  2. cuda : new cublas gemm branch for multi-batch quantized src0

    @ggerganov
    ggerganov committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    52af782 View commit details
    Browse the repository at this point in the history

  3. cuda : add F32 sgemm branch

    @ggerganov
    ggerganov committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    16b60dd View commit details
    Browse the repository at this point in the history

  4. cuda : fine-tune >= VOLTA params + use MMQ only for small batches

    @ggerganov
    ggerganov committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    a3c2843 View commit details
    Browse the repository at this point in the history

  5. cuda : remove duplicated cuBLAS GEMM code

    @ggerganov
    ggerganov committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    4c6744b View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. cuda : add CUDA_USE_TENSOR_CORES and GGML_CUDA_FORCE_MMQ macros

    @ggerganov
    ggerganov committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    a4e15a3 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. build : add compile option to force use of MMQ kernels

    @ggerganov
    ggerganov committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    49af767 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2023

  1. Super hacky starting implementation of Min P

    @kalomaze
    kalomaze committed Oct 28, 2023
    Configuration menu
    Copy the full SHA
    a9e2b74 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2023

  1. Transform Min P into a proper CLI option

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    a235a0d View commit details
    Browse the repository at this point in the history

  2. Min P disabled if set to 1.0 or 0, otherwise Top P

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    838d58d View commit details
    Browse the repository at this point in the history

  3. Debugging print statements removed

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    69ef4ca View commit details
    Browse the repository at this point in the history

  4. erring on the side of caution; disable by default

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    833637b View commit details
    Browse the repository at this point in the history

  5. Remove accidentally kept prints + min_keep support

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    62fc771 View commit details
    Browse the repository at this point in the history

  6. Standardize 0.0 disabling min_p upon feedback

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    49b68e8 View commit details
    Browse the repository at this point in the history

  7. Simplified counter by checking candidates size 

    + fixed 0.0 default for min_p
    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    6f7cdec View commit details
    Browse the repository at this point in the history

  8. minor whitespace fix

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    cb23358 View commit details
    Browse the repository at this point in the history

  9. Even formatting + exclusively 0.0f to disable now

    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    fcbbfc1 View commit details
    Browse the repository at this point in the history

  10. cleanup

    @cebtenzzre
    cebtenzzre committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    69e638e View commit details
    Browse the repository at this point in the history

  11. permit simultaneous use of top_p and min_p

    @cebtenzzre
    cebtenzzre committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    3ddfd67 View commit details
    Browse the repository at this point in the history

  12. Merge remote-tracking branch 'original/cuda-quantum-batch' into min-p… 

    …-sampling
    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    18c0aa7 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'min-p-sampling' of https://github.com/kalomaze/koboldcpp … 

    …into min-p-sampling
    @kalomaze
    kalomaze committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    87adfad View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. Update README & set 0.05 default

    @kalomaze
    kalomaze committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    9248325 View commit details
    Browse the repository at this point in the history

  2. added a bit more context to the README

    @kalomaze
    kalomaze committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    512cac6 View commit details
    Browse the repository at this point in the history

  3. Update README for consistency

    @kalomaze
    kalomaze committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    974640a View commit details
    Browse the repository at this point in the history

  4. forgot one small thing!

    @kalomaze
    kalomaze committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    3b58af2 View commit details
    Browse the repository at this point in the history