Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nomic Vulkan backend #4456

Merged
merged 155 commits into from
Jan 29, 2024
Merged

Nomic Vulkan backend #4456

merged 155 commits into from
Jan 29, 2024

Commits on Oct 5, 2023

  1. Nomic vulkan backend licensed under the Software for Open Models Lice… 

    …nse (SOM), version 1.0.
    @niansa @cebtenzzre
    niansa authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    ba15dfd View commit details
    Browse the repository at this point in the history

  2. Remove warning which fails on windows.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    48a45ea View commit details
    Browse the repository at this point in the history

  3. remove dynamic deps from kompute build 

    should no longer have new external deps other than libvulkan

```
ubuntu@ip-172-31-1-24:~/repo/gpt4all/gpt4all-backend/build$ ldd ./libllamamodel-mainline-avxonly.so
        linux-vdso.so.1 (0x00007ffcb53bb000)
        libvulkan.so.1 => /lib/x86_64-linux-gnu/libvulkan.so.1 (0x00007f239dab5000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f239d800000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f239d719000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f239da95000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f239d400000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f239dd1d000)
```
    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    8563fa0 View commit details
    Browse the repository at this point in the history

  4. Switch to a dynamic dispatch table instead of linking hard against li… 

    …bvulkan.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    45c8778 View commit details
    Browse the repository at this point in the history

  5. Completely revamp how we do object management with the vulkan backend… 

    … and

stop using so many static objects so we can tear down and bring up vulkan
on new devices in the same runtime.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    b7e2e69 View commit details
    Browse the repository at this point in the history

  6. Make kompute actually include external SDK headers when requested

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    beee572 View commit details
    Browse the repository at this point in the history

  7. Throw an exception when allocation fails for vulkan.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    68cf1df View commit details
    Browse the repository at this point in the history

  8. vulkan: disambiguate gpus with the same name

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    8bea719 View commit details
    Browse the repository at this point in the history

  9. Don't try and install kompute artifacts.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    bd5f639 View commit details
    Browse the repository at this point in the history

  10. Sync from device back to host at begin of new prompt.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    4ed25b2 View commit details
    Browse the repository at this point in the history

  11. Only use vulkan with known quant that work.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    68aca6b View commit details
    Browse the repository at this point in the history

  12. Set the singleton to nullptr here.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    addac25 View commit details
    Browse the repository at this point in the history

  13. Don't crash on available devices if we can't even create an instance.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    2c24d67 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. Support for gguf.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    1b1416d View commit details
    Browse the repository at this point in the history

  2. kompute : don't fail build because of -Warray-bounds 

    There are some warnings in debug builds that are likely to be false
positives.
    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    6b6c73a View commit details
    Browse the repository at this point in the history

  3. Upload immediately to device.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    9e4f8b4 View commit details
    Browse the repository at this point in the history

  4. Add a common boilerplate code via include and elim copy pasta

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    77135a3 View commit details
    Browse the repository at this point in the history

  5. Consolidate code for mat x vec kernels and use subgroups more extensi… 

    …vely.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    93306f1 View commit details
    Browse the repository at this point in the history

  6. Move the subgroups and printf into common.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    601905e View commit details
    Browse the repository at this point in the history

  7. Minor cleanup.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    5509f74 View commit details
    Browse the repository at this point in the history

  8. Refactor getrows to use common code and get ready for q6_k.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    4b223ec View commit details
    Browse the repository at this point in the history

  9. Add q6_k getrows and mul*vec kernel.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    f1c9bc1 View commit details
    Browse the repository at this point in the history

  10. Fix offset into the qh and now we have working vulkan accelerated for… 

    … gguff'd llama.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    06d4b21 View commit details
    Browse the repository at this point in the history

  11. Fixes for norm.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    32289aa View commit details
    Browse the repository at this point in the history

  12. Fixup the upstream CMakelists.txt so we can build just llama.cpp with… 

    … our branch.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    6ac3975 View commit details
    Browse the repository at this point in the history

  13. Change this back to be in agreement with metal and our previous softm… 

    …ax kernel.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    de589ce View commit details
    Browse the repository at this point in the history

  14. Fixes for subgroup size to bring AMD and NVIDIA inline with eachother… 

    … for all kernels.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    bc4b5ed View commit details
    Browse the repository at this point in the history

  15. kompute : only try to use Vulkan for LLaMA itself

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    24a4a59 View commit details
    Browse the repository at this point in the history

  16. kompute : remove Q6_K from list of supported quant types

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    3d850db View commit details
    Browse the repository at this point in the history

  17. f16 mv broadcasting fix (gqa fix)

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    9db90cb View commit details
    Browse the repository at this point in the history

  18. q8 mat*vec

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    ff4212d View commit details
    Browse the repository at this point in the history

  19. vulkan: implement neox mode for rope

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    020b174 View commit details
    Browse the repository at this point in the history

  20. falcon h2d + reenable vulkan

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    8564f79 View commit details
    Browse the repository at this point in the history

  21. Delete TODO now that we have q8_0.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    09d83f0 View commit details
    Browse the repository at this point in the history

  22. add mat*mat ops

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    f0cd38b View commit details
    Browse the repository at this point in the history

  23. misc vulkan cleanup 

    make pushconts consistent w/ dispatch, avoid a double free
    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    46385ee View commit details
    Browse the repository at this point in the history

  24. perf: use bigger threadgroups in mm

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    3327d84 View commit details
    Browse the repository at this point in the history

  25. use op param epsilon for norms

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    d5741c0 View commit details
    Browse the repository at this point in the history

  26. q6k mm works

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    b78a94b View commit details
    Browse the repository at this point in the history

  27. rm commented dbg print

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    4809890 View commit details
    Browse the repository at this point in the history

  28. q4_1 mat*mat

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    cd0257e View commit details
    Browse the repository at this point in the history

  29. clean up vulkan/cpu switch

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    8dc79ac View commit details
    Browse the repository at this point in the history

  30. attempted speedups

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    9bc52eb View commit details
    Browse the repository at this point in the history

  31. attempted speedups 2

    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    c1fd645 View commit details
    Browse the repository at this point in the history

  32. use mat*vec shaders for mat*mat 

    I wrote the mat*mat shaders from scratch so I understand them better but
they are currently not faster than just multiply-invoking the mat*vec
shaders, by a significant degree - so, except for f32 which needed a new
shader, revert to the m*v ones here.
    @apage43 @cebtenzzre
    apage43 authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    cc05a60 View commit details
    Browse the repository at this point in the history

  33. kompute : enable kp_logger and make it static (#8)

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    21841d3 View commit details
    Browse the repository at this point in the history

  34. kompute : make scripts executable

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    cbc0d1a View commit details
    Browse the repository at this point in the history

  35. Don't try an allocation on a heap that is smaller than the size we re… 

    …quire.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    8400015 View commit details
    Browse the repository at this point in the history

  36. Remove unused push constant that was giving validation errors.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    752f7eb View commit details
    Browse the repository at this point in the history

  37. Lower the workgroup count for some shaders by providing a loop that p… 

    …rocesses

four floats at a time.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    8d9efbf View commit details
    Browse the repository at this point in the history

  38. Fix synchronization problem for AMD Radeon with amdvlk driver or windows 

    drivers. Does not have any performance or fidelity effect on other gpu/driver
combos I've tested.

FIXES: nomic-ai/gpt4all#1507
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    74ddf0f View commit details
    Browse the repository at this point in the history

  39. vulkan : fix missing break in matmul selection (#9)

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    1c17010 View commit details
    Browse the repository at this point in the history

  40. llama : decide to disable Vulkan before loading tensors (#7)

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    89b7127 View commit details
    Browse the repository at this point in the history

  41. Scale the workgroup count down to allow correct generation for falcon… 

    … with

AMD radeon cards with lower workgroup count limit

Partially fixes #1581
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    e006d37 View commit details
    Browse the repository at this point in the history

  42. Revert the prompt processing on gpu for now. 

    Fixes issues #1580 and #1581
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    a5eb001 View commit details
    Browse the repository at this point in the history

  43. Remove this debug code.

    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    ffd0624 View commit details
    Browse the repository at this point in the history

  44. llama : fix Vulkan whitelist (#11)

    @cebtenzzre
    cebtenzzre committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    f88b198 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. kompute : fix issues with debug layers

    @cebtenzzre
    cebtenzzre committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    a8cac53 View commit details
    Browse the repository at this point in the history

  2. fix build with external fmtlib (v10) 

    Co-authored-by: ToKiNoBug <tokinobug@163.com>
    @cebtenzzre @ToKiNoBug
    cebtenzzre and ToKiNoBug committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    c438c16 View commit details
    Browse the repository at this point in the history

  3. Merge commit 'ec893798b7a2a803466cc8f063051499ec3d96f7' into HEAD

    @cebtenzzre
    cebtenzzre committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    af00cca View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2023

  1. vulkan : replace ggml_diag_mask_inf with ggml_add (custom -inf mask)

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    71565eb View commit details
    Browse the repository at this point in the history

  2. vulkan : rope n_past is now KQ_pos, f16 rope kernel

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    84f7fc4 View commit details
    Browse the repository at this point in the history

  3. vulkan : optimize workgroup sizes

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    39abedd View commit details
    Browse the repository at this point in the history

  4. Merge commit 'fcca0a700487999d52a525c96d6661e9f6a8703a' into nomic-vu… 

    …lkan
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    f194e1b View commit details
    Browse the repository at this point in the history

  5. vulkan : assert various kernel requirements

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    a934b2c View commit details
    Browse the repository at this point in the history

  6. Merge commit '469c9addef75893e6be12edda852d12e840bf064' into nomic-vu… 

    …lkan
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    2a41ba7 View commit details
    Browse the repository at this point in the history

  7. vulkan : handle ggml_scale for n%8 != 0 

    ref #3754
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    6474fc8 View commit details
    Browse the repository at this point in the history

  8. Merge commit 'e16b9fa4baa8a09c6619b116159830e898050942' into nomic-vu… 

    …lkan
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    fe26e6a View commit details
    Browse the repository at this point in the history

  9. mention skipped change

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    9c4dfd0 View commit details
    Browse the repository at this point in the history

  10. merge fixup (e16b9fa)

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    02c3309 View commit details
    Browse the repository at this point in the history

  11. Merge commit '4760e7cc0b68570d58f55e8dda469805d1759d0d~' into nomic-v… 

    …ulkan
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    1829f1d View commit details
    Browse the repository at this point in the history

  12. vulkan : implement YaRN RoPE scaling (#2268) 

    The NeoX cur_rot part is different because I'm pretty sure my original
implementation was wrong.
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    208cd52 View commit details
    Browse the repository at this point in the history

  13. Merge commit '4760e7cc0b68570d58f55e8dda469805d1759d0d' into nomic-vu… 

    …lkan
    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    23f6d51 View commit details
    Browse the repository at this point in the history

  14. vulkan : sync with "migrate to dynamic graphs"

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    a4bb9c5 View commit details
    Browse the repository at this point in the history

  15. Merge remote-tracking branch 'upstream/master' into nomic-vulkan-redo

    @cebtenzzre
    cebtenzzre committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    9ae88ba View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. relicense Vulkan backend as MIT

    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    56430c3 View commit details
    Browse the repository at this point in the history

  2. rename ggml-vulkan -> ggml-kompute

    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    3e09e12 View commit details
    Browse the repository at this point in the history

  3. separate shaders from kompute itself

    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    27631db View commit details
    Browse the repository at this point in the history

  4. Merge commit '81bc9214a389362010f7a57f4cbc30e5f83a2d28' into nomic-vu… 

    …lkan
    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    747e1ea View commit details
    Browse the repository at this point in the history

  5. kompute : fix compile warnings

    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    b906e12 View commit details
    Browse the repository at this point in the history

  6. move kompute to a submodule

    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    9af7f58 View commit details
    Browse the repository at this point in the history

  7. remove script with unclear purpose

    @cebtenzzre
    cebtenzzre committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    f7cb0a6 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. ggml : restore 'static' specifiers

    @cebtenzzre
    cebtenzzre committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    c8fd4ba View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. refactor llama.cpp modifications

    @cebtenzzre
    cebtenzzre committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    f58f581 View commit details
    Browse the repository at this point in the history

  2. vulkan : fix free of stack addr in llama_buffer

    @cebtenzzre
    cebtenzzre committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    2d2c76a View commit details
    Browse the repository at this point in the history

  3. kompute : always destroy Manager via the destructor

    @cebtenzzre
    cebtenzzre committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    8072706 View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2024

  1. kompute : fix -Wunused-private-field warnings from clang 

    Fixes nomic-ai/gpt4all#1722

(cherry picked from commit 3cd9532)
    @cebtenzzre
    cebtenzzre committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    44b1a97 View commit details
    Browse the repository at this point in the history

  2. Merge commit 'bcc0eb4591bec5ec02fad3f2bdcb1b265052ea56' into ceb/nomi… 

    …c-vulkan
    @cebtenzzre
    cebtenzzre committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    8b65f4c View commit details
    Browse the repository at this point in the history

  3. Merge commit '31f27758faf4a4bd08101a57c7ec3a473f771f86' into ceb/nomi… 

    …c-vulkan
    @cebtenzzre
    cebtenzzre committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    3959283 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2024

  1. sync xxd commands with GPT4All llama.cpp.cmake

    @cebtenzzre
    cebtenzzre committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    904c563 View commit details
    Browse the repository at this point in the history

  2. Merge commit 'd232aca5a73b290e218a2e48b91023d5e994203f' into ceb/nomi… 

    …c-vulkan
    @cebtenzzre
    cebtenzzre committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    ae6d682 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of https://github.com/ggerganov/llama.cpp into … 

    …ceb/nomic-vulkan
    @cebtenzzre
    cebtenzzre committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    3773e1a View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. Merge commit 'e7e4df031b9e29d4b55a4e0b0295187f6b213db1' into HEAD

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    7c527eb View commit details
    Browse the repository at this point in the history

  2. kompute : initial attempt at ggml-backend v2 support

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    298d6ee View commit details
    Browse the repository at this point in the history

  3. fix assertion failure

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    5f660da View commit details
    Browse the repository at this point in the history

  4. attempt to get test-backend-ops working

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    070919d View commit details
    Browse the repository at this point in the history

  5. add sanity check and fix kompute teardown order

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    cad72e1 View commit details
    Browse the repository at this point in the history

  6. kompute : ignore exceptions in ggml_vk_available_devices (#12) 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    76474a7 View commit details
    Browse the repository at this point in the history

  7. kompute : fix rope_f32 and scale ops (#5008)

    @ggerganov @cebtenzzre
    ggerganov authored and cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    d6bd471 View commit details
    Browse the repository at this point in the history

  8. clean up old backend code

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    9431026 View commit details
    Browse the repository at this point in the history

  9. actually fix this assertion

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    e9d5223 View commit details
    Browse the repository at this point in the history

  10. sync op_rope_f16 with recent op_rope_f32 changes

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    729e1a4 View commit details
    Browse the repository at this point in the history

  11. never try to evaluate an empty command buffer 

    This fixes the immediate crashes with test-backend-ops - when
evaluatating individual no-ops like OP_VIEW, it tries to submit an empty
command buffer, which crashes RADV and hangs AMDVLK.
    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    0753073 View commit details
    Browse the repository at this point in the history

  12. fix supported ops for kompute backend

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    2f6a279 View commit details
    Browse the repository at this point in the history

  13. kompute : fix ggml_add kernel (#5027)

    @ggerganov @cebtenzzre
    ggerganov authored and cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    33e8d6a View commit details
    Browse the repository at this point in the history

  14. minor cleanup

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    cb9ceff View commit details
    Browse the repository at this point in the history

  15. kompute : fix get_rows dispatch -> 4 less failures

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    0899adf View commit details
    Browse the repository at this point in the history

  16. kompute : fix op_mul kernel -> 13 less test failures

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    08e23fd View commit details
    Browse the repository at this point in the history

  17. kompute : fix more dispatch ambiguity -> 12 less failures

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    2755ae3 View commit details
    Browse the repository at this point in the history

  18. kompute : remove broken mulrow kernel -> 1 less test failure

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    0787b80 View commit details
    Browse the repository at this point in the history

  19. fix q4_0/q4_1 mmv, 65 -> 49 failures

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    1a14099 View commit details
    Browse the repository at this point in the history

  20. fix f16 mmv, 49 -> 41 failures

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    2b0f642 View commit details
    Browse the repository at this point in the history

  21. test-backend-ops : add llama test

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    2852902 View commit details
    Browse the repository at this point in the history

  22. test-backend-ops : test scale parameter of ggml_soft_max_ext

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    1450966 View commit details
    Browse the repository at this point in the history

  23. kompute : support scale parameter of softmax

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    308f279 View commit details
    Browse the repository at this point in the history

  24. test-backend-ops : test mask parameter of ggml_soft_max_ext

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    8bd38fe View commit details
    Browse the repository at this point in the history

  25. kompute : support mask parameter of softmax

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    df687b1 View commit details
    Browse the repository at this point in the history

  26. test-backend-ops : test llama with different batch sizes

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    ebb5f7e View commit details
    Browse the repository at this point in the history

  27. test-backend-ops : increase max_nmse_err so Llama passes

    @cebtenzzre
    cebtenzzre committed Jan 24, 2024
    Configuration menu
    Copy the full SHA
    ec68a96 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2024

  1. kompute : fix algorithm names

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    987335e View commit details
    Browse the repository at this point in the history

  2. kompute : fix q8_0 mmv, 41 -> 28 failures

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    f5ac635 View commit details
    Browse the repository at this point in the history

  3. test-backend-ops : add Falcon test

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    1849b85 View commit details
    Browse the repository at this point in the history

  4. test-backend-ops : test larger GELU range

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    6fc99a6 View commit details
    Browse the repository at this point in the history

  5. kompute : fix op_gelu -> Falcon is working on AMDVLK

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    38d1f0c View commit details
    Browse the repository at this point in the history

  6. test-backend-ops : restore softmax tests

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    11b3050 View commit details
    Browse the repository at this point in the history

  7. kompute : fix basic f16 get_rows, 28 -> 26 failures

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    de9fba0 View commit details
    Browse the repository at this point in the history

  8. kompute : fix basic Q6_K get_rows, 26 -> 24 failures

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    445a373 View commit details
    Browse the repository at this point in the history

  9. kompute : mark last few failing ops as unsupported

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    3fbf052 View commit details
    Browse the repository at this point in the history

  10. test-backend-ops : make Falcon test faster with a smaller model

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    3915194 View commit details
    Browse the repository at this point in the history

  11. kompute : remove unused immintrin.h #include

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    bc28704 View commit details
    Browse the repository at this point in the history

  12. kompute : fix a -Wstrict-aliasing warning

    @cebtenzzre
    cebtenzzre committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    91654ff View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2024

  1. kompute : remove unnecessary use_mmap=false

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    61a5cf8 View commit details
    Browse the repository at this point in the history

  2. llama : revert unintended whitespace change

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    e6ce5f2 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of https://github.com/ggerganov/llama.cpp into … 

    …ceb/nomic-vulkan
    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    aea8498 View commit details
    Browse the repository at this point in the history

  4. test-backend-ops : comment out Llama and Falcon tests

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    2512799 View commit details
    Browse the repository at this point in the history

  5. test-backend-ops : check all the ops in the test for support in the b… 

    …ackends
    @slaren
    slaren committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    8ca33de View commit details
    Browse the repository at this point in the history

  6. kompute : init device automatically and remove an unnecessary free

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    6af02b1 View commit details
    Browse the repository at this point in the history

  7. ggml-kompute.h : remove anything that doesn't need to be public 

    The remaining functions are either used by llama.cpp or GPT4All.
    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    2ff2d16 View commit details
    Browse the repository at this point in the history

  8. kompute : fix #includes

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    cdab404 View commit details
    Browse the repository at this point in the history

  9. op_mul_mat_mat_f32.comp : fix missing final newline

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    454baeb View commit details
    Browse the repository at this point in the history

  10. editorconfig-checker : exclude .gitmodules

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    297fde5 View commit details
    Browse the repository at this point in the history

  11. ci : initial attempt at testing Kompute backend

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    9132485 View commit details
    Browse the repository at this point in the history

  12. main : remove ggml-kompute.h #include

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    57cecad View commit details
    Browse the repository at this point in the history

  13. kompute : adapt ggml-kompute API to be compatible with C

    @cebtenzzre
    cebtenzzre committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    4b0c96a View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2024

  1. ci : attempt to fix Vulkan installer path

    @cebtenzzre
    cebtenzzre committed Jan 27, 2024
    Configuration menu
    Copy the full SHA
    e6edd44 View commit details
    Browse the repository at this point in the history

  2. ci : do not run tests for Kompute (no GPU)

    @cebtenzzre
    cebtenzzre committed Jan 27, 2024
    Configuration menu
    Copy the full SHA
    050d450 View commit details
    Browse the repository at this point in the history

  3. kompute : use llama_backend_init/llama_backend_free to manage device

    @cebtenzzre
    cebtenzzre committed Jan 27, 2024
    3 Configuration menu
    Copy the full SHA
    5304625 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2024

  1. kompute : better device management

    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    be7c055 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ggerganov/llama.cpp into … 

    …ceb/nomic-vulkan
    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    da1dc66 View commit details
    Browse the repository at this point in the history

  3. kompute : fix merge issues

    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    dc08e51 View commit details
    Browse the repository at this point in the history

  4. kompute : remove llama_load_model_from_file_internal

    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    7e11fe0 View commit details
    Browse the repository at this point in the history

  5. vulkan : correctly fix use-after-free in ggml_vk_current_device 

    The previous attempt actually broke GPU inference with the 'main'
example, which was previously working.

deviceName is a vk::ArrayWrapper1D. Be careful when we convert it to a
std::string, so we don't get null bytes at the end.
    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    b932cd7 View commit details
    Browse the repository at this point in the history

  6. minor fixup

    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    48db724 View commit details
    Browse the repository at this point in the history

  7. fix trailing whitespace

    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    1f98dff View commit details
    Browse the repository at this point in the history

  8. fix incorrect memcpy

    @cebtenzzre
    cebtenzzre committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    2998211 View commit details
    Browse the repository at this point in the history