ggerganov
Follow
Pinned
2,258 contributions in the last year
Contribution activity
July 2023
Created 194 commits in 5 repositories
Created 2 repositories
- ggerganov/llama2.c Python
- ggerganov/ggtag C
Created a pull request in ggerganov/llama.cpp that received 61 comments
llama : grouped-query attention + LLaMAv2 70B support
ref #2262
Added support for GQA. Currently, the grouping factor (8 for 70Bv2, 1 for everything else) has to be passed from the command line: -gqa 8.…
+215
−108
•
61
comments
Opened 24 other pull requests in 5 repositories
ggerganov/llama.cpp
4
open
11
merged
1
closed
- metal : fix out-of-bounds access + style changes
- GGUF
- ggml : mul mat tweaks
- ggml : sync (unary ops refactor, static-correctness)
- llama : optimize memory buffers
- ci : add 7B CUDA tests
- llama : make tensor_split ptr instead of array
- ggml : adapt Metal to new ggml_backend interface
- ci : integrate with ggml-org/ci
- ggml : broadcast mul_mat + conv batch support
- sync : ggml (cuda gelu + ggml_pool)
- ggml : revert CUDA broadcast changes from #2183
- ggml : sync (abort callback, mul / add broadcast, fix alibi)
- ci : switch threads to 1
- Test mac os ci
- ggml : sync latest (new ops, macros, refactoring)
ggerganov/ggml
5
merged
karpathy/llama2.c
1
open
ggerganov/whisper.cpp
1
open
evanmiller/llama.cpp
1
merged
Reviewed 145 pull requests in 4 repositories
ggerganov/llama.cpp
25 pull requests
- ggml : fix assert in ggml_set_unary_op
- make : build with -Wmissing-prototypes
- ggml : allocate graphs in a context
- llama : fix tokenizer
- ggml : fix ggml_flash_attn to use op_params
- support bpe tokenizer in convert
- Relax contiguous contraints in activation function
- ggml : mul mat tweaks
- Improve graph build time
- Fix line breaking error in build-info.sh
-
add
--in-prefix-bosto prefix BOS to user inputs; keep EOS - CI: Add non-AVX scalar build/test
- metal: concurrently dispatch commands
- add log_callback to llama_context_params for custom logging.
- Another speed gain for Q4_0 and Q4_1 on Metal
- make rms_norm_eps a parameter
- Fix scalar version of Q5_K when QK_K = 64
- Fix #2345, fix incorrect n_threads
- Fix op params change
- ggml: move op parameters from tensors to ggml_tensor::op_params
- Llama--Enable-pipe-friendly-help-output
-
Support
nix build '.#opencl' - Print max tensor size to stderr
- Fixes for CLBLAST compile support in FreeBSD
- Improvements to VIM plugin
- Some pull request reviews not shown.
ggerganov/ggml
25 pull requests
- GGUF file format specification
- starcoder : fix unused variable warnings
- readme: add chatglm example
- starcoder : fix windows build, fix cast warnings
- Add metal kernel to target resources when built as shared library
- Refactor unary ops in ggml
- Add get_no_alloc api corresponding to set_no_alloc
- ggml : build with -Wmissing-prototypes, fix warnings
- Make the project building with Windows
- Fix: Parameter parsing
- Add coverage measurement for Clang and increase test coverage
- Added vector scaling using Accelerate
- Fix gpt-neox docs
- CMake PPC64 support
- Sync metal compile scripts from llama.cpp
- Add the dense image positional encoding in SAM state
- Support GLM-style RoPE on CUDA
- ggml : fix mul_mat src1 indexing when src1 is not contiguous
- Batch inference
- ggml : basic implementation of 1d and 2d pools
- callback to abort ggml_graph_compute()
- Build on Windows fixed (dynamic linking of stdc++)
- Use a single kernel for cuda mul op
- tests : ifdef for #pragma GCC
- Support broadcast addition for fp32
- Some pull request reviews not shown.
ggerganov/whisper.cpp
15 pull requests
- Fix MSVC compile error C3688
- whisper : fix visibility warning of struct whisper_full_params by declaring in advance
- Fixed the issue of OpenBLAS not being enabled on Windows.
- Fix CLBlast build on MacOS
- set NVCC -arch flag by cuda version
- Go binding: Implement SetSplitOnWord
- Now make tests can be called as make tests base.en
- Fix MSVC compile error C3688 on non-unicode Windows
- README.md: Add OpenVINO support details
- Add a larger (30min) sample
- expose api to let user control log output
- chore: move progress calculation out of whisper.cpp
- OpenVINO support
- Minor: updated readme
- whisper : support speaker segmentation (local diarization) of mono audio via tinydiarize
ggerganov/ggwave
3 pull requests
Created an issue in ggerganov/llama.cpp that received 36 comments
mpi : attempt inference of 65B LLaMA on a cluster of Raspberry Pis
Now that distributed inference is supported thanks to the work of @evanmiller in #2099 it would be fun to try to utilize it for something cool. One…
36
comments
Opened 5 other issues in 3 repositories
ggerganov/llama.cpp
1
open
1
closed
ggerganov/ggml
1
open
1
closed
rgerganov/ggtag
1
open
Answered 1 discussion in 1 repository
ggerganov/llama.cpp
ggerganov/llama.cpp
-
Q: Supported quantizations for MacOS Metal ?
This contribution was made on Jul 22