Skip to content

llama : add llm_build helper functions #3848

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Oct 31, 2023
+844 −1,409
Merged

llama : add llm_build helper functions #3848

Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
7db9c96
llama : add llm_build_norm helper function
ggerganov Oct 29, 2023
dbf836b
llama : add llm_build_ffn helper function (#3849)
ggerganov Oct 29, 2023
38728a0
llama : add llm_build_k_shift helper
ggerganov Oct 29, 2023
909d644
llama : fix offloading after recent changes
ggerganov Oct 29, 2023
3e04625
llama : add llm_build_kv_store helper
ggerganov Oct 29, 2023
5990861
llama : remove obsolete offload names
ggerganov Oct 29, 2023
31a12f3
llama : fix llm_build_k_shift to use n_head_kv instead of n_head
ggerganov Oct 29, 2023
a104abe
llama : simplify falcon Q, K, V computation
ggerganov Oct 29, 2023
c9121fd
llama : remove obsolete comments in build graphs
ggerganov Oct 29, 2023
f39e607
llama : add llm_build_kqv helper
ggerganov Oct 29, 2023
792d1a1
llama : minor
ggerganov Oct 30, 2023
a3f8001
llama : add LLAMA_OFFLOAD_DEBUG + fix starcoder offloading
ggerganov Oct 30, 2023
2926ef6
llama : fix input allocation logic
ggerganov Oct 31, 2023
6669cd8
llama : update offload functions for KQ tensors
ggerganov Oct 31, 2023
0bfdcdd
llama : normalize tensor names
ggerganov Oct 31, 2023
fc5a26a
llama : enable warning about not offloaded tensors
ggerganov Oct 31, 2023
2073347
llama : remove extra ; + deduplicate gate_b logic
ggerganov Oct 31, 2023
7923b70
llama : add llm_build_inp_embd helper
ggerganov Oct 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading