Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sequence mode prototype #89

Merged
merged 30 commits into from
Jun 12, 2023
Merged

Sequence mode prototype #89

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0ec5333
Sequence mode prototype
LoganDark Jun 4, 2023
26ae3e7
Move out rwkv_att_inner
LoganDark Jun 4, 2023
aff44aa
Move out more graph functions
LoganDark Jun 4, 2023
226d6ef
Print system info in sequence.c
LoganDark Jun 4, 2023
092591b
Small single-token optimizations
LoganDark Jun 4, 2023
960cc81
Add function to estimate graph work size
LoganDark Jun 4, 2023
1c91c47
Avoid allocating new sequence graph every rwkv_eval_sequence
LoganDark Jun 4, 2023
a534429
Remove sequence capability from ops that do not need it
LoganDark Jun 4, 2023
a3f3892
Add GPU offload to sequence.c benchmark
LoganDark Jun 4, 2023
f4319da
Only calculate 1 - x tensors once per layer
LoganDark Jun 6, 2023
dd8bb25
use ggml_cpy in sequence mode xx output
LoganDark Jun 6, 2023
e207194
Rename "inputs" to "state" in rwkv_eval_sequence
LoganDark Jun 6, 2023
084c191
Basic sequence mode graph caching
LoganDark Jun 6, 2023
a072a84
Revert "Only calculate 1 - x tensors once per layer"
LoganDark Jun 6, 2023
3298425
Clean up code around graph building and ggml contexts
LoganDark Jun 6, 2023
dd6f778
Remove unused parameter from rwkv_att_wkv_size
LoganDark Jun 6, 2023
c0e32a8
Fix printf integer width in rwkv_eval
LoganDark Jun 6, 2023
5857dda
Correct assert return types, whoops
LoganDark Jun 6, 2023
cab3d33
Free rwkv_context at the end of sequence.c
LoganDark Jun 6, 2023
7a9f1f4
Merge remote-tracking branch 'upstream' into sequence-mode
LoganDark Jun 7, 2023
7d03e1b
Fix typo I didn't make
LoganDark Jun 8, 2023
29165a6
Expand single-line return conditions
LoganDark Jun 8, 2023
ff405b6
Enable sanitizer in macOS workflows
LoganDark Jun 10, 2023
e5250f0
Add doc comments and expand ser->serial, seq->sequence
LoganDark Jun 10, 2023
17495b3
Adjust doc comment in rwkv.h
LoganDark Jun 10, 2023
8983823
Add thread safety note to rwkv_eval_sequence as well
LoganDark Jun 10, 2023
8523841
Remove entire rwkv.cpp source code from sequence.c
LoganDark Jun 10, 2023
f0ec611
Don't validate when sequence is NULL
LoganDark Jun 10, 2023
ff8e3d8
Fix OOM on cuBLAS-enabled quantized models
LoganDark Jun 10, 2023
bf8073a
Remove sequence.c
LoganDark Jun 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ jobs:
run: |
mkdir build
cd build
cmake -DRWKV_AVX2=OFF -DRWKV_FMA=OFF ..
cmake -DRWKV_AVX2=OFF -DRWKV_FMA=OFF -DRWKV_SANITIZE_ADDRESS=ON ..
cmake --build . --config Release

- name: Test
Expand Down
Loading