Skip to content

Commit 2ba0dd8

Browse files
committed
Add newest llama-3 benchmarks
1 parent c2bce76 commit 2ba0dd8

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

benchmarks/summary.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ Date | Device | dtype | batch size | cache length |max input length |max output
77
----| ------- | ------ |---------- | -------------|-----------------|------------------|----------------------
88
2024-04-24 | TPU v5e-8 | bfloat16 | 128 | 2048 | 1024 | 1024 | 8249
99
2024-04-24 | TPU v5e-8 | int8 | 256 | 2048 | 1024 | 1024 | 10873
10+
2024-07-29 | TPU v5e-8 | int8 | 256 | 2048 | 1024 | 1024 | 8471.54
11+
12+
**NOTE:(2024-07-29)** Looks like we have a regression in the past 3 month. We are working in fixing it.
1013

1114

1215
## Gemma - 7B

0 commit comments

Comments
 (0)