Skip to content

Pull requests: karpathy/llm.c

New pull request New
49 Open 287 Closed
49 Open 287 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

run Adam only for parameters that are available on the local device
#463 opened May 25, 2024 by ngc92 Loading…
Fix MSVC compilation error with openMP
#460 opened May 25, 2024 by avflyer Loading…
6
Added new cuda kernel for encoder forwards using three dimensional kernels
#459 opened May 25, 2024 by ChrisDryden Loading…
Add cmake and fix OMP
#452 opened May 23, 2024 by avflyer Loading…
3
move all kernels into a dedicated cuda stream
#448 opened May 22, 2024 by ngc92 • Draft
Use proper GeLU on CPU
#441 opened May 21, 2024 by jart Loading…
3
Added warpsize as a constant expr for dev/cuda files
#438 opened May 20, 2024 by lancerts Loading…
AdamW constant pre-calculation outside of kernel
#437 opened May 19, 2024 by ChrisDryden Loading…
3
only save missing bits to reconstruct fp32 master weights
#432 opened May 19, 2024 by ngc92 Loading…
weight reordering: attempt 1
#427 opened May 17, 2024 by ngc92 • Draft
NCCL only multi gpu training for slurm enabled cluster
#426 opened May 17, 2024 by chinthysl Loading…
10
vectorized gemm loading and use register to hold the intermediate value
#424 opened May 17, 2024 by patricxu Loading…
Include a matmul_backward_bias kernel based on PMPP CoarsenedSumReduction kernel in 10.15
#419 opened May 16, 2024 by lancerts Loading…
5
Make ceil_div constexpr and inline
#404 opened May 12, 2024 by lancerts Loading…
Autodetect GPU compute capability using nvidia-smi.
#388 opened May 8, 2024 by akulchik Loading…
1
4
Modified version of ademeure's fused gelu_forward kernel
#363 opened May 5, 2024 by ChrisDryden Loading…
1
Overlap gradient computation and NCCL AllReduce
#361 opened May 5, 2024 by PeterZhizhin Loading…
4
Experimenting with global instantiation for the layouts
#347 opened May 3, 2024 by ChrisDryden • Draft
GPU auto-detect capability for kernel builds
#341 opened May 3, 2024 by rosslwheeler Loading…
6
GELU Fusion with cuBLASLt (SLOWER because it only merges in FP16 mode, not BF16/FP32...)
#338 opened May 3, 2024 by ademeure • Draft
Added FlameGraphs for nsys reports and some nsys documentation
#333 opened May 2, 2024 by PeterZhizhin Loading…
2
convert all float to floatX for layernorm_forward
#319 opened May 1, 2024 by JaneIllario Loading…
Improve tanh derivative in backward gelu
#307 opened Apr 30, 2024 by akbariyeh Loading…
1
Updated adamw to use packed data types
#303 opened Apr 30, 2024 by ChrisDryden Loading…
17
Rewrite the encoder_forward float4 kernel with pack128
#302 opened Apr 30, 2024 by lancerts Loading…
ProTip! Type g i on any issue or pull request to go back to the issue listing page.