Skip to content

Pull requests: llm-jp/scripts

New pull request New
20 Open 68 Closed
20 Open 68 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

add NUM_NODES variable to converter launcher
#93 opened Aug 19, 2025 by odashi Loading…
Refine merge script
#92 opened Aug 19, 2025 by odashi Loading…
3
ckpt conversion on cpu
#91 opened Aug 10, 2025 by cr-liu Loading…
3
[WIP] Edit installer script for midtraining
#90 opened Aug 5, 2025 by koshieguchi Loading…
[WIP] Midtraining w/ v4 tokenizer
#89 opened Aug 3, 2025 by koshieguchi Loading…
Scripts for tokenizer test, exp 0138
#83 opened May 24, 2025 by cr-liu Loading…
update setup script name from setup_mdx.sh to setup.sh
#77 opened Apr 14, 2025 by tatsuropfgt Loading…
Add scripts to run minhash deduplication for Japanese corpus
#71 opened Feb 25, 2025 by ytivy Loading…
@ytivy
Add scripts to preprocess ja-kokkai-giji
#67 opened Jan 27, 2025 by hkiyomaru Loading…
Add scripts to preprocess WARP-pdf
#64 opened Jan 16, 2025 by hkiyomaru Loading…
Add scripts of high quality cpt experiments
#62 opened Dec 2, 2024 by ytivy Loading…
Add converter script for 13B CPT
#59 opened Oct 26, 2024 by odashi Loading…
Add installer of model pre-taing with Megatron-LM for llm-jp cluster
#57 opened Oct 25, 2024 by ytivy • Draft
Add hf2megatron converter
#55 opened Oct 24, 2024 by ytivy • Draft
2
Add moe pretrain
#49 opened Oct 14, 2024 by Taishi-N324 Loading…
Cpt lr scheduling
#40 opened Sep 13, 2024 by Taka008 • Draft
Add scripts for FP8 behavior check
#37 opened Sep 11, 2024 by odashi Loading…
add training scripts for v3-1.7b-exp2-cpt-2epoch
#36 opened Sep 9, 2024 by Taka008 Loading…
Add scripts to automate evaluation on llm-jp-eval v1.3.1
#24 opened Aug 23, 2024 by hkiyomaru • Draft
4
Fix Fused Attention Error
#15 opened Aug 7, 2024 by k141303 Loading…
3
ProTip! Mix and match filters to narrow down what you’re looking for.