Skip to content

stas00/TransformerSizing

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
__pycache__
__pycache__
 
 
cutlass
cutlass
 
 
cutlass-old
cutlass-old
 
 
megatron-proportion-flash
megatron-proportion-flash
 
 
megatron-proportion-nonflash
megatron-proportion-nonflash
 
 
megatron
megatron
 
 
miniconda3-plot
miniconda3-plot
 
 
profiles
profiles
 
 
results
results
 
 
.get_kernel.py.swo
.get_kernel.py.swo
 
 
3d_figure.ipynb
3d_figure.ipynb
 
 
BMM_output.py
BMM_output.py
 
 
Miniconda3-latest-Linux-x86_64.sh
Miniconda3-latest-Linux-x86_64.sh
 
 
b_outer_a100_test.out
b_outer_a100_test.out
 
 
benchmarks_50346.out
benchmarks_50346.out
 
 
benchmarks_51826.out
benchmarks_51826.out
 
 
benchmarks_51842.out
benchmarks_51842.out
 
 
benchmarks_51847.out
benchmarks_51847.out
 
 
benchmarks_51848.out
benchmarks_51848.out
 
 
benchmarks_51849.out
benchmarks_51849.out
 
 
benchmarks_51850.out
benchmarks_51850.out
 
 
benchmarks_51851.out
benchmarks_51851.out
 
 
bmm_figures.ipynb
bmm_figures.ipynb
 
 
bmm_flops.py
bmm_flops.py
 
 
collect_env.py
collect_env.py
 
 
compiled_cache.db
compiled_cache.db
 
 
cutlass_batch.sh
cutlass_batch.sh
 
 
cutlass_bmm.py
cutlass_bmm.py
 
 
cutlass_mlp_4h_to_h.out
cutlass_mlp_4h_to_h.out
 
 
cutlass_python_compilation_device_error.txt
cutlass_python_compilation_device_error.txt
 
 
cutlass_tiles.py
cutlass_tiles.py
 
 
cutlass_transformer_flops.py
cutlass_transformer_flops.py
 
 
cutlass_transformer_flops.sh
cutlass_transformer_flops.sh
 
 
cutlass_transformer_sweep.out
cutlass_transformer_sweep.out
 
 
embeddings.py
embeddings.py
 
 
embeddings_batch.sh
embeddings_batch.sh
 
 
get_kernel.py
get_kernel.py
 
 
hiddenSweep256.sh
hiddenSweep256.sh
 
 
large_gemms_prop_fig.png
large_gemms_prop_fig.png
 
 
layer_norm_batch.sh
layer_norm_batch.sh
 
 
layernorm_flops.py
layernorm_flops.py
 
 
megatron_wrapper.py
megatron_wrapper.py
 
 
mm_figures.ipynb
mm_figures.ipynb
 
 
mm_flops.py
mm_flops.py
 
 
nkSweep_mm_b.sh
nkSweep_mm_b.sh
 
 
printTorchVersion.py
printTorchVersion.py
 
 
print_torch_version.sh
print_torch_version.sh
 
 
read_ind_latency.ipynb
read_ind_latency.ipynb
 
 
setup.sh
setup.sh
 
 
test.py
test.py
 
 
torchVersion.txt
torchVersion.txt
 
 
torch_transformer_flops.py
torch_transformer_flops.py
 
 
torch_transformer_flops.sh
torch_transformer_flops.sh
 
 
torch_transformer_flops_copy.py
torch_transformer_flops_copy.py
 
 
torch_transformer_flops_copy_2.py
torch_transformer_flops_copy_2.py
 
 
torch_transformer_flops_copy_3.py
torch_transformer_flops_copy_3.py
 
 
torch_transformer_flops_copy_4.py
torch_transformer_flops_copy_4.py
 
 
torch_transformer_flops_copy_5.py
torch_transformer_flops_copy_5.py
 
 
torch_transformer_flops_copy_6.py
torch_transformer_flops_copy_6.py
 
 
torch_transformer_flops_copy_7.py
torch_transformer_flops_copy_7.py
 
 
torch_transformer_flops_copy_8.py
torch_transformer_flops_copy_8.py
 
 
transformer_figures.ipynb
transformer_figures.ipynb
 
 
transformer_flops_256.py
transformer_flops_256.py
 
 
transformer_flops_64.py
transformer_flops_64.py
 
 

About

No description, website, or topics provided.
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 75.7%
  • Jupyter Notebook 10.3%
  • C 6.9%
  • JavaScript 2.3%
  • Tcl 1.9%
  • Cython 0.8%
  • Other 2.1%