Matmul via f16 when possible

[EricLBuehler]

Use matmul via f16 to take advantage of the faster kernels. This is only enabled if:

• Running a prompt
• Sequence length is greater than 32 (needs to be tuned)

This extends #238 and includes a small but wide reaching refactor.

[github-actions]

Code Metrics Report

  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Dockerfile              1           34           25            0            9
 Happy                   1          442          369            0           73
 JSON                    5            9            9            0            0
 Python                 21          741          622           21           98
 TOML                   16          420          380            1           39
-------------------------------------------------------------------------------
 Jupyter Notebooks       1            0            0            0            0
 |- Markdown             1           60           30           22            8
 |- Python               1           96           87            1            8
 (Total)                            156          117           23           16
-------------------------------------------------------------------------------
 Markdown               16         1026            0          758          268
 |- BASH                 6          205          192            0           13
 |- Python               6          121          110            0           11
 |- Rust                 3          185          172            9            4
 (Total)                           1537          474          767          296
-------------------------------------------------------------------------------
 Rust                   81        26581        24455          337         1789
 |- Markdown            38          375            0          370            5
 (Total)                          26956        24455          707         1794
===============================================================================
 Total                 143        29253        25860         1117         2276
===============================================================================
  

[EricLBuehler]

Gives +60% performance improvement for prompt (750->1200)