You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+7-4Lines changed: 7 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -70,6 +70,10 @@ This is very tentative.
70
70
* Add convnet examples: MNIST and CIFAR.
71
71
* Bindings to a tokenizer (e.g. _llama.cpp_).
72
72
* Transformer inference for a small open-weights model (one of GPT2, LLaMA, Gemma).
73
+
***0.6.5: shape manipulation enhancements.**
74
+
* Add concatenation to the einsum syntax (an axis that is a concatenation of two axes each from another tensor); it's a generalization of stacking tensors.
75
+
* Handle shifting and explicit padding as special cases of concatenating with a fixed index: e.g. `1^i=>i` is shifting left by 1, and `i=>1^i` is padding on the left by 1.
76
+
* Draft an academic-style paper e.g. for the OCaml Workshop.
73
77
***0.7: CPU-style performance and memory efficiency.**
74
78
* Cleanup of deprecated streams functionality.
75
79
* Migrating from the "hosted tensor" idea to always requiring a context when accessing tensors and dealing with devices directly.
@@ -82,10 +86,6 @@ This is very tentative.
82
86
* Finally from [llm.c](https://github.com/karpathy/llm.c).
83
87
* These will either require splitting a routine into multiple kernels, or implementing the megakernel approach.
84
88
* Milestone phrasing: GPU tiling and related optimizations in the polyhedral style, with heuristic syntactic metrics for now.
85
-
***0.8.1: shape understanding and manipulation enhancements.**
86
-
* Verify or rethink usefulness of dimension labels aka. dimension units, and whether to introduce axis labels.
87
-
* Add concatenation to the einsum syntax (an axis that isq a concatenation of two axes each from another tensor); it's a generalization of stacking tensors.
88
-
* An academic-style paper e.g. for the OCaml Workshop.
89
89
***0.9: Optimize performance: program search.**
90
90
* Instead of dynamic scheduling as in tinygrad, we can schedule statically by program search.
91
91
* We should also reproduce the search that tinygrad is doing. Inspiration: Halide.
@@ -95,6 +95,9 @@ This is very tentative.
95
95
* Feature completeness demonstrated by resolving / implementing a few of the $\color{green}{\text{explore}}$ issues.
96
96
* Concise syntax for transfers into the merge buffer since we know which tensor node is transferred and where to.
97
97
* Similarly to how contexts track initialization dependencies for compilation, we should also track them for execution.
98
+
***1.1: shape inference and safety enhancements.**
99
+
* Consider introducing axis labels (as opposed to dimension units).
100
+
* Consider introducing shape schemes for tensor functions.
0 commit comments