- name: Setup pandoc

uses: pandoc/actions/setup@main

with:

version: 'latest'

- name: Migration guide

run: pandoc --toc -s --embed-resources --css=docs/html/style.css docs/migration_guide.md -o docs/html/migration_guide.html

- name: Syntax extensions

run: pandoc --toc -s --embed-resources --css=docs/html/style.css docs/syntax_extensions.md -o docs/html/syntax_extensions.html

- Prefer small, composable functions; avoid needless global state. PPX usage (`%op`, `%cd`) is described in `docs/syntax_extensions.md`.

- See `docs/syntax_extensions.md` for comprehensive documentation

3. Read the syntax extensions documentation [docs/syntax_extensions.md](docs/syntax_extensions.md).

4. Read the introductory part of the shape inference documentation [docs/shape_inference.md](docs/shape_inference.md).

7. Read [docs/anatomy_of_a_backend.md](arrayjit/lib/anatomy_of_a_backend.md).

1. Shape inference details [docs/shape_inference.md](docs/shape_inference.md).

2. Backend-independent optimizations [docs/lowering_and_inlining.md](arrayjit/lib/lowering_and_inlining.md) -- _lowering_ means translating (compiling) from the high-level representation (as assignments) to the low-level representation.

Let's write a user-centered introduction to how shapes work in OCANNL. Let's put the slides in docs/slides-shapes_and_einsum.md , and write them using slipshow navigation metadata as described in docs/CLAUDE.md . The slides should take a user from a beginner to advanced in making full use of shape inference and generalized einsum notation when building neural network models. They should end up aware of how projections work, how to lean on shape inference or row variables / ellipsis notation to not commit to dimension sizes or for example the number of batch axes unnecessarily. They should learn when to use the dedicated einsum operators `++` and `+*` (these operators are translated by syntax extensions to `einsum1` and `einsum`). They should be able to use what they learned to construct a max pooling layer operation, and any other challenges they encounter in NN modeling. Consider these sources of information: files docs/syntax_extensions.md , docs/shape_inference.md , docs/shape.mli , selected parts of lib/operation.ml , selected parts of docs/slides-basics_backprop_training_codegen.md . Let me also provide some points that might not be stated sufficiently explicitly in other documentation. (1) The split of axes into kinds does not enforce semantics, because the generalized einsum notation can make aribtrary use of the axes. However, it offers expressivity gains: (a) outside of einsum spec, there is a shape logic specification with syntax `~logic:"@"`, where all input axes of the first tensor are reduced with all output axes of the second tensor, generalizing matrix multiplication to tensor multiplication -- with einsum spec, any two kinds of axes can be picked to reduce together, but it would not be possible without having distinct kinds; (b) having multiple kinds, thus opportunity for multiple row variables per tensor, allows more patterns of reorganizing and reducing axes, while being agnostic to the total number of axes -- for example, one could build code for a multihead-attention tranformer, that is agnostic whether one uses one batch axis or two batch+microbatch axes, and simultaneously is agnostic whether one uses one single-axis regular 1D attention or two-axes 2D axial attention, while handling the head-number axis as needed. (2) It's important to stress the syntactic difference with NumPy: since we use `->` to separate input and output axes, it cannot mean separating the argument tensor(s) from the result tensor -- thus `=>` is used to the left of the result tensor. (3) Remember to use kind separators where you intend to use the distinct axis kinds, e.g use `|` after batch axes . (4) To trigger multichar mode there must be a comma in the spec, it can be a trailing comma e.g. "input->output, => output->input" . (5) A reminder that, as defined in lib/operation.ml , `*` stands for tensor multiplication and `*.` stands for pointwise multiplication when working with tensor expressions (rather than low-level assignments in the `%cd` syntax). (7) The user can define operations analogous to the `einsum1` and `einsum` operation in lib/operation.ml , for example with the max operator as theaccumulation operator -- this is not so scary, operations can be easily added by users even if not inside lib/operation.ml .

{

name: 'migration_guide.html',

title: 'Migration Guide',

description: 'Migration Guide: PyTorch/TensorFlow to OCANNL'

},

{

name: 'syntax_extensions.html',

title: 'Syntax Extensions',

description: 'Syntax Extensions: %op and %cd'

},

body {

max-width: 1000px;

margin: 0 auto;

padding: 20px;

}

|-----|------|----|

| `x.view(-1, d)` or `x.reshape(-1, d)` | Not supported yet | Use shape inference and let tensors have the shape they want |

| `x.flatten()` | Not supported yet | Future syntax might be: `"x,y => x&y"` |

| `F.softmax(x, dim=-1)` | `softmax ~spec:"... | ... -> ... d" () x` | Specify axes explicitly |

| `x.mean(dim=[1,2])` | `x ++ "... | h, w, c => ... | 0, 0, c" ["h"; "w"] /. (dim h *. dim w)` | Sum then divide |

| `x.sum(dim=-1)` | `x ++ "... | ... d => ... | 0"` | Reduce by summing |

- Trailing commas ignored

- Makes convolution syntax less confusing: `stride*out+kernel`

|--------|------------------|-------------------|-------------------|

| Attention scores | `torch.einsum('bqhd,bkhd->bhqk', q, k)` | `q +* "bq|hd; bk|hd => b|qk->h" k` | `q +* "b, q | h, d; b, k | h, d => b | q, k -> h" k` |

| Convolution | N/A | better use multi-char | `x +* "... | stride*oh+kh, stride*ow+kw, ic; kh, kw, ic -> oc => ... | oh, ow, oc" kernel` |

- [Shape Inference Documentation](../dev/neural_nets_lib/Ocannl/Shape/index.html) - Detailed einsum notation spec

- [Syntax Extensions Guide](syntax_extensions.html) - `%op` and `%cd` details

- [Neural Network Blocks](https://github.com/ahrefs/ocannl/blob/master/lib/nn_blocks.ml) - Example implementations