ML systems & performance engineering · CS @ University of Toronto, 2024-2027
Interested in ML systems, inference runtimes, heterogeneous compute, performance engineering, and Apple Silicon.
coreml-compute-placement -
measuring where Core ML actually places operations on Apple Silicon, and what each choice
costs. The Neural Engine is faster than the GPU on M4 Pro and 4.7x slower on M5 Max; the
default ComputeUnit.ALL is sometimes the slowest of the available options. Includes a
cost-weighted ANE residency verifier with a CI gate, a concurrency harness, and raw
per-run data.
Merged
- DuckDB (#24539) - fixed a hash-join regression where dictionary emission changed a chain field from a pointer to an index, breaking the existing dead-end sentinel.
Open / under review
- MLX (#4035) -
mx.searchsortedas a native primitive with CPU, Metal and CUDA kernels, including cross-backend correctness and performance work. - llama.cpp (#26724) - RPC failure handling, so losing a remote compute node fails cleanly instead of aborting the coordinator process. Reproduced across two Macs over Thunderbolt.
- coremltools (#2780) - model structure and compute plan APIs aborted the host process with an uncatchable SIGABRT when given a valid path to the wrong kind of model. Now raises.
- DuckDB (#24527) - remove a redundant DISTINCT on the existence-only side of a semi join.
- workers-sdk (#15046) - a
compound SQL statement whose
ENDwas lowercase was never closed by wrangler's D1 statement splitter, silently truncating migrations. Approved, awaiting a CI run.
- Apple Silicon ML runtimes
- Metal / GPU compute
- Core ML and ANE execution
- Quantized inference
- Multi-node inference
- Compiler / runtime behavior
- Performance and correctness bugs that only appear at scale
Available for reproduction and benchmarking:
- M5 Max - 128 GB unified memory
- 2x M4 Pro Mac mini - 64 GB
- 2x M4 Pro Mac mini - 48 GB
- Multi-node Apple Silicon over Thunderbolt 5
I am particularly interested in issues that need recent Apple Silicon, large unified-memory configurations, or multiple Macs to reproduce. Several of the items above were reproductions nobody had managed before, for exactly that reason.
Earlier academic / course projects: @Erwinnnnnnn