ggblas

ggblas is a library aimed to provide a simple and ergonomic access to the matrixmuliplication implemented in ggml

This library adds on top a threadpool with the physical number of cores each thread being pinned to their respective counterpart.

Usage

use ggblas::batched_sgemm;

let a = vec![1., 2., 3., 4.];
let b = vec![1., 2., 3., 4.];
let mut c = vec![0., 0., 0., 0.];

// Simple (2, 2) x (2, 2)
batched_sgemm(&a, &b, &mut c, 2, 2, 2);
assert_eq!(c, &[7., 10., 15., 22.]);

// Different shape (1, 4), (4, 1)
let mut c = vec![0.];
batched_sgemm(&a, &b, &mut c, 1, 1, 4);
assert_eq!(c, &[30.]);

// batched  (2, 2, 1), (2, 1, 2)
// batching is done implicitly
let mut c = vec![0., 0., 0., 0., 0., 0., 0., 0.];
batched_sgemm(&a, &b, &mut c, 2, 2, 1);
assert_eq!(c, &[1.0, 2.0, 2.0, 4.0, 9.0, 12.0, 12.0, 16.0]);

Performance

Current performance can be see here

Intel

i5-9300 (avx2)

test bench_ggblas_n ... bench:     469,739 ns/iter (+/- 3,111)
test bench_ggblas_t ... bench:     317,049 ns/iter (+/- 5,450)
test bench_mkl_n    ... bench:     140,561 ns/iter (+/- 1,095)
test bench_mkl_t    ... bench:     185,928 ns/iter (+/- 2,781)
# (cblas)
test bench_blas_n   ... bench:   5,955,545 ns/iter (+/- 87,172)
test bench_blas_t   ... bench:  10,153,008 ns/iter (+/- 528,645)
# (matrixmultiply+threading)
test bench_matrixmultiply_n ... bench:     869,372 ns/iter (+/- 205,883)
test bench_matrixmultiply_t ... bench:     841,705 ns/iter (+/- 12,706)

M1 (neon)

test bench_ggml_n           ... bench:     640,552 ns/iter (+/- 21,558)
test bench_ggml_t           ... bench:     270,919 ns/iter (+/- 10,761)
test bench_matrixmultiply_n ... bench:     944,152 ns/iter (+/- 38,737)
test bench_matrixmultiply_t ... bench:     809,709 ns/iter (+/- 13,350)
test bench_blas_n ... bench:      97,389 ns/iter (+/- 701)
test bench_blas_t ... bench:     628,720 ns/iter (+/- 87,855)

License: Apache-2.0