ggmlR - CPU Tensor Operations for R

R bindings for the GGML tensor library, optimized for CPU computations. This package provides low-level tensor operations for machine learning, particularly useful for LLM inference and other deep learning tasks on CPU.

Features

• ✅ Efficient CPU tensor operations
• ✅ Support for multiple data types (F32, F16, quantized formats)
• ✅ Common neural network operations (matmul, activations, normalization)
• ✅ Computation graph building and execution
• ✅ Memory-efficient design
• ✅ No external dependencies (all C/C++ code included)

Installation

# From source
install.packages("ggmlR_0.1.0.tar.gz", repos = NULL, type = "source")

# Or using devtools
devtools::install_github("yourusername/ggmlR")

Quick Start

Basic Tensor Operations

library(ggmlR)

# Initialize context
ctx <- ggml_init(16 * 1024 * 1024)  # 16MB

# Create tensors
a <- ggml_new_tensor_1d(ctx, GGML_TYPE_F32, 10)
b <- ggml_new_tensor_1d(ctx, GGML_TYPE_F32, 10)

# Set data
ggml_set_f32(a, rnorm(10))
ggml_set_f32(b, rnorm(10))

# Perform operations
c <- ggml_add(ctx, a, b)

# Compute
graph <- ggml_build_forward_expand(ctx, c)
ggml_graph_compute(ctx, graph)

# Get results
result <- ggml_get_f32(c)

# Cleanup
ggml_free(ctx)

Matrix Multiplication

ctx <- ggml_init(16 * 1024 * 1024)

# Create matrices
A <- ggml_new_tensor_2d(ctx, GGML_TYPE_F32, 100, 200)  # 100x200
B <- ggml_new_tensor_2d(ctx, GGML_TYPE_F32, 200, 50)   # 200x50

# Initialize with random data
ggml_set_f32(A, rnorm(100 * 200))
ggml_set_f32(B, rnorm(200 * 50))

# Matrix multiplication: C = A * B (100x50)
C <- ggml_mul_mat(ctx, A, B)

# Compute
graph <- ggml_build_forward_expand(ctx, C)
ggml_graph_compute(ctx, graph)

result <- ggml_get_f32(C)

ggml_free(ctx)

Neural Network Layer

ctx <- ggml_init(128 * 1024 * 1024)

# Input
input <- ggml_new_tensor_1d(ctx, GGML_TYPE_F32, 128)
ggml_set_f32(input, rnorm(128))

# Weights and bias
W <- ggml_new_tensor_2d(ctx, GGML_TYPE_F32, 128, 256)
b <- ggml_new_tensor_1d(ctx, GGML_TYPE_F32, 256)

ggml_set_f32(W, rnorm(128 * 256, sd = 0.01))
ggml_set_f32(b, rep(0, 256))

# Forward: GELU(W * input + b)
h <- ggml_mul_mat(ctx, W, input)
h <- ggml_add(ctx, h, b)
output <- ggml_gelu(ctx, h)

# Compute
graph <- ggml_build_forward_expand(ctx, output)
ggml_graph_compute(ctx, graph)

result <- ggml_get_f32(output)

ggml_free(ctx)

Supported Operations

Tensor Creation

• ggml_new_tensor_1d() - 1D tensor (vector)
• ggml_new_tensor_2d() - 2D tensor (matrix)
• ggml_new_tensor_3d() - 3D tensor
• ggml_new_tensor_4d() - 4D tensor

Data Types

• GGML_TYPE_F32 - 32-bit float
• GGML_TYPE_F16 - 16-bit float
• GGML_TYPE_Q4_0, GGML_TYPE_Q4_1 - 4-bit quantized
• GGML_TYPE_Q5_0, GGML_TYPE_Q5_1 - 5-bit quantized
• GGML_TYPE_Q8_0, GGML_TYPE_Q8_1 - 8-bit quantized

Operations

• ggml_mul_mat() - Matrix multiplication
• ggml_add() - Element-wise addition
• ggml_mul() - Element-wise multiplication
• ggml_relu() - ReLU activation
• ggml_gelu() - GELU activation
• ggml_silu() - SiLU/Swish activation
• ggml_norm() - Layer normalization
• ggml_rms_norm() - RMS normalization

Use Cases

LLM Inference

This package is designed for running language model inference on CPU:

• Load quantized model weights
• Build transformer layers
• Run token-by-token generation
• Efficient memory usage with quantization

Stable Diffusion

Can be used for diffusion model inference:

• U-Net architecture building blocks
• Attention mechanisms
• Residual connections
• Normalization layers

Performance

Optimized for x86-64 CPUs with:

• SIMD vectorization
• Multi-threading support
• Efficient memory layout
• Cache-friendly operations

Future Plans

• GPU acceleration via separate Vulkan package (ggmlR.vulkan)
• Additional operations (softmax, attention, etc.)
• Model loading utilities
• Pre-built model examples

System Requirements

• C++17 compiler
• x86-64 CPU (ARM support planned)
• R >= 4.0.0

License

MIT License

Citation

If you use this package in your research, please cite:

@software{ggmlR,
  author = {Yuri Baramykov},
  title = {ggmlR: CPU Tensor Operations for R},
  year = {2026},
  url = {https://github.com/Zabis13/ggmlR}
}

See Also

• GGML library: https://github.com/ggerganov/ggml
• llama.cpp: https://github.com/ggerganov/llama.cpp