GSoC 2026: Quantized Inference Optimization on ARM

[pvarshh]

Hi @v-Golubev and @alvoron

I'm Pranav, a Computer Science student at the University of Michigan, and I'm highly interested in contributing to OpenVINO with the Optimize Quantized Model Inference Performance on ARM Devices project. This project particularly excites me because of its focus on low-level runtime optimizations.

I have a strong background in C++ and Python, and I've spent considerable time analyzing the alignment of model quantization, and now want to explore inference. I am eager to tackle ARM-specific features like NEON to reduce latency and memory footprint using my experience in parallel programming and system design.

After doing a deep dive into the project's goals for improving throughput and compilation times on ARM platforms, I have a few technical questions regarding the implementation:

• Instruction Set Prioritization: Beyond standard NEON instructions, will the scope of this project include optimizing for ARMv8.2-A Dot Product extensions (SDOT/UDOT) or the Scalable Vector Extension (SVE) for newer ARM architectures?
• Weight Packing & Layouts: To minimize the memory footprint on resource-constrained devices, are there specific custom block layouts or weight-packing schemes for 4-bit/8-bit quantization that you are looking to prioritize within the OpenVINO runtime?
• ACL Integration: Regarding the ARM Compute Library (ACL) integrations, are there identified bottlenecks in the current dispatching logic or data movement between OpenVINO and ACL that should be addressed first?
• Compilation Benchmarks: For the goal of faster model initialization, is there a particular gold standard for compilation time (e.g., a specific model size or type) that I should use as a baseline during profiling?

I have a Mac device with an ARM chip ready for development and testing, and I'm excited to help drive the adoption of quantized models on ARM platforms.

@adrianboguszewski Can you help connect me with the mentors for further discussion? Thanks in advance!

[pvarshh]

As a followup, I am looking at this first-task assignment for the GSoC prerequisite task yet its stil in draft mode.

I was hoping to get more insights and .take the task!

Looking forward to contributing : )

[alvoron]

@pvarshh
I've prepared the instructions here: https://github.com/alvoron/gsoc-2026-openvino/tree/main
Please don't hesitate to ask if you have any questions!

[alvoron]

I added a new section to README, please take a look: https://github.com/alvoron/gsoc-2026-openvino/tree/main?tab=readme-ov-file#6-start-addressing-a-technical-gap-optional