[Tracker] General feature requests for torchao

[supriyar]

This issue tracks outstanding feature requests for torchao. If you'd like a specific feature to be added to torchao, please comment directly here.

Quantization Techniques (based on planned, new requests)

• GPTQ
• HQQ

DTypes

• fp8
• mx format

Sparsity APIs

• int8 + 2:4 sparsity
• fp8 + 2:4 sparsity

Kernels

• kernel autotuner for dynamic quant
• C++ extension for custom kernel - starting with PagedAttention ([RFC] PagedAttention Support pytorch#121465)
• CUTLASS w4a8 kernel [New Feature] CUTLASS kernels for w4a8 quantization #64

cc @cpuhrsch

[mergennachin]

GGUF conversion and kernels

• Add gguf conversion scripts using gpt-fast/model.py as source
• Add torch subclass to ao that does the quantization
• Add ggml kernel with the wrapper to make sure it executes in eager but calls the specific kernel
• Reference python implementation

cc @larryliu0820