Welcome to quant_eval_public_corpus_tools Discussions!

[pbhappliedsystems]

Welcome to quant_eval Discussions

This repo holds the public tooling for the quant_eval Public Corpus: the dataset builder, the figure generator, JSON schemas, and the qe-corpus reader that recomputes every table from the published datasets. The evaluation harness itself is private.

Use Discussions for:
• Q&A — reading the datasets, reproducing a table, the statistics.
• Model requests — which model and precision should be measured next.
• Show and tell — your own quantization comparisons.

Bugs in the tooling go to Issues.

Whitepaper: https://doi.org/10.5281/zenodo.22948597 · Datasets: https://doi.org/10.5281/zenodo.22009419 · Discord: https://discord.gg/bbrda5mhs9

— Patrick Hill, PBH Applied Systems