Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration

Zhongzhi Yu^1,*, Zheng Wang^1,*, Yonggan Fu¹, Huihong Shi¹, Khalid Shaikh¹, Yingyan (Celine) Lin¹

¹ Georgia Institute of Technology

Accepted by ICML 2024

More info: [Paper]

The code is coming soon, stay tuned!