One of the key techniques we have used is NTK-aware interpolation (bloc97, 2023), which adjusts the scale to prevent the loss of high-frequency information in a training-free manner.

[eyuansu62]

Why the high-frequency information would be lost?
Maybe the low-freq?

[logicwong]

Thanks for pointing this out, we originally meant to say "which adjusts the scale to prevent the loss of high-frequency information in a training-free manner compared to position interpolation (PI)." We will modify the expression in the paper

[eyuansu62]

this would be great