At the moment packet.Signature.Sign supports only concrete implementations in priv.PrivateKey (like rsa.PrivateKey etc). From the stdlib docs crypto.Signer seems exactly intended to help cover cases for signing where the implementation is external, could come using a hardware module.
Wondering if a patch that switched the relevant signing code paths in Signature.Sign to work (optionally) with priv.PrivateKey implementing just crypto.Signer could be accepted? It would be fairly localized, the code would of course maintain the same shape switching over priv.PubKeyAlgo, and considering crypto.Signer only for RSA and ECDSA whose implementations expose support for that in the stdlib itself.