use https://github.com/go-piv/piv-go  to interact with smart card like Yubikey.

sigstore uses it to make it easy to sign software 

https://github.com/sigstore/cosign/blob/main/pkg/cosign/pivkey/util.go

I have not tested this, but from what i can see if does not require other software to be installed to interact with a yubikey.
I think it uses the OS native drivers.