Availability for embeddings created with transformer models

[SB168]

Hi, I'm using transformer models to create my embeddings but would like to calculate the bias. Is there a way to just feed in the embeddings instead of the embedding models or to introduce a transformer model as embedding model?
Would there be a difference in the calculation of the WEAT score?
Many thanks