CPU and GPU device returns different result

[william-trustlab]

When I encode text with device = "cpu", I get slightly different results from device="gpu".

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-mpnet-base-v2")
cpu_embs = model.encode("this is a test", device="cpu")
gpu_embs = model.encode("this is a test", device="cuda:0")
cpu_embs.eq(gpu_embs) == False

From investigation, this is a result of floating point arithmetic errors (the diffs are on the order of 1e-6). Is this an issue with sentence-transformers, or with the underlying pytorch?

[nreimers]

It is normal, you can ignore it

[william-trustlab]

Would it be possible to add this to the documentation? It isn't clear that this is an effect.

FWIW, this does have a quality impact for modeling that uses these features.

[poornaprudhvigu-kore]

Ideally this minor floating point changes wouldn't impact the accuracy.