BUG/MAINT: Have all metrics return tensors #701
Comments
NickleDave
added a commit
that referenced
this issue
Sep 23, 2023
Makes functions in `vak.transforms.distance.functional` return tensors so we don't cause errors when lightning tries to convert from numpy to tensors to log. Letting lightning do the conversion kind of works, but it can cause a fatal error for someone using an Apple M1 with 'mps' as the accelerator, see https://forum.vocalpy.org/t/vak-tweetynet-with-an-apple-m1-max/78/4?u=nicholdav I don't find any explicit statement in either the Lightning or Torchmetrics docs that metrics should always be tensors, and that this guarantees there won't be weird issues (right now we get a warning on start-up that all logged scalars should be float32, but I would expect one should be able to log integers too?). But from various issues I read, it seems like that should be the case, Lightning-AI/pytorch-lightning#2143 and I notice that torchmetrics classes tend to do things like convert to a float tensor
NickleDave
added a commit
that referenced
this issue
Sep 23, 2023
Makes functions in `vak.transforms.distance.functional` return tensors so we don't cause errors when lightning tries to convert from numpy to tensors to log. Letting lightning do the conversion kind of works, but it can cause a fatal error for someone using an Apple M1 with 'mps' as the accelerator, see https://forum.vocalpy.org/t/vak-tweetynet-with-an-apple-m1-max/78/4?u=nicholdav I don't find any explicit statement in either the Lightning or Torchmetrics docs that metrics should always be tensors, and that this guarantees there won't be weird issues (right now we get a warning on start-up that all logged scalars should be float32, but I would expect one should be able to log integers too?). But from various issues I read, it seems like that should be the case, Lightning-AI/pytorch-lightning#2143 and I notice that torchmetrics classes tend to do things like convert to a float tensor
|
Fixed by #702 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The edit distance metrics in
vak.metrics.distancecurrently return either Pythonints ornp.float64scalars.When
modelscall thelightningmethodself.logwith these values, there's an implicit conversion to a torch tensor with a dtype of float32--we currently get a warning about this during the validation step. In most cases this is good enough but it's at the very least kind of weird to store int counts of edits as floats.What's more problematic is that this can cause a failure on some platforms, e.g. on an M1 Mac using
mpsas theaccelarator, as described in #700--because the returned segment edit distance is np.float64, so converting to tensor gives a torch.float64, that the mps backend can't deal with. We should fix thisfloat32orint32as neededThe text was updated successfully, but these errors were encountered: