Not working with GPU on Windows

[osamahothman]

I am trying to use surya on Windows with Nvidia GPU, and cuda enabled. It returns empty predictions and the logs through the following error:
Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED

[VikParuchuri]

I have only seen this as a warning with pytorch 2.3 (it doesn't stop execution). You may have an older version of CUDA that needs to be updated if it's an exception.

[osamahothman]

Yes, it is a warning, The script executes, but returns empty predictions for any given file. Only the image included in the repo gives true predictions.
This is the only warning in the logs.

[akshay6893]

It turns out that the layer -

hidden_states = self.sr(hidden_states)

where,

self.sr = nn.Conv2d(
                hidden_size, hidden_size, kernel_size=sequence_reduction_ratio, stride=sequence_reduction_ratio
            )

in the class SegformerEfficientSelfAttention was the problem.

It is a known problem with conv2d that on some architectures with float16 ( weights and inputs) it gives nan values.

So i changed float16 to float32 as shown below in settings.py -

@computed_field
@property
 def MODEL_DTYPE(self) -> torch.dtype:
      return torch.float32 if self.TORCH_DEVICE_MODEL == "cpu" else torch.float32

@computed_field
@property
def MODEL_DTYPE_DETECTION(self) -> torch.dtype:
    return torch.float32 if self.TORCH_DEVICE_DETECTION == "cpu" else torch.float32

Another error i got was -

CUDA error: an illegal memory access was encountered

which i solved by adding os.environ['TORCH_CUDNN_V8_API_DISABLED'] = "1" in my jupyter notebook cell.