Great but Very Slow with 24GB

[deepbeepmeep]

Hi,

First let me congratulate you as I think your approach is spot on: in order to have an efficient visual processing it is very likely that one should simulate how fovea is working that is one should focus on a reduced area and move around depending on what has been found.

I have tried using your demo on a RTX 4090 GB with 24GB and there isnt obviously enough memory as it is offloaded to the CPU since it takes 5 minutes to run one example.

By setting the flag 'load in 8 bits' to True the models can fit in the GPU memory however the code is obviously not compatible with bits and bytes since a few blocking exceptions are raised.

I would be grateful if you could do the required changes or simply reduce the GPU memory requirements. This would allow more people to test your great work.

[s9xie]

Please take a look at #2 #3 - load in 8 bits should work

[penghao-wu]

You can also try our online demo at https://craigwu-vstar.hf.space.

[deepbeepmeep]

I have applied patches #2 and #3 and it works great ! Many thanks