New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove GPU dependency for interference? #71
Comments
+1 for this. Ensuring a GPU is available in a production environment in the cloud can be a real nuisance. Also since MMdetection 2.0 there is support for CPU-only mode. So if someone is able to reproduce or convert the model to mmdetection 2.0-compatible format, then this model can be used for inference in a CPU-only environment. The nice part is that training can still be done with GPU, but the resulting checkpoints will be able to load and run in a CPU-only environment too. See also this page on cpu-only mode and this page on upgrading from 1.x to 2.0. Unfortunately I wasn't able to succesfully convert the model myself using the provided conversion tool. Hopefully the creator could help out and provide trained models compatible with mmdetection 2.0. |
Since the creator of issue #77 mentioned he was able to convert the model (but unfortunately did not share his config or conversion steps), I decided to give it another shot myself. Succesfully this time. I would like to refer you all to my branch at iiLaurens/CascadeTabNet:mmdet2x. It includes a demo notebook on how to run using mmdetection v2.3.0 in a cpu only colab environment. You can find that notebook here. All checkpoint files can be found under the releases on this page. Happy inferencing! |
Hi @iiLaurens , thank you for sharing the workflow! https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html I had to install |
As far as I know there is no windows version for mmcv-full. And as you
noticed mmcv simply doesn't work at all. I run in Linux environment.
…On Fri, Sep 25, 2020, 12:33 Luke ***@***.***> wrote:
Hi @iiLaurens <https://github.com/iiLaurens> , thank you for sharing the
workflow!
Noticed you are using mmcv-full==1.0.5 There seems no distribution
available for the Windows platform in below link, and mmcv-full relies on
CUDA? (correct me if I am wrong).
https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html
I had to install mmcv=1.0.5, but ran into an error ModuleNotFoundError:
No module named 'mmcv._ext' Btw, do you manage to run the model on a
Windows platform with only CPU?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#71 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACLUZ5I2CILVSA5G6BEZIBTSHRWUHANCNFSM4QFKPUUA>
.
|
hi @iiLaurens you only converted the models or after converting you have trained for some epochs? |
I did not do any further training, just converting. If my memory serves me correctly, I had to convert both the model and the config file. Did you convert both?
…On Mon, Nov 2, 2020, 08:32 Kumar Rajwani ***@***.***> wrote:
hi @iiLaurens <https://github.com/iiLaurens> you only converted the
models or after converting you have trained for some epochs?
i am able to convert the model but its output is not perfect as your model.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#71 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACLUZ5OQFT7VPVVCITJHUJLSNZN65ANCNFSM4QFKPUUA>
.
|
No, i have only converted model and i am using config file of mmdetection version 2 which is compatible with the model. |
How to convert the model and the config file to mmdetection version 2 from version 1? |
i have done something like this
|
@iiLaurens thanks for this effort. |
@iiLaurens Thank you soo much for your work. only thing I changed to work on my cpu is run this instead of this |
I fine-tune/trained a model and was able to upgrade using
However in my CPU version from my checkpoint for inference I get back all empty arrays for the 81 classes. Only difference is that I started If anyone has some ideas of what to try or change would greatly appreciate it. UPDATE: In case it helps anyone who is also fine-tuning their model, I can't upgrade a model I fine tuned in mmdet 1.2 upgrade it and train w/ mmdet > 2 or infer on CPU from it. I was able to upgrade their checkpoint and train and infer on CPU (I used General Model table detection epoch_24.pth) If it is possible please let me know. |
@iiLaurens , Thank you for your time, also thanks in advance if anyone could help out with some ideas. |
I was able to get it to run from docker container (for use in AWS Lambda). This is the dockerfile:
And you need the converted checkpoint and config files that you can find in my repo. Then some code like this should make it work: from mmdet.apis import inference_detector, init_detector
config = '/pdfextract/cascadeTabNet/cascade_mask_rcnn_hrnetv2p_w32_20e.py'
checkpoint = '/pdfextract/cascadeTabNet/General.Model.table.detection.v2.pth'
model = init_detector(config, checkpoint, device='cpu')
results = inference_detector(model, img) |
Thanks so much for your suggestions. |
Thanks a lot again. I just want to let you know that I was able to build a running ubuntu container on windows thanks to your suggestion. Now, I could get the inference results without any problems on windows with just the CPU. Awesome work! |
can you please elaborate your steps |
Hi Folks I admire the work of @iiLaurens and appreciate the team. However I'm find error as below. Please I request any of you to resolve this issues it would be highly be appreciated. As I'm using colab notebook with cpu |
Hey Abhishek this is related to more your dependencies
…On Tue, 4 Jul, 2023, 11:47 Abhishek G, ***@***.***> wrote:
Hi Folks I admire the work of @iiLaurens <https://github.com/iiLaurens>
and appreciate the team. However I'm find error as below. Please I request
any of you to resolve this issues it would be highly be appreciated. As I'm
using colab notebook with cpu
ERROR: Could not find a version that satisfies the requirement
torch==1.5.1+cpu (from versions: 1.11.0, 1.11.0+cpu, 1.11.0+cu102,
1.11.0+cu113, 1.11.0+cu115, 1.11.0+rocm4.3.1, 1.11.0+rocm4.5.2, 1.12.0,
1.12.0+cpu, 1.12.0+cu102, 1.12.0+cu113, 1.12.0+cu116, 1.12.0+rocm5.0,
1.12.0+rocm5.1.1, 1.12.1, 1.12.1+cpu, 1.12.1+cu102, 1.12.1+cu113,
1.12.1+cu116, 1.12.1+rocm5.0, 1.12.1+rocm5.1.1, 1.13.0, 1.13.0+cpu,
1.13.0+cu116, 1.13.0+cu117, 1.13.0+cu117.with.pypi.cudnn, 1.13.0+rocm5.1.1,
1.13.0+rocm5.2, 1.13.1, 1.13.1+cpu, 1.13.1+cu116, 1.13.1+cu117,
1.13.1+cu117.with.pypi.cudnn, 1.13.1+rocm5.1.1, 1.13.1+rocm5.2, 2.0.0,
2.0.0+cpu, 2.0.0+cpu.cxx11.abi, 2.0.0+cu117, 2.0.0+cu117.with.pypi.cudnn,
2.0.0+cu118, 2.0.0+rocm5.3, 2.0.0+rocm5.4.2, 2.0.1, 2.0.1+cpu,
2.0.1+cpu.cxx11.abi, 2.0.1+cu117, 2.0.1+cu117.with.pypi.cudnn, 2.0.1+cu118,
2.0.1+rocm5.3, 2.0.1+rocm5.4.2)
ERROR: No matching distribution found for torch==1.5.1+cpu
—
Reply to this email directly, view it on GitHub
<#71 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMJEFL6UHR7RW5ASAA4TZ6LXOOYPDANCNFSM4QFKPUUA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Could you please elaborate my friend |
Please elaborate more |
Hi Guys,
First of all, thank you so much for sharing this amazing work. I have run the demo colab and got a good result.
To confirm, to run interference, cuda-enabled GPU is a must?
As #34 mentioned, do you consider to ease the dependency on GPU? This could make the model more scalable.
Thanks again.
Luke
The text was updated successfully, but these errors were encountered: