Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Correct CTC plugin compilation error #3853

Closed
wants to merge 5 commits into from
Closed

Correct CTC plugin compilation error #3853

wants to merge 5 commits into from

Conversation

ghost
Copy link

@ghost ghost commented Nov 16, 2016

Fix of "plugin/warpctc/./warpctc-inl.h:129:5: error: 'ctcComputeInfo' was not declared in this scope" according to issue #3747

@piiswrong
Copy link
Contributor

@xlvector

@xlvector
Copy link
Contributor

I have tested in my computer, it build ok.
I also run toy_ctc example, use gpu will throw following execptions:

terminate called after throwing an instance of 'std::runtime_error'
what(): Error: compute_ctc_loss, stat = execution failed

use cpu seems will not converge.

Seems I need more time to see what changes baidu had made.

@xlvector
Copy link
Contributor

@flyingmouse can you run warpctc examples in your machine?

@ghost
Copy link
Author

ghost commented Nov 17, 2016

@xlvector I have tested on my virtual machine and AWS g2 instance (both Ubuntu 14.04). In CPU mode if I change ctcComputeInfo to ctcOptions, there is no longer compilation error (otherwise the compilation doesn't pass with the error above, cf. issue #3747 ). I've also added the Warpctc bulid path to LD_LIBRARY_PATH. But when I run toy_ctc, it displays:

begin fit
2016-11-17 14:14:36,880 Start training with [cpu(0)]
[1] 45153 segmentation fault (core dumped) python toy_ctc.py

In GPU mode the execution fails after two batches:

begin fit
2016-11-17 13:16:05,084 Start training with [gpu(0)]
2016-11-17 13:16:08,529 Epoch[0] Batch [50] Speed: 709.90 samples/sec Train-Accuracy=0.000000
2016-11-17 13:16:10,503 Epoch[0] Batch [100] Speed: 810.70 samples/sec Train-Accuracy=0.000000
terminate called after throwing an instance of 'std::runtime_error'
what(): Error: compute_ctc_loss, stat = cuda memcpy or memset failed
Aborted (core dumped)

@xlvector
Copy link
Contributor

Thanks , I will try to fix this by review code changes in baidu-ctc this weekend.

@younfor
Copy link

younfor commented Nov 20, 2016

use this old git code
https://github.com/baidu-research/warp-ctc/tree/5bfb46e83bcb338a09fbf875cb0b02bc39746b36

@jinxuan777
Copy link

@xlvector ,Have you fix this? I met the same problem!
1

@xlvector
Copy link
Contributor

Seems this PR cause error baidu-research/warp-ctc#47

@xlvector
Copy link
Contributor

baidu-research/warp-ctc#46 I have also test failed with GTX 1080 in latest warpctc master. Seems the easiest fix is to change this document and let other people use my fork (an old version). @piiswrong do you think this is OK?

I will spend more time to check this PR (baidu-research/warp-ctc#47)

@lushuicongsheng
Copy link

@xlvector I use chstone/mxnet-gpu docker image and integrate latest warpctc,it seems work.So old version mxnet work well with latest warpctc?

@piiswrong
Copy link
Contributor

@xlvector So this PR doesn't fix the error?

@xlvector
Copy link
Contributor

@lushuicongsheng latest mxnet works with old warp-ctc. Can you test without docker? And what's your card? Are you run under CPU or GPU?

@piiswrong Yes, this PR does not fix problem.

@piiswrong
Copy link
Contributor

Ok. Then let's use old ctc. Could you add a comment on this line to tell people to rollback ctc?
Like ctcComputeInfo info; // please rollback ctc to xxx
Since this line breaks build people using new ctc would see it
Meanwhile post an issue to baidu repo?

@lushuicongsheng
Copy link

@xlvector I got no machine with gpu personal,so i can only use docker in centos7. The card is GeForce GTX 950. I install the latest baidu warpctc by following the readme,and change ctcComputeInfo to ctcOptions by above tip.Then it build success,and can both run under GPU and CPU.

@BrianZhu01
Copy link

when run the example, I meet the error AttributeError: 'module' object has no attribute 'WarpCTC',who tell me what's wrong?

@BobLiu20
Copy link
Contributor

BobLiu20 commented Dec 7, 2016

Up to now , it seems still not solution for this point. I following younfor's advise to use the newest of mxnet and warp-ctc reset to 5bfb46e83bcb338a09fbf875cb0b02bc39746b36 (git reset --hard ). Now I can build and run in GPU correctly (ubuntu14.04 + GTX1080 in docker.)

Additional, if you use GTX1080 the same as me , please reference to another answer by me to solve warp-ctc error in GTX1080.

看起来这里好多中国人哈哈。。。

@BobLiu20
Copy link
Contributor

BobLiu20 commented Dec 7, 2016

@BrianZhu01 Please review this guiding page. you have to enable warp-ctc in mxnet.

@piiswrong
Copy link
Contributor

piiswrong commented Dec 7, 2016

I think @BobLiu20's solution should work for now until we support updated warpctc

So we should change this line to notify users:
ctcComputeInfo info; // please use warp-ctc commit 5bfb46e (cd warp-ctc && git checkout 5bfb46e)
also leave a note in plugin/warpctc/README.md

@BrianZhu01
Copy link

@BobLiu20 ,我解决了,更换版本就好了,另外config.mk必须是mxnet的根目录,readme有点歧义,让改make/下的config.mk, 改完需要替换到根目录了,

@piiswrong
Copy link
Contributor

Should have been solved

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants