Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Dockerfile for kaldi installation #2246

Merged
merged 8 commits into from Mar 3, 2018

Conversation

sih4sing5hong5
Copy link
Contributor

Sorry for slow response.
I mentioned this PR in https://groups.google.com/d/msg/kaldi-help/lke3CPycg6U/M4aciiXIAgAJ
The dockerfiles can be guides for installation, preventing from wasting time for system setting.
Like https://groups.google.com/d/msg/kaldi-help/5smFF2RW9do/j9xkLZonBgAJ

I finished the dockerfile for ubuntu.
The fedora and centos ones are WIP.

@danpovey
Copy link
Contributor

danpovey commented Mar 1, 2018

I'd prefer it go in misc/docker/ instead of docker/. Also remove the Chinese comments.

@sih4sing5hong5
Copy link
Contributor Author

Ok, no problem.

@sih4sing5hong5
Copy link
Contributor Author

sih4sing5hong5 commented Mar 1, 2018

I traced the problem in tools/ with Fedora Linux, https://groups.google.com/d/msg/kaldi-help/5smFF2RW9do/xJI0jUV7BQAJ

The docker got sctk errors instead of openfst errors.
All things worked in CentOS.

I found the problem in the sctk tool.
I am not familiar in perl.
It need help to solve this issue.

Update:
The perl version of Fedora is 5.26.1.
The perl version of CentOS is 5.16.
I upgrade the perl version to 5.26.1 in CentOS, and it still worked.

Trace path
[root@83b1d23a1905 tools]# pwd
/usr/local/kaldi/tools
[root@83b1d23a1905 tools]# make 
...
./RunTests.pl
   Running test 'test1-sastt', operation 'test', options '-G -f rttm -F rttm -a', directory 'test1-sastt.test'
      Executing command
Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/{_recursive_/_recur_{ <-- HERE _sive_/_si_ve_}_}/ at ../../md-eval/md-eval.pl line 1099, <DATA> line 12.
Error: MDEVAL failed
      Command: md-eval.pl -nafcs -c 0.25 -o  -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap 1> sastt-case1.sys.rttm.filt.mdeval at ../hubscr.pl line 679.
[ERROR] Execution failed
...

went into the sctk directory

[root@83b1d23a1905 sctk-2.4.10]# pwd
/usr/local/kaldi/tools/sctk-2.4.10
[root@83b1d23a1905 sctk-2.4.10]# make check

got the same error.

traced the makefile
[root@83b1d23a1905 sctk-2.4.10]# cd src/hubscr/    
[root@83b1d23a1905 hubscr]# pwd
/usr/local/kaldi/tools/sctk-2.4.10/src/hubscr
[root@83b1d23a1905 hubscr]# ./RunTests.pl 
   Running test 'test1-sastt', operation 'test', options '-G -f rttm -F rttm -a', directory 'test1-sastt.test'
      Executing command
Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/{_recursive_/_recur_{ <-- HERE _sive_/_si_ve_}_}/ at ../../md-eval/md-eval.pl line 1099, <DATA> line 12.
Error: MDEVAL failed
      Command: md-eval.pl -nafcs -c 0.25 -o  -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap 1> sastt-case1.sys.rttm.filt.mdeval at ../hubscr.pl line 679.
[ERROR] Execution failed

Finally found.

[root@83b1d23a1905 test1-sastt.base]# pwd
/usr/local/kaldi/tools/sctk-2.4.10/src/hubscr/test1-sastt.base
[root@83b1d23a1905 test1-sastt.base]# md-eval.pl -nafcs -c 0.25 -o  -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap 1> sastt-case1.sys.rttm.filt.mdeval
bash: md-eval.pl: command not found
[root@83b1d23a1905 test1-sastt.base]# ../../md-eval/md-eval.pl -nafcs -c 0.25 -o  -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap 1> sastt-case1.sys.rttm.filt.mdeval
Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/{_recursive_/_recur_{ <-- HERE _sive_/_si_ve_}_}/ at ../../md-eval/md-eval.pl line 1099, <DATA> line 12.

@jtrmal
Copy link
Contributor

jtrmal commented Mar 1, 2018

I'll handle the sctk error -- this is connected to the version of perl.
There is a certain behavior of perl that was declared as obsolete in perl 22 and declared as illegal in 26. However, from looking at the script, I believe the error was caused by error in logic in that script.

For now, you could install version lower than 26 to "resolve" it.
EDIT: Actually, I think it's ok to skip sctk's make check
y.


WORKDIR /usr/local/
# Use the newest kaldi version
RUN git clone https://github.com/kaldi-asr/kaldi.git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentioned this in the email thread a while back, but my problem with this style of Dockerfile is that it does not allow you to build Kaldi interactively. e.g., you can't modify in the C++ files or Makefiles. In order to do that, the only way that I know of is putting the Dockerfile at the top of this repo, but that's rather ugly, as well.

@danpovey
Copy link
Contributor

danpovey commented Mar 2, 2018

@galv, I'm not familiar with docker issues. Let me know if you think this should be merged.

@galv
Copy link
Contributor

galv commented Mar 2, 2018

@danpovey A lot of projects like CNTK and Caffe2 have Dockerfiles like this, which simply clone master from the remote. You could say it's helpful because it documents how to install dependencies and whatnot, even though it's not useful in practice for serious work.

The two reservations I have are:

  • Anyone serious about developing (e.g., for a business-related use case) should probably not be building from the HEAD of master of something outside their corporation. So a Dockerfile is demonstrating bad coding practices.
  • If you ever want to modify the C++ code, you can't use this kind of Dockerfile for development, since it pulls from a remote. This is a much hrader problem to solve in a general fashion (This is a common problem. I've asked around. My friends who use Docker in production work mostly use scripting languages for which they can simply mount a local git repo containing their code sitting on their host machine when they run their docker container for quick development. I'm sure a solution to this exists, but I don't know it.) This description making sense depends upon you knowing the difference between the build and run stages of Docker (in particular, you can't mount the host file system at build time. The Docker development team cites security concerns). Feel free to ask for clarifications. I'm in a rush as I write this.

If you're okay with those two caveats, then you can merge this.

@danpovey
Copy link
Contributor

danpovey commented Mar 3, 2018

OK, well this doesn't break anything and I think it's harmless to merge.

@danpovey danpovey merged commit bc0fcad into kaldi-asr:master Mar 3, 2018
LvHang pushed a commit to LvHang/kaldi that referenced this pull request Apr 14, 2018
Skaiste pushed a commit to Skaiste/idlak that referenced this pull request Sep 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants