TCP GPU decoding

[lalimili6]

Hi
Now Kaldi support GPU decoding (#3114) is it possible to configure TCP port (#2938) on GPU?
What your opinions? Is it possible to use GPU for TCP port?
How can change TCP port script to use GPU decoding?

Best regards

[danpovey]

@hhadian perhaps you could comment on how much would be involved in such a project?
I'm wondering whether it would be necessary to allow multiple simultaneous TCP connections to get good utilization?

[galv]

I don't think it's worthwhile. The GPU decoder is designed for throughput, not latency.

…

On Sun, Aug 4, 2019 at 12:33 PM Daniel Povey ***@***.***> wrote: @hhadian <https://github.com/hhadian> perhaps you could comment on how much would be involved in such a project? I'm wondering whether it would be necessary to allow multiple simultaneous TCP connections to get good utilization? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3504?email_source=notifications&email_token=ABEL6UFSH7UNOL2BNOAVPNDQC4VG7A5CNFSM4IJEQRX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3QIFFY#issuecomment-518029975>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABEL6UCK5LC5X7XTNJPEGWTQC4VG7ANCNFSM4IJEQRXQ> .

-- Daniel Galvez http://danielgalvez.me https://github.com/galv

[hhadian]

Yes, it seems like an interesting project for GPU-based ASR servers, e.g. to replace a CPU machine (with many cores) with a single-GPU machine. Especially for asynchronous services perhaps, because I'm not sure how we can ensure real-time results.

I'll need to study #3114 to know how much work this requires. Also I'm not sure how cuda works in terms of thread-safety and access from multiple threads. Anyway I guess it depends on how much we want from this, e.g. single-thread (not much point in it IMO) or multi-thread, real-time or not, etc.

[hugovbraun]

@hhadian I'll create a new PR soon including a new low-latency high-throughput pipeline designed for online. I'd suggest not spending too much time studying the current pipeline.

[hugovbraun]

@hhadian I've created #3568 if you want to take a look

[lalimili6]

@hhadian Is there any results?