Add non-streaming ASR server implementation

[csukuangfj]

This PR tries to implement the framework proposed in k2-fsa/k2#975

The current implementation is all in Python.

The following shows a screen recording for the running results: 1 server + 4 clients.

Untitled.mov

Usage

# start the server
cd icefall/bin
./offline_server.py

# start the client(s)
./offline_client.py


# to start multiple clinets, open a new terminal, and run ./offline_client.py again

From the output of py-spy top --pid <pid-of-the-server>, we can see that a large part of the time is spent in neural network computation. I am going to write a c++ extension to do the nerual network computation, which can be run in a separate thread with the global interpretor lock being released.

The decoding part will still be in Python, we can move it to C++ if needed.

Shall we put the server stuff in a separate repo, e.g., k2-fsa/server or can I put it in csukuangfj/server?

[pzelasko]

Very cool. If it's an "official" server that is going to be supported with the rest of the project, then I vote for k2-fsa/server

[csukuangfj]

Closing it.

See https://github.com/k2-fsa/sherpa, which is dedicated to the ASR server stuff.