Modularity and threading

Currently there's a monolithic blob that wraps some light logic around the ogg/vorbis/theora/opus libraries, then JS that uses that and is itself relatively ogg-independent. Want to change this:

Separate emscripten modules for:

• demuxers:
• ogg demuxer + skeleton decoder
• webm/mkv demuxer
• audio decoders:
• vorbis decoder
• opus decoder
• video decoders:
• theora decoder
• vp8 decoder
• vp9 encoder

The core that runs in the main thread will load separate modules for each of the demuxer, audio decoder, and video decoder that are relevant. Thus each runs with a separate memory space, and only necessary modules are loaded. Same vorbis decoder module would be used for ogv and webm, for instance. The module isolation allows for easy conversion to Web Workers for multithreading as well.

Data flow

Input:

• [main] ArrayBuffer of raw input arrives from network
• [main] sends inputBuffer to demuxer
• [demuxer] -> copy into emscripten, process, copy out of emscripten
• [demuxer] -> sends packets back to main thread

Audio:

• [main] -> as needed, sends audio packet to [audio]
• [audio] -> copy into emscripten
• [audio] process
• [audio] copy out of emscripten
• [audio] send audio to [main]
• [main] -> place decoded audio in output buffer

Video:

• [main] -> as needed, sends video packet to [video]
• [video] -> copy into emscripten
• [video] process
• [video] -> copy out of emscripten into spare buffer
• [video] send video buffer to [main]
• [main] draw decoded video on canvas

Threading

Ideally these should be isolated worker threads, but for debugging it may be easier to start with in-process modules.

Moving data in/out of emscripten requires at least one copy in most cases; at least two to go between threads.

Moving data between threads requires either a copy or a transfer; transfers are not yet supported in IE or Edge. :(

Demux interface

• codecjs_demuxer_init(demuxer)
• codecjs_demuxer_start_seek(demuxer, targetms)
• codecjs_demuxer_start_decode(demuxer)
• codecjs_demuxer_buffer(demuxer, buffer, buflen)
• codecjs_demuxer_process(demuxer)
• codecjs_demuxer_is_metadata_ready(demuxer) : bool
• codecjs_demuxer_get_current_timestamp(demuxer) : long
• codecjs_demuxer_get_stream_count(demuxer) : int
• codecjs_demuxer_get_stream_type(demuxer, stream_index) : string
• codecjs_demuxer_get_stream_packet(demuxer, stream_index, out packet) : CodecJSPacket
• codecjs_demuxer_destroy(demuxer)