Better encapsulate livekit-runtime using a new Runtime trait

[1egoman]

Background

The livekit-runtime crate was introduced in ~2024 by the folks at zed so they could use livekit in zed, which wasn't using a tokio runtime (my understanding is, a completely custom runtime!). The general design patterns haven't really changed since then.

However, a few pain points that have been encountered recently touch on livekit-runtime:

1. The migration wasn't done completely, and there are quite a few tokio runtime specific calls which were missed (or introduced later inadvertently). One example I know of offhand is the tokio::fs usage in the send_file data stream method. These could be fixed piecemeal, but we'd need to extend the abstractions further to deal with a bunch of different async operations which is fairly burdensome.
2. Because of 1, today, the async and dispatcher features (what zed was using specifically) don't actually build on main properly 😞 .
3. We should be testing to ensure that all feature combinations build in ci, but in order to do this with the current design of livekit-runtime (which isn't super well encapsulated - higher level features in the livekit crate sometimes imply certain runtimes), it would be challenging. We'd have to effectively do a build for each platform x runtime x ssl configuration x any ad hoc features which is impractical.

From what I can tell looking through zed's code, they seem to have since migrated from the custom runtime approach they were using to a more typical looking tokio backed approach. It also looks like they might be using a rust-sdks fork, not actually mainline. I've done a search across all of github, and the only places that dispatcher seems to be used is by stale zed forks, so I'm fairly confident this can be dropped with little to no consequence.

Work in progress design

Update the livekit-runtime crate to instead expose a new Runtime trait, mirrored after the way the Future trait in the futures crate works:

pub type BoxFuture<T = ()> = Pin<Box<dyn Future<Output = T> + Send + 'static>>;

pub trait Runtime: Send + Sync + 'static {
    /// Spawn a detached task. Dropping the caller's handle does not cancel it.
    fn spawn_future(&self, fut: BoxFuture);

    /// A future that completes after `dur`.
    fn sleep(&self, dur: Duration) -> BoxFuture;
}

Then, a series of implementers are exposed as TokioRuntime, AsyncStdRuntime, and (newly added) SmolRuntime. Each implements Runtime - for example:

pub struct TokioRuntime;

impl Runtime for TokioRuntime {
    fn spawn_future(&self, fut: BoxFuture) {
        tokio::task::spawn(fut);
    }

    fn sleep(&self, dur: Duration) -> BoxFuture {
        Box::pin(tokio::time::sleep(dur))
    }
}

By default, TokioRuntime is used. If a user wants to change runtime, they call set_runtime before using the sdk:

livekit_runtime::set_runtime(livekit_runtime::SmolRuntime)?;

This also leaves the door open to custom runtimes still via implementing Runtime. In addition, for extra behaviors (for example, filesystem logic), there would be relevant extension traits - for example:

pub trait RuntimeFileSystem: Runtime {
    /// Length of the file at `path`, in bytes.
    fn file_len(&self, path: PathBuf) -> BoxFuture<io::Result<u64>>;

    /// Open `path` for streaming reads.
    fn open_read(&self, path: PathBuf) -> BoxFuture<io::Result<BoxedReader>>;

    /// Create or truncate `path` and open it for streaming writes.
    fn create_write(&self, path: PathBuf) -> BoxFuture<io::Result<BoxedWriter>>;
}

Open questions

• Given dispatcher is not not being used by zed and both async / dispatcher don't compile on main, would it be better to drop livekit-runtime all together and go back to this crate being tokio only?

TODO

• Run this by the zed folks and confirm dispatcher is no longer used
• Finish porting over all unwrapped tokio uses to livekit_runtime.

[1egoman]

Closing in favor of #1375