Proposal: Remove distributed-locking (for now)

[thomastaylor312]

Hey all! Since we're wanting to move towards finalizing these interfaces, I wanted to propose that we remove the distributed locking interface for now. Below is a (roughly ordered by importance) list of reasoning behind this:

• The proposal has no wit definitions yet. Adding another thing we have to all get agreement on will just prolong the process before we can get the wasi-cloud stuff in under the preview 2/0.2 umbrella
• Although distributed locking needs are a common thing, I don't think it necessarily fits into scope for a first draft of wasi-cloud. It totally could be something we add in the future as people say they need the feature. It is also likely to be very specific depending on the system and the lock guarantees needed. Put another way, when people talk about the key items of functionality they need, I don't hear them asking for distributed locking and instead asking for stuff like blobstore and keyvalue
• Most distributed locks are done in the cloud native space via some sort of keyvalue store. I can definitely see an argument for having an interface to distinguish between keyvalue and a lock

Thoughts from others? Totally fine to open a PR, but thought it would be good to discuss first

[Mossaka]

I don't know at the moment anyone is working on wit definitions for distributed locking, so you are right about that. We know that it's important, but I agree with you for now we can drop it for wasi-cloud-core. Later we may discuss to bring it back. Would you like to open a PR to drop it?

[Mossaka]

Related to this, I would like to have wasi-observe in wasi-cloud-core.

[devigned]

I believe that distributed locks are a critical building block for distributed systems. I also agree with you that if you have a key value store with proper semantics you can relatively easily build a distributed lock.

We (mostly I) have delayed on this since I viewed it as a small surface interface. You are right to push for exclusion or movement. I will commit to opening the PR before the end of next week. If I do not, I will move to exclude the interface. WDYT?

[devigned]

WebAssembly/wasi-distributed-lock-service#3

[thomastaylor312]

I'll take a look at that and let you know what I think

[thomastaylor312]

Ok, I took some time to think some more about this and I still don't think this completely fits into wasi-cloud-core. As I thought about it, there were 2 main reasons why I don't think this fits (in order of importance):

1. Not every type of application needs this. Specifically, things that are FaaS (or FaaS-like) generally don't need a distributed lock system. When we make this a "core" requirement, runtimes that are more FaaS focused will have to stub out functionality they don't need/want.
2. There are many different guarantees needed from a distributed lock system and it would be difficult to represent all of those properly. The big thread in keyvalue is an example of how these kind of ideas come up and could likely make a distributed lock interface way more complicated. This has even popped up inside of the PR @devigned opened. It is a complex topic and each service from cloud providers as well as things like Kubernetes, Consul, Vault, and others have their own different APIs and guarantees

What I would like to propose as an alternative (because I definitely consider this a useful set of functionality to have) is to have a wasi-cloud-ext world for these types of interfaces that might not work for all types of applications (can also use it as a test bed before adding something to wasi-cloud-core. Any thoughts?

[devigned]

I agree with the FaaS or FaaS like environment will find much less value in a distributed locking interface.

Longer running applications, like a Kubernetes controller, would benefit from leader election (having a lease on a shared resource), and having a thread to periodically attempt to renew the lease.

+1 for wasi-cloud-ext.

[Mossaka]

That's a interesting question - should we strictly define wasi-cloud-core as a core set of APIs for FaaS-like applications, and wasi-cloud-ext for long-running processes? I am seeing more and more discussion towards this distinction.

[thomastaylor312]

I don't think the debate is around "core APIs for FaaS applications" as I think that is rather limiting. I think it is the core set of APIs common to a large majority of applications. Calling it "the stuff FaaS applications need" is somewhat misleading as well. I really do think of this as more of the most common of things used by 80%+ of applications

[mikkelhegn]

I like the idea of keeping the world simple, until there is broad adoption. So +1 to this proposal.

Whether wasi-cloud-ext is a good idea or not, I'm not sure. But I think it would be great to see more adoption before standardizing.