Tart Clustering

[fkorotkov]

Right now Tart runs on a single machine where you can only run at most two virtual machine at a time. It will be great to provide a solution that can cluster bunch of Apple Silicon hardware together and allow to manage VMs onto them with ease.

We’ve tried to experiment with the Nomad plugin as well as a virtual kubelet for k8s for Kubernetes. In both cases we found some blockers and in general very high complexity of a setup.

We decided on building our own solution with a simple setup and tailored for running macOS VMs. We call it internally Orchard. For the initial release of Orchard we'll focus on API-only use cases with the following endpoints:

1. Schedule a VM to start (provide priority, optional startup/shutdown script with a restart policy and an optional TTL).
2. Status for a VM with logs
3. Stopping a VM

This is a tracking issue for the initial release we'll move conversation/feature requests and feedback into a separate repository and will work from there.

[barkingfoodog]

Very interested to watch this progress. And fantastic name, BTW!

[dinvlad]

Right now Tart runs on a single machine where you can only run at most two virtual machine at a time

Could you clarify, does it mean an Apple silicon Macbook can only run at most 2 macOS VMs at a time? And is this a limitation of the Apple Virtualization framework or Tart itself?

[nholloh]

As far as I know this limit was introduced by Apple through recent EULA changes. I'm not aware of any technical limitations. @fkorotkov please correct me if I'm wrong on this.

[lanycrost]

At Krisp we already using tart for our build/test automation system, and we have very simple API for scheduling VM's across machines.

Thanks for great job. I would like to take part on Orchard development, if you need any help there.

Do you have any estimations about the release of first version of Orchard?

[lanycrost]

Regarding the jenkins plugin issue: #388

I guess we can have such plugin (something like this one), after having this system.

[fkorotkov]

We finally have something you all can try! Meet Orchard: https://github.com/cirruslabs/orchard

You can easily try it locally via:

brew install cirruslabs/cli/orchard
orchard dev

This command will launch an Orchard Controller and a single Orchard Worker on your local machine so you can play around the API.

Right now Orchard can start and stop Tart virtual machines. You can provide startup and shutdown scripts. We also implemented port-forwarding and SSH/VNC access on top of it. Note that Orchard Controller is proxing all the port-forwarding connections and therefore your Orchard Workers can be under a firewall.

There is also a guide on how to deploy and actual cluster of many machines.

It might be early for production use but it's stable enough to start exploring it and see how it can fit in your setup. So please do try it out and give us feedback!