Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VM integration #346

Merged
merged 1 commit into from
Mar 3, 2023
Merged

Add VM integration #346

merged 1 commit into from
Mar 3, 2023

Conversation

swalkinshaw
Copy link
Member

@swalkinshaw swalkinshaw commented Jan 7, 2023

Adds full integration for managing virtual machines in development via https://github.com/lima-vm/lima as an alternative to Vagrant. Lima stands for "Linux virtual machines (on macOS, in most cases)" and that's exactly our use case.

Lima isn't just a replacement for Vagrant, it also replaces the VM provider like VirtualBox or Parallels too. Lima supports two ways of running guest machines:

  • QEMU
  • macOS Virtualization.Framework ("vz")

For this initial integration, we're only supporting the macOS Virtualization.Framework because it offers near-native performance, and that includes file syncing via virtiofs.

For macOS Ventura (13.0+) users, we'll eventually recommend using trellis-cli's VM feature over Vagrant as the default since it's easier and faster. We'll look into expanding support for Linux users as well depending on performance of the file mounts on the VM.

Requirements:

  • Intel or Apple Silicon
  • macOS 13 (Ventura)
  • Lima >= 0.15

Usage:
There's 5 new commands:

  • trellis vm start
  • trellis vm stop
  • trellis vm delete
  • trellis vm shell
  • trellis vm sudoers

Under the hood, those commands wrap equivalent limactl features. Just like the previous Vagrant integration, you can always run limactl directly to manage your VMs.

For default use cases, trellis vm start can be run without any customization first. It will create a new virtual machine (using Lima) from a generated config file (project/trellis/.trellis/lima/config/<name>.yml). The site's local_path will be automatically mounted on the VM and your /etc/hosts file will be updated.

Note: run trellis vm sudoers -h to make /etc/hosts file passwordless:

$ trellis vm sudoers | sudo tee /etc/sudoers.d/trellis

Configuration:
Right now configuration options for the VM are limited. The intention is to the most common use cases without any configuration needed.

A trellis-cli config file (global or project level) supports a new vm option. The only useful config option right now is ubuntu (to switch between Ubuntu 20.04 and 22.04 for example).

Here's an example of specifying Jammy 22.04:

vm:
  ubuntu: 22.04

@swalkinshaw swalkinshaw marked this pull request as ready for review January 7, 2023 02:24
@swalkinshaw swalkinshaw force-pushed the add-vm-integration branch 5 times, most recently from 85ad3a5 to eef4b8e Compare January 15, 2023 21:24
Adds full integration for managing virtual machines in development via https://github.com/lima-vm/lima as an alternative to Vagrant. Lima stands for "Linux virtual machines (on macOS, in most cases)" and that's exactly our use case.

Lima isn't just a replacement for Vagrant, it also replaces the VM provider like VirtualBox or Parallels too. Lima supports two ways of running guest machines:

* [QEMU](https://www.qemu.org/)
* macOS Virtualization.Framework ("vz")

For this initial integration, we're only supporting the macOS Virtualization.Framework because it offers near-native performance, and that includes file syncing via `virtiofs`.

For macOS Ventura (13.0+) users, we'll eventually recommend using trellis-cli's VM feature over Vagrant as the default since it's easier and faster. We'll look into expanding support for Linux users as well depending on performance of the file mounts on the VM.

Requirements:
* Intel or Apple Silicon
* macOS 13 (Ventura)
* Lima >= 0.14

Usage:
There's 5 new commands:

* `trellis vm start`
* `trellis vm stop`
* `trellis vm delete`
* `trellis vm shell`
* `trellis vm sudoers`

Under the hood, those commands wrap equivalent `limactl` features. Just like the previous Vagrant integration, you can always run `limactl` directly to manage your VMs.

For default use cases, `trellis vm start` can be run without any customization first. It will create a new virtual machine (using Lima) from a generated config file (`project/trellis/.trellis/lima/config/<name>.yml`). The site's `local_path` will be automatically mounted on the VM and your `/etc/hosts` file will be updated.

Note: run `trellis vm sudoers -h` to make `/etc/hosts` file passwordless:
```bash
$ trellis sudoers | sudo tee /etc/sudoers.d/trellis
```

Configuration:
Right now configuration options for the VM are limited. The intention is to the most common use cases without any configuration needed.

A trellis-cli config file (global or project level) supports a new `vm` option. The only useful config option right now is `images` (to switch between Ubuntu 20.04 and 22.04 for example). 20.04.

Here's an example of specifying Jammy 22.04 for ARM (`aarch64`) only, since that's what I use):

```yml
vm:
  images:
    - location: https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-arm64.img
      arch: aarch64
```
@swalkinshaw
Copy link
Member Author

https://github.com/lima-vm/lima/releases/tag/v0.15.0 is finally out so I'm going to merge this now.

Note: this is still under "experimental" status until we get more people testing it out.

@swalkinshaw swalkinshaw merged commit a0f6468 into master Mar 3, 2023
@swalkinshaw swalkinshaw deleted the add-vm-integration branch March 3, 2023 04:29
@artshostak
Copy link

artshostak commented Mar 9, 2023

Great improvement, congrats @swalkinshaw!

Screenshot 2023-03-09 at 10 32 38 AM

Following the guide above now to try this out, but quick note the command above should be:
trellis vm sudoers | sudo tee /etc/sudoers.d/trellis

@artshostak
Copy link

artshostak commented Mar 9, 2023

Just logging my experience here in case it helps someone else trying this out or aids the project with the documentation:

[1] I have trellis-cli-dev installed instead of trellis-cli — so my first move was to replace it:
brew uninstall roots/tap/trellis-cli-dev
brew install roots/tap/trellis-cli
This required a library "gcc" which can be installed with brew install gcc in full or apparently is quicker as part of XCode Command Line Tools, which can be installed with xcode-select --install

[2] Once I have trellis-cli version 1.10.0, I noticed my Ventura 13.1 didn't have Lima, so I had to install that as well:
brew install lima

[3] Created a new project with trellis new {project}

[4] The part that wasn't clear here is the next step to take, do I run trellis up the way I would to setup, or do I setup VM with trellis vm start but it looks like the latter proceeded to do first-time provisioning of the local development environment and created a lima folder in trellis/.trellis as anticipated. Was up and running in 7 minutes, definitely an improvement over the Parallels setup!

[5] Then gave it access to /etc/hosts with:
trellis vm sudoers | sudo tee /etc/sudoers.d/trellis

Results:
✅ Created new project, provisioned development
✅ Provisioned a product environment and deployed to it
❌ Not able to access the shell in development with trellis vm shell produces the following error:

Running command => limactl shell {project}
exit status 255

Questions:

  • what happens if I have a project using Parallels and want to switch it over to VM midway for local development, or do you not recommend that? Just wondering how to tell trellis to use vm over vagrant?
  • if I wanted to specify my Ubuntu installation (20.04 instead of 22.04), when should I have done this prior to on a project level for trellis-cli config with the following:
vm:
  ubuntu: 20.04

Issues:

  • A good amount of the time while developing overtop of this on the latest Sage 10 theme, bud.watch would miss changes and would fall behind, especially for blade template changes (which I would typically vagrant ssh and wp acorn view:clear on the dev server to address, but since I was not able to access the shell this became a little frustrating).
  • Multiple times the server just hangs up with ERR_ADDRESS_UNREACHABLE which requires a vm stop/start
  • A lot of the times I would run into this error when refreshing (which would eventually go away): Undefined array key "app" with the following exception:
    image
  • Finally when updating the database, received this error:
    Screenshot 2023-03-10 at 9 49 52 AM
  • Which then looks to have taken the vm offline:
    image
  • Noticed the /etc/hosts file was missing the entry and re-added, which got me back up and running for the time being.

While it's blazing fast, it's unfortunately not so stable to do long sessions of local development with. Hope that helps and thanks again!

@jkananen
Copy link

jkananen commented Mar 12, 2023

  • what happens if I have a project using Parallels and want to switch it over to VM midway for local development

I didn't get this right on the first go and had to start over a few times, but I think the necessary steps are:

  • Latest trellis (1.20.1, or at least later than 1.18.0 that I first had)
  • Removing / commenting out vagrant-hostmanager-start entries on your Mac's /etc/hosts file
  • Updating the IPs in trellis/hosts/development. I'm not quite sure of this, but things started to work finally after I updated this one.
  • trellis vm start
  • Database migration / import in a way or another

Just wondering how to tell trellis to use vm over vagrant?

Just use trellis vm commands.

@artshostak
Copy link

Thanks @jkananen — that all totally makes sense!

So adjusting the /etc/hosts file and changing the trellis/hosts/development IP address (what did you adjust this to by the way?) and then using the vm commands is all that's needed? So having both, vagrant and vm, in place doesn't pose any interference? That's great if that's the case, let me know!

@jkananen
Copy link

Yes, just like that. I changed the last digit of the IP to one that's so far not been used.

@swalkinshaw
Copy link
Member Author

Thank you for the testing and feedback 😄

I'll write more documentation on this feature soon including details on how to migrate or use it concurrently with Vagrant since you might run into /etc/hosts conflicts as you noticed.

One quick clarification: in most situations, you shouldn't need to manually edit the hosts/development file as it's not used. trellis-cli generates and manages an inventory/hosts file in .trellis/lima (just like Vagrant does in .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory).

trellis provision has proper integration to know about those inventory files for example, but some other commands are still missing it which I'll work on improving. trellis alias for example will still use the wrong development host with Lima.

@swalkinshaw
Copy link
Member Author

Some replies to your questions @artshostak

❌ Not able to access the shell in development with trellis vm shell produces the following error:

Running command => limactl shell {project}
exit status 255

If you run into that again, I'd recommend running the lima command directly. eg: limactl shell --debug {project} and see if there's more output to help troubleshoot.

what happens if I have a project using Parallels and want to switch it over to VM midway for local development, or do you not recommend that? Just wondering how to tell trellis to use vm over vagrant?

As @jkananen said, right now trellis vm will only ever use the new VM feature with Lima. trellis up might be deprecated/removed at some point to reduce confusion (since vagrant up can easily be used; trellis up does nothing more than running that command).

if I wanted to specify my Ubuntu installation (20.04 instead of 22.04), when should I have done this prior to on a project level for trellis-cli config with the following:

vm:
  ubuntu: 20.04

Before running trellis vm start - or by re-creating a new one after trellis vm delete.

While it's blazing fast, it's unfortunately not so stable to do long sessions of local development with. Hope that helps and thanks again!

I wonder if all your stability/networking issues were related 🤔 I haven't heard of any issues like that yet. I'm wondering if a reboot helped (or would have)? If you run into that situation again, it would be good to confirm networking/DNS by continously ping {dev host} and see what comes back.

@jkananen
Copy link

One quick clarification: in most situations, you shouldn't need to manually edit the hosts/development file as it's not used.

Correct, it's not needed. It coincided earlier with rebooting my Mac, which seems to be necessary after the first trellis vm start run. This has been necessary now with all my three trellis vm setups, and the latest was otherwise very clear & straightforward.

Anyone else experiencing the need for a reboot?

@swalkinshaw
Copy link
Member Author

What makes you need to reboot?

I've had an issue before where it never got passed the SSH requirement and rebooting fixed it.

@jkananen
Copy link

Browser fails to load the page and finally gives up with "This site can't be reached / ERR_CONNECTION_REFUSED"

@swalkinshaw
Copy link
Member Author

@jkananen next time that happens it would be useful to see:

  1. you can shell into the VM
  2. ping the hostname locally to see what IP is resolved
  3. that /etc/hosts still looks correct

@Twansparant
Copy link

Twansparant commented Mar 16, 2023

I wonder if all your stability/networking issues were related 🤔 I haven't heard of any issues like that yet. I'm wondering if a reboot helped (or would have)? If you run into that situation again, it would be good to confirm networking/DNS by continously ping {dev host} and see what comes back.

I'm actually having similar connection issues, but I'm not using Lima yet, but still Vagrant & Parallels on a M1 machine.
At some point, my VM becomes unreachable and I get a ERR_HTTP2_PING_FAILED error.
I need to reboot the VM at that point and all is good again.

When I ping the hostname at that moment in the VM with ping example.test I get a normal response back:

...
64 bytes from example.test (127.0.2.1): icmp_seq=100 ttl=64 time=0.229 ms
64 bytes from example.test (127.0.2.1): icmp_seq=101 ttl=64 time=0.076 ms
64 bytes from example.test (127.0.2.1): icmp_seq=102 ttl=64 time=0.137 ms
64 bytes from example.test (127.0.2.1): icmp_seq=103 ttl=64 time=0.159 ms
...

My etc/hosts contains these lines at that moment:

## vagrant-hostmanager-start id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
192.168.56.5	example.test
192.168.56.5	www.example.test
## vagrant-hostmanager-end

10.211.55.16   	example.test.shared example.test #prl_hostonly shared

My ~/.ssh/config contains these lines at that moment:

Host example.test
  HostName 10.211.55.16
  User vagrant
  Port 22
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /path/to/my/project/trellis/.vagrant/machines/default/parallels/private_key
  IdentitiesOnly yes
  LogLevel FATAL
  ForwardAgent yes

I don't know if it's related?
I have the feeling it started after updating MacOS to Ventura.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants